Indeed but it does not mean the conclusion can't be made.Concluding that something is designed is very different from concluding that something we know is designed may have a common origin.
If you come across a house you won't conclude it wasn't designed because you have no knowledge about the designer. By examining the house you'll gain knowledge about who designed it.Indeed I'd venture it may be impossible to decern design without having some knowledge of the designer.