If you want the bran, the endosperm and the germ, then you have to be willing to sacrifice on a whole host of flour applications. Another thing to note is that most flours are enriched post-processing, so have their chemical nutrients reinstated into the product, albeit not in natural form. But arguing against the health benefits of synthetic nutrients is a whole other ball-game altogether and is based on technical analysis of the metabolic rate of synthetic chemical variants of these nutrients. The flour manufacturers have done their homework here imo.
Whole-grains however will always be healthier for you than processed flour, but their real-world applications are sin-fully minimal by contrast. Your processed flour isn't killing you though, unless you're munching through an entire loaf a day. Everything in moderation I say. I repeat this every time someone decides to go on a diet that involves completely cutting something out - bananas are radioactive and in high enough doses will kill you. Eating a banana though won't kill you and they shouldn't be avoided altogether. Chocolate in high enough quantities will kill you. But that doesn't mean that you have to cut them out of your diet completely. Likewise, MSG in high enough doses will kill you, but that doesn't mean it must be avoided like the plague - in fact it is naturally occuring in your food probably each and every day without you even realising it.
Likewise, processed flour doesn't have to be avoided either. There are healthier alternatives for a small number of applications, but flour cannot be replicated in any other form outside of wheat-based-processed-flour...