I am a self-confessed Google fanboy, but after using my girlfriend's SE X10 for about 6 months I've switched to an iPhone 4. I have 3 main problems with Android:
1) The fragmentation is a huge problem. To find out your 6-month old phone will not receive any future Android updates is hugely frustrating. And even if you buy the latest phone now, when Google release the next version of Android (ice-cream?), how long will it take before you get it? Maybe if you've got a "pure" Android phone like the Nexus S, but what if you've got a Desire S? You'll have to wait until HTC update Sense to the new version, and then wait for Vodacom or whoever to push it out locally. Google can fix this by taking control of Android, but until they do, there will be lots of frustrated users. With iOS I get access to the latest version at the same time as everyone else around the world. And no Vodacom bloatware like The Grid and Vodacom Live!
2) iTunes is far from perfect, but its better than no-tunes from Google! To manage the media on my Android phone, I was forced to use DoubleTwist, which just creates a new interface for iTunes. Where is Google's interface? I have 60GB of music on my PC, and with a few simple playlists iTunes automatically refreshes my music every time I connect my iPhone. I also battled to get my Android phone to recognise the difference between music, podcasts and audiobooks - all MP3 files show up as music!
3) The battery life on every Android phone I have encountered sucks. The Desire HD is a beautiful phone, but you have to switch off the cool features like live wallpaper, and you have to toggle bluetooth and wi-fi on and off as required if you want your phone to last through a normal day. I have bluetooth and wi-fi switched on permanently on my iPhone, and the battery is seldom below 50% at the end of the day.
There are features of Android that I miss. The native Google apps are (mostly) better on Android, but all my Google contacts and calendars sync up via exchange, and the Google Mobile App for iPhone is awesome! I loved being able to select apps on my PC and have them load automatically on the phone, but that's hardly a deal breaker! The way Android handles notifications is better than iOS, but I'm hoping that'll be fixed with iOS 5.
I am sure Google will fix the above problems, and in 2 years time when I'm due for a replacement phone I may give Android another chance, but right now I believe that the iPhone offers the best overall package!