I was debating between the s4 and the xperia z for a while. In the end I decided to go with the s4 but only because the contract comes with an additional 500mb data.
The phones for me both have their pros and cons. The s4 seems to have slightly better specs but the xperia looks beautiful, and is built strong. However, the amazing build quality comes with a small price to pay.. The display seems a little washed out and the battery is non-removable (for waterproofing reasons). Which is a real pity as the battery just doesn't seem good enough.. I recently dropped my lumia 920 and the screen shattered, so having a phone that's durable enough to withstand a drop or two is a huge feature! And only needing a screen protector for both sides of the phones beats any cover, which will just spoil it's amazing looks.
The s4 has more features but i feel that most of them are just gimmicks. The IR blaster is cool, as are the air gestures and such, but ultimately, they're not necessary. It's the newer generation processor, the better screen, and battery life that do matter.
Also, I'm not 100% sure, but a couple of guys told me that Samsung provides you with two no-questions-asked repairs on any sort of damage/flaw. That's a major game-changer.
Samsung do offer this. It may sound really cool, but the truth is that you only offer such a service if you need to offer such a service. All this tells me is that it's built poorly and prone to breaking.
Out of all the flagship phones, it really is built the worst. It's just a cheap plastic and it lacks that premium feel. Every other flagship phone, be it from Nokia, Sony, HTC, LG, Huawei, Apple, is built better than the s4. It's really the only thing letting the phone down.