Then it comes down to GPU performance, since they use the same CPU's.
To prove just how capable the X1 can be, Huang showed the Unreal Engine 4 "Elemental" demo running off of the chip (it's unclear if it was in real time or just a recording). That's a graphically complex 3D scene that's been used to show off the rising computational capability of video cards and consoles over the past few years. While the demo didn't look perfect on the X1 -- it wasn't as detailed as I remember it; there weren't as many particle effects; and there was quite a bit of slowdown -- it was still impressive for something mobile.
Of course, since the X1 is something meant for smartphones and tablets, power efficiency is key. Huang pointed out that it was able to run the Elemental demo using just 10 watts of power. A year ago, the Xbox One needed 100 watts to run Elemental. And over two years ago, a leading Nvidia GPU had to burn 300 watts to run it. You won't be stressing your phone with complicated 3D rendering all of the time, but the X1's performance in this demo is a good sign that it could be more efficient at more mundane mobile tasks.