Quite a bit (gonna make this brief)... First of all 32bit computers have certian limit on the amount of memory they can access or rather use at one time. There are more but thats the most prominent difference. Now 64bit can access far more amounts of data.
For all the advantages of 64bit you MUST run a 64bit operating system.
Now a lot of 32bit OS's tend to only be able to use 3.5GB at most, sure there are exceptions and one can apply long workarounds but not always affective. XP 32bit tends to have this problem and so does Vista 32bit.
Windows XP 32bit kernal system is build 2600+ (SP2 if you have installed it)
However Windows XP x64 bit edition (which can process far more than 4GB RAM, I think it can go up to 128GB not sure though) is actuley build 3900. Which is exactly the same as Windows Server 2003.
In other words Xp x64 is a stripped down version of Windows Server 2003, thats why often drivers will work for both OS's etc. Due to it being 64bit and in essence a different a kernal to regular XP, special drivers are needed for the hardware to run.
If you have a 64bit PC with 4GB Ram, all the drivers for a 64bit OS,, (and of cause the software runs on it); you will have a system thats far more stable than its 32bit counterpart.