I personally had only x86 version, but my friend (sysadmin) said he used x64 Office on x64 Windows and office had problem, so he reinstalled on x86 office. Don't you think it's good sign that x64 is "not ready yet"? (note that it's the same vendor!)
Many other companies say "no x64 version yet, use x86" - is it so hard for 'em to recompile? Of course not! Problem is code itself, which have to be written VERY careful to be "recompillable" for x64.
As a programmer I had issue with .NET: I have x64 add-in (x86 tested as well) on x86 Office uder x64 Windows with x64 .NET; System didn't work at all, while full x86 line worked perfectly. In FOSS world also a lot of issues recompiling programs for x64 - they crash due to careless code.
All of that is not enough? OK, everybody is free to select his own rakes!

I just warned YingYang that if don't want to risk, use x86.
PS
From now I'll specially concern about x64 issues and post here, if you wish. But I think it's enough to google (and read not only the first link, huh?)