First question is, as mentioned above, what software are you using? And what types of VM disks?
Host Defrag - If you are using a type 2 hypervisor and assign all the disk space to the virtual disk, then it cannot fragment and therefore does not need to be defrag'd (as with any large file created on continuous disk space). If your disk has been set to grow, then the performance is probably lousy in the first place, and defrag will not speed things up much. In fact, defraging the VM allows the disk to shrink, which will result in a performance decrease when the disk is forced to grow again. A type 1 VM also cannot fragment so there is no need to defrag here.
Guest - Defrag only improves performance in very, very few instances. It is very simple to prove and quantify. The virtual guests would benefit the same as a physical machine would from defrag; again mostly it will not improve performance. Be careful if you are using snaps, or shared storage. Defrag is very IO intensive and will reduce performance on shared storage. It will also make the snaps grow....
In short, the benefits of defrag are 99% myth, the "proof" of the benefits created in a lab under ideal conditions. Defrag is a total waste of time....