If I enter 196.23.168.144 in my browser, it takes me to google, not youtube.
It's supposed to. Google owns YouTube. It detects what you entered as the url (youtube.com or www.youtube.com) then takes you to the correct data on the cache server. I.e. it only works through the hosts.