None of the other stock Android apps (including the stock browser) have an exit button; why would Chrome be any different? Also, Google has always said that people should not use task-killers and has resisted the idea of putting in task-killing functionality (for example, swiping tasks away in ICS simply removes them from the list, it doesn't actually kill the task). The point of Android is that you're supposed to let it manage your memory automatically, and generally I've had good results doing that.
As for Chrome, I tried it out yesterday, and I honestly think it has the best UI of any mobile browser I have ever seen. The tabs-as-a-deck-of-cards metaphor works incredibly well on a smartphone-sized screen, and the integration with the desktop browser is fantastic. That said, I also found it to be buggy and unstable. The fact that I was running it on a custom ROM using hacked GPU drivers might have had something to do with this.
Ultimately, I uninstalled Chrome and rolled back my phone to a Gingerbread ROM. GB isn't as pretty, but on my phone it feels faster, more stable and more mature than any ICS ROM I've tried. However, when I upgrade to a decent ICS phone (Galaxy Nexus!), I look forward to using Chrome as my default browser.