You don't have to factory reset if you don't want. It's definitely not a bad thing if you do.
Well it is a bit of a pain, but I thought moving from one level to the next, i.e. Jelly Bean to Kit Kat or now Kit Kat to Lollipop, was a good thang. Start afresh with app data and all.
So might as well. I've got nothing else to do except be cold.