Refs tend to get irritated by some players ,and the build up of tackles and dirty play during a match can further influence a decision. In isolation the Vidic decision was harsh and perhaps warranted a yellow, however it was still a dangerous tackle and could have caused serious injury.
Had the ref seen Rafael's tackle properly ,it was going to be a straight red...doubt he or his assistant caught a good view of the incident .
Actually if you look at it, Vidic only clipped Hazard - Hazard didn't even make much of a fuss - knowing full well at most it's a yellow card... How's Luiz's comment when he saw the red card... coming from the dirtiest player on the pitch...

