But isn't it rare for a company to actually contact a previous employer to ask about performance reviews when hiring?

Common sense suggests that such a company wouldn't exist.
Does that actually happen?
I don't know.
In my opinion, I doubt there are many companies that would engage in such a pointless task.
If I were the hiring manager, I wouldn't feel the need to go that far;
it feels like it would hurt one's pride and come across as somewhat desperate.