Peter

In the US, at least, calling a man a "boy" can have some pretty strong racial connotations.

Calling a woman a "girl" doesn't, but is generally insulting, especially since it tends to get applied to women such that a man the same age would never be called a "boy" (except with the abovementioned racist connotations).