Here's an intriguing study that explains why female voices are heard more clearly and provides an excuse for why some men tire of hearing women speak (yeah, right; I think there are other factors involved! just showing my female bias here). It's not so much a matter of pitch, but the greater complexity of female voices -- a product of the number and vibration of sound waves.
The imaging study compared the activity in men's brains when they heard male and female voices (no, I wouldn't conclude anything about what's happening in women's brains from this study). They found that voices perceived as female activated the auditory region (logical enough), but voices perceived as male activated another area, where people compare other individuals and things to themselves. The study was actually looking at auditory hallucinations in schizophrenics, and it's worth noting that the voices schizophrenics hear tend to be male; the researchers suggest that it is easier to create a false voice in the "mind's eye" region where male voices are processed.