Did you check for non-linerity? Maybe the pattern in the ends (tales of distribution) is different than in the average. Maybe his hands are even smaller than expected (or the opposite). This is, perhaps the taller men have a different pattern that cannot be detected analizing the hole sample… Just a comment to go on with this interesting study!