Thinking Critically About Men Driving Lost
A recent filler story running through the media is the survey that reports men drive (on average) 276 lost miles per year while women drive “only” 256. This is based on a study by Sheila’s Wheels (an insurance company for women).
In an interesting coincidence, I also happened to be teaching about such inductive generalizations in my critical thinking class when the story was making the rounds and I think that some of what I teach in that section in well worth repeating here.
The first thing to keep in mind is that such studies/surveys are inductive generalizations. The basic idea is that a sample is surveyed or studied and the results are then extended (inductively) to the whole population. In this case, the sample consisted of the drivers studied/surveyed and the target population is the general population of drivers. Since this is inductive reasoning, even if the information acquired from the sample is accurate, extending this to the whole population is not without risk. To be specific, the sample data could be completely correct, but the sample might not be an accurate representation of the whole sample (that is, the sample might be biased). To avoid such bias, careful thinkers take steps to ensure that the sample is large enough and diverse enough to provide a proper foundation for the inductive leap from sample to target. Failure to do this can result in fallacious reasoning, specifically a hasty generalization or a biased generalization.
Even if a sample is properly taken, a sample that is smaller than the whole population will almost certainly differ from the population to a degree. Because of this people make use of margins of error. This is, roughly put, a percentage by which the sample is supposed to differ from the population due to sampling errors. In general terms, the margin of error is based largely on the size of the sample relative to the population. For example, in a large population (10,000+) a sample of 10 will yield a margin of error of +/- 30%. The “gold” standard for professional surveys/studies is getting to a margin of error of +/- 3%. This requires a sample of 1,000. Getting to a margin of error of +/- 2% requires increasing the sample size to 1,500. Given the cost and effort required to take effective samples, it is no surprise that getting to that margin of error is considered accurate enough. Naturally, folks who are hard core statistics nerds will be aware that I am oversimplifying things a bit,
The study cited by Sheila’s Wheels had a sample population of 1,009 people. Keeping things simple, this would give the results a margin of error of +/- 3%. However, this is not a simple situation because there are actually two distinct target populations: male drivers and female drivers. Assuming the sample was split evenly by gender, this would mean that the sample size for each gender would be about 500 people. As such, the result that men drive 276 lost miles would have a margin of error of +/- 4%. Likewise for the result that women drive 256 lost miles. This means that the actual lost miles could be 4% less for men and 4% more for women. As such, it could easily be the case that men do not drive more lost miles than women.
Given the margin of error and the relatively small difference between the lost miles in the sample, it seem unreasonable to conclude that men certainly drive more lost miles than women. It is also unreasonable to make a “big deal” about these findings because they simply fail to show a significant difference that would warrant the sort of stereotypical claims that have been made by folks in the media.
Of course, I am also motivated to be critical of this survey because it struck me that it was being employed in some minor “male bashing” in the media. I imagine that the feeling I get when I see female media folks chuckling over these results and stereotyping men is somewhat like what women feel when they are treated in a similar manner. Not surprisingly, I would prefer to see less such stereotyping and bashing.