We all know that statistics can lie and confuse. We can easily confuse ourselves, in fact -- if we don't think very carefully about what a statistic is measuring, and what we want to know. A subtle mismatch between these can cause "bugs" in our calculations and our thinking. Here is one interesting sort of statistical bug, based on very simple math.
Instance 1: The Average Class Size
A college boasts in its advertising materials that its average class size is 25. Curious, we look into it further. We find that there are three types of class run every semester:
Stadium lectures: 200 students each - x6 Lectures: 25 students each - x20 Recitations: 15 students each - x100
Just to check their math: there are 3,200 enrollments in 126 classes, which is indeed a mean of 25.4 students per class. (Of course the college rounds down.)
This measure is useful to the college: in planning schedules and capacities, it is convenient to consider the theoretical, if all classes were the same size, this is how big they would be, which is the mean size.
But this is not useful to students, who care about the classroom sizes they peronsally experience. This is different from the general case because the number of student experiences is not the same as the number of classrooms: far more people will sit in a stadium lecture than in a recitation; and therefore the stadium experience -- of having 199 fellow students -- is much more common.
The solution is to take an average not of the classes, but of the students' experiences in terms of classmates. This requires multiplying class size by the enrollment in that catregory:
Stadium lectures: 1200 enrollments x200 classmates Normal lectures: 500 enrollments x25 classmates Recitations: 1500 enrollments x15 classmates
Even before we perform the calculation, it is obvious how little those small recitations will really count. Calculating this way, there are 275,000 classmate experiences over 3200 student enrollments -- which yields an average of 85, more than three times what was reported by the school.
Each of these two weighted averages say something -- but not the same thing. We must be careful to get what we hope we're getting.
Instance 2: Average Speed
The same "bug" is used to trip up high-school students in a well-known class of math problem.
A car is taken on a 180-mile road trip and driven at two different speeds: on the way there, 60 mph on the highway; on the way back, 30 mph on country roads. What is the average speed?
The bug occurs when students take the easy way out and average the two speeds to get 45 mph. This is incorrect because it calculates the average speed per mile traveled, when what we really want is the average speed per hour spent: that is, if you spread the driving out equally over the time it actually took, how fast would the car be going?
The trap answer is made more appealing by being easier to compute: the mileages are the same, so a simple (non-weighted) average is all that's needed. This result could be useful, of course, in answering some other question, perhaps related to fuel efficiency or tire wear (which are functions of distance first and foremost) but it's different from what's asked for.
To compute the desired answer, we must again view the correct slice of the data, and then weight our cases accordingly. It is trickier here because a key variable, hours, is implicit in the units of the data provided, and we must extract it before proceding.
Leg 1: 60 mph - 180 mi - 3 hours Leg 2: 30 mph - 180 mi - 6 hours
The total distance is 360 miles, traveled in 9 hours. Computing the overall speed from these numbers produces the right kind of average -- because it pretends that the trip was uniform, from the perspective of each hour spent. We thus get an answer of 40 mph. We should be reassured that this is lower than the naively computed 45 mph because the passengers have correctly spent twice as long on the slower return trip.
In both of these cases, the "bug" is to assume that only one kind of average exists for a given set of data; and that whatever is easiest, or most obvious, or already done must be that correct way. The reality is that there are many ways to group data, and therefore many ways to produce an average.
Think carefully about what items go in the numerator and the denominator: what measure are you "spreading out evenly" across all cases? Are these really the cases you're interested in? Often, the real case is hidden, or implicit. But by finding the right frame, we can get correct answers (as with the speed example) and startling results (as with the class sizes).