re: "Half the schools are below average" - not always true

quote:

---

Apology in advance for any grammar/spelling errors; I am on my phone.

As cptbengal said this is a nested design. Therefore one would use heirachical-linear modeling (goes by a number of names) to account for the between-school differences (or between-classroom differences as well for a three-level design) by allowing students' to vary within the nested units(can allow slopes, intercepts, or both to randomly vary). These analyses are frequently used.

In addition, while it is true in your small sample example that only 25% were below average, as has been pointed out, with a large enough sample, the mean will be approximately normally distributed, regardless of the distribution; you can downplay it but it is an important component to the argument.

I'm addition, although you are right that many variables do not have an underlying normal distribution, that is an irrelevant argument in this case due to the variables of measurment. In particular, because the tests are attempting to measure a latent construct (e.g., math achievement) that cannot be directly observed, they have to create and norm items and scales then psychometrically evaluate them (individually items are usually based on item-response theory, then factor analysis for larger constructs). They then ensure that the tests are normally distributed. Therefore, the underlying distribution is normal and the law of large numbers will enable a large sample to approximate to its expected value (the mean of that normal distribution).

Basically your whole premise is fundamentally flawed. In addition, although I am no expert on statistics, I know enough to believe that you don't know as much as you seem to think.

---