A statistical test is a procedure for deciding whether an assertion ( e.g. an Hypothesis ) about a quantitative feature of a population is true or false. We test an hypothesis of this sort by drawing a random sample from the population in question and calculating an appropriate statistic on its items. If, in doing so, we obtain a value of the statistic that would occure rarely when the hypothesis is true, we would have reason to reject the hypothesis.

With this procedure it is customary to reject the hypothesis tested when our statistic has a value that is among those that, theoretically, would be expected to occure no more than 5 out of every 100 times that a random sample (of the same size) is drawn from the population in question when the hypothesis is, in fact, true. Much of the text of this tutorial is devoted to explanations of exactly how this kind of theoretical expectation is developed.

Finally, it is noteworthy that the appropriate conduct of any statistical test invariably requires many thoughtful decisions. It is, for example, always necessary to decide what statistic to use, what sample size to employ and what criteria to establish for rejection of the hypothesis tested.