Why does we make statistics so hard for our students?

Why does we make statistics so hard for our students?

(Warning:long and slightly wonkish)

If you ' re like me, you're continually frustrated by the fact that undergraduate students struggle to understand statistics . Actually, that's putting it mildly:a large fraction of undergraduates simply refuse to understand statistics; Mention a requirement for statistical data analysis in your course and you ' ll get eye-rolling, groans, or (if it's early E Nough in the semester) a rash of course-dropping.

This bothers me, because we can ' t does inference in the science without statistics*. Why is students so unreceptive to something so important? In unguarded moments, I ' ve blamed it on the students themselves for have decided,  a priori  and in a s Elf-fulfilling prophecy, that statistics was math, and they can ' t do math. I ' ve blamed it on high-school math teachers for making math dull. I ' ve blamed it on high-school guidance counselors for telling students so if they don ' t like math, they should become bi Ology majors. I ' ve blamed it on parents for allowing their kids to dislike math. I ' ve even blamed it on the boogie**.

All these parties (except the boogie) is guilty. But I ' ve come to understand that my list left out the most guilty party of All:us. By "US" I mean University faculty The WHO teach Statistics–whether they ' re in departments of Mathematics, Department S of Statistics, or (gasp) departments of Biology. We make statistics needlessly difficult to our students, and I don ' t understand why.

The problem is captured with the image above–the formulas needed to calculate Welch ' s  t -test. They ' re arithmetically a bit complicated, and they ' re used in one particular situation:comparing both means when sample Si Zes and variances are unequal. If you want to compare three means, you need a different set of formulas; If you want to test for a non-zero slope, you need another set again; If you want to compare success rates in binary trials, another set still; And so on. And each set of formulas works is given the correctness of its own particular set of assumptions, about the data.

Given This, can we blame students for thinking statistics are complicated? No, we can ' t; But we can blame ourselves for letting them think the it is. They think so because we consistently underemphasize the "most important thing" about statistics:that this Complicat Ion is an illusion. In fact, every significance test works exactly the same.

Every significance test works exactly the same. We should teach this first, teach it often, and teach it loudly; But we don ' t. Instead, we make a huge mistake:we whiz by it and begin teaching test after test, bombarding students with derivations of Test statistics and distributions and paying more attention to differences among tests than to their crucial, underlying Identity. No wonder students resent statistics.

What does I mean by "every significance test works exactly the same"? All (NHST) statistical tests respond to one problem with the simple steps.

The problem:

• We see apparent pattern, but we aren ' t sure if we should believe it's real, because our data is noisy.

The steps:

• Step 1. Measure the strength of pattern in our data.
• Step 2. Ask ourselves, is this the pattern strong enough to be believed?

Teaching the problem motivates the use of statistics in the first place (many math-taught courses, and nearly all Biolo Gy-taught ones, do a good job of this). Teaching the steps gives students the tools to test  any  hypothesis–understanding that it ' s just a Matter of choosing the right arithmetic for their particular data. This is the where we seem to fall down.

Step 1, of course, is the test statistic. Our job was to find (or invent) a number, measures the strength of any given pattern. It's not surprising so the details of computing such a number depend on the pattern we want to measure (difference in TW o means, slope of a line, whatever). But those details always involve the three things, we intuitively understand to being part of a pattern ' s "strength" (ill Ustrated below): The raw size of the apparent effect (in Welch ' s  t , the difference in the and the other means); The amount of noise in the data (in Welch ' s  t , the both sample standard deviations), and the amount of data In hand (in Welch ' s  t , the both of the sample sizes). You can see by inspection this these behave in the Welch ' s formulas just the the-they should:  T  gets Bi Gger If the means is farther apart, the samples is less noisy, and/or the sample sizes is larger. All the rest is uninteresting arithmetical detail.

Step 2 is the p-value. We have to obtain a p-value corresponding to our test statistic, which means knowing whether assumptions is met (so we CA n Use a lookup table) or not (so we should use randomization or switch to a different test***). Every test uses a different table–but all the tables-work the same-a-to, so the differences is again just arithmetic. Interpreting the P-value once we have it's a snap, because it doesn ' t matter what arithmetic we did along the way:the p Value for any test are the probability of a pattern as strong as ours (or stronger), in the absence of any true underlying Effect. If This is low, we ' d rather believe, we have pattern arose from real biology than believe it arose from a staggering coinc Idence (Deborah Mayo explains the philosophy behind this here, or see her excellent blog).

Of course, there is lots of details in the differences among tests. These matter, but they matter in a second-order way:until we understand the underlying identity of how every test works, There ' s no point worrying about the differences. And even then, the differences is not things we need to remember; They ' re things we need to know to look up when needed. That's why if I know how to does one statistical test– any one statistical test–i know what does all of them.

Does this mean I ' m advocating teaching "cookbook" Statistics? Yes, but only if we use the metaphor carefully and not pejoratively. A Cookbook is the little use to someone who knows nothing at all about cooking; But if you know a handful of basic principles, a cookbook guides you through thousands of cooking situations, for differen T ingredients and different goals. All cooks own cookbooks; Few memorize them.

So if we ' re teaching statistics all wrong, here's how to do it right:organize everything around the underlying I Dentity. Start with it, spend lots of time in it, and illustrate it with one test (no test) worked through with detailed attention The "not to the computations," but "to", "Test takes us through the" steps. Don ' t try to cover the "8 tests every undergraduate should know"; There ' s no such list. Offer a statistical problem:some real data and a pattern, and ask the students what they might design a test to address th At problem. There won ' is one right-and even if there was, it would is less important than the exercise of thinking through the Steps of the underlying identity.

Finally: Why does instructors make statistics on the differences, not the underlying identity? I said I don ' t know, but I can speculate.

When statistics was taught by mathematicians, I can see the temptation. in  Mathematical  terms, the differences between tests is the interesting part. This is where mathematicians show their chops, and it's where they do the difficult and important job of inventing new rec Ipes to cook reliable results from new ingredients in new situations.  Users  of statistics, though, Woul D is happy to stipulate that mathematicians has been clever, and that we ' re all grateful to them, so we can get onto the Job of doing the statistics we need to do.

When statistics was taught by biologists, the mystery was deeper. I think (i hope!) those of us who teach statistics all understand the underlying identity of all tests, but that doesn ' t s Eem to stop us from the parade-of-tests approach. One hypothesis:we may is responding to pressure (perceived or real) from mathematics departments, who can disapprove of s Tatistics being taught outside their units and was quick to claim insufficient mathematical rigour when it was. Focus on lots of mathematical detail gives a veneer of apparent rigour. I ' m not sure that my hypothesis was correct, but I ' ve certainly been part of discussions with Math departments that were CO Nsistent with it.

Whatever the reasons, we ' re doing real damage to our students when we make statistics complicated. It isn ' t. Remember, every statistical test works exactly the same. Teach a student that today.

Note:for A rather different take in the cookbook-stats metaphor, see Joan Strassmann ' s interesting post Here . I think I agree with she only in part, so you should read her piece too.

Another related piece by Christie Bahlai are here: "Hey, let's all just relax about statistics" –but with a broader MESSAG e about nhst across fields.

Finally, here's the story of the both ecologists who learned to love Statistics–and it's lots of fun.

©stephen Heard ([email protected]) October 6

*in this post I ' m going to discuss frequentist inferential statistics, or traditional "null-hypothesis significance testin G ". I ' ll leave aside debates about whether Bayesian methods is superior and whether p-values get misapplied (see my Defence O f the P-value). I ' m going to refrain from snorting derisively at claims so we don ' t need inferential statistics at all.

**ok, not really, but slipping, the there lets me link to this. Similarly I ' m tempted to blame it in the rain, to blame it on Cain, to blame it on the Bossa Nova, and to blame it on Rio. OK, I ll stop now; But if you've got one I missed, why isn't drop a link in the replies?

I ' d include transforming the data as "switch to a different test", and if you ' d rather draw a distinction there, which ' s Fine.

Why does we make statistics so hard for our students?