All entries for Friday 23 September 2016

September 23, 2016

Radio 4 does statistical significance

There was an item on “Today” on Radio 4 on 22 September about Family Drug and Alcohol Courts – which essentially are a different type of court system for dealing with issues about the care of children in families affected by drugs and alcohol. I know nothing about the topic, but it seems they offer a much more supportive approach and are claimed to be more successful at keeping parents off drugs and alcohol and reducing disruption to family life.

This item featured an interview with one of the authors, Mary Ryan, of a new report comparing the effectiveness of Family Drug and Alcohol Courts with the standard system: keeping children with their parents, and keeping parents off drugs and alcohol. Twice she said that differences they found were “statistically significant”, emphasising the “statistically”, and the phrase was also repeated by the Radio 4 presenter.

I would be pretty confident that the presenter, almost all of the audience, and very possibly Mary Ryan, have no idea what the technical meaning of “statistically significant” is. But the words have everyday meanings that we understand, and when put together they sound as though a result must be important, impressive and reliable. It’s “significant” – that means it’s important, right? And it’s not just ordinary significance, but “statistical” significance – that means that it’s backed up by statistics, which is science, so we can be sure it’s true.

I don’t know for sure, but I would guess that this is the sort of understanding that most people would take from a discussion on Radio 4 of “statistically significant” results. It’s a problem of using familiar words to refer to specific technical concepts; people can understand the words without understanding the concept.

Just after writing this I came across this blog post from Alex Etz which confirms what I thought, with numbers and everything:

September 2016

Mo Tu We Th Fr Sa Su
Aug |  Today  | Oct
         1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30      

Search this blog



Most recent comments

  • Hi Tom Sorry for delay in replying – taken out by family issues then holiday for the last month or s… by Simon Gates on this entry
  • Simon, I can see where you're coming from on this. If MCID (in its various guises) is not an optimal… by Chee-Wee Tan on this entry
  • Hi Simon I am currently doing my PhD in clinical based research. We want to use the MCID to determin… by tomwilks on this entry
  • I think your comment reveals how nonsensical null hypothesis testing is (and I see from your other p… by matt on this entry
  • Thanks for commenting Matt – I do wonder if anyone ever looks at any of this, not that this is a pro… by Simon Gates on this entry

Blog archive

Not signed in
Sign in

Powered by BlogBuilder