Monday, August 16, 2021
Not blinded by stats!
Here's a refreshingly HARDASS critique of the FRAUD called statistics, in a mainstream science mag. Bruce Bower tells the history of the 'p-value' that allegedly measures significance. Unsurprisingly it began shortly after WW2, like all the other frauds and crimes of science.
= = = = =
Calibrating: I played the p-value game once. I wrote some rather neat software for one of the speech grad students at Penn State. She wanted to try computers in teaching foreign languages. (This was 1988; sound handling was still a new trick and an expensive option on computers.) She used the software with a fairly large number of students, and taught the same words the old-fashioned way with a control group. She got ambiguous results, which isn't surprising. Tech is NOT magic. The old-fashioned way is often better! As the deadline for her thesis got closer, she wondered if I could use a neural net to reprocess the results. I did, and the significance looked better that way.
I'm proud of making the sample-and-compare software, which was original at the time. I'm not proud of playing the p game.
Here's a refreshingly HARDASS critique of the FRAUD called statistics, in a mainstream science mag. Bruce Bower tells the history of the 'p-value' that allegedly measures significance. Unsurprisingly it began shortly after WW2, like all the other frauds and crimes of science.
By calculating just one number from their experimental results, called a P value, researchers could now deem those results “statistically significant.” That was all it took to claim — even if mistakenly — that an interesting and powerful effect had been demonstrated. The idea took off, and soon legions of researchers were reporting statistically significant results.Bower states the solution clearly:
Keep it simple, Loftus advised. Remember that a picture is worth a thousand reckonings of statistical significance. In that spirit, he recommended reporting straightforward averages to compare groups of volunteers in a psychology experiment. Graphs could show whether individuals’ scores covered a broad range or clumped around the average, enabling a calculation of whether the average score would likely change a little or a lot in a repeat study. In this way, researchers could evaluate, say, whether volunteers scored better on a difficult math test if first allowed to write about their thoughts and feelings for 10 minutes, versus sitting quietly for 10 minutes.Graphs, graphs, graphs. Let readers SEE what's there. Don't smash the graphs into one number that collapses the time axis.
= = = = =
Calibrating: I played the p-value game once. I wrote some rather neat software for one of the speech grad students at Penn State. She wanted to try computers in teaching foreign languages. (This was 1988; sound handling was still a new trick and an expensive option on computers.) She used the software with a fairly large number of students, and taught the same words the old-fashioned way with a control group. She got ambiguous results, which isn't surprising. Tech is NOT magic. The old-fashioned way is often better! As the deadline for her thesis got closer, she wondered if I could use a neural net to reprocess the results. I did, and the significance looked better that way.
I'm proud of making the sample-and-compare software, which was original at the time. I'm not proud of playing the p game.
Labels: Blinded by Stats, Carbon Cult, Carver
¶ 2:04 PM
