Tuesday, November 20, 2018
Self-destructing "science"
I suppose it's sort of vaguely advantageous that social "science" is trying to reform itself. But even within the context of the reform, the academics are still missing all the important points.
From a solid fair-minded article in Atlantic:
Despite the large sample sizes and the blessings of the original teams, the team failed to replicate half of the studies it focused on. It couldn’t, for example, show that people subconsciously exposed to the concept of heat were more likely to believe in global warming, or that moral transgressions create a need for physical cleanliness in the style of Lady Macbeth, or that people who grow up with more siblings are more altruistic. And as in previous big projects, online bettors were surprisingly good at predicting beforehand which studies would ultimately replicate. Somehow, they could intuit which studies were reliable.Well then, you should be paying the online bettors instead of paying the academics.
Sanjay Srivastava from the University of Oregon says the lack of variation in Many Labs 2 is actually a positive thing. Sure, it suggests that the large number of failed replications really might be due to sloppy science. But it also hints that the fundamental business of psychology—creating careful lab experiments to study the tricky, slippery, complicated world of the human mind—works pretty well. “Outside the lab, real-world phenomena can and probably do vary by context,” he says. “But within our carefully designed studies and experiments, the results are not chaotic or unpredictable. That means we can do valid social-science research.”No. In fact the results within the study context are uniformly random, predictably uncorrelated to the academic expectations. That's what the big replication test tells us. In other words, the big study has proved decisively that you CANNOT do valid research within the usual academic context. The more research you do by this method, the more you disprove your own research by this method. Outside of academia there's plenty of valid and useful analysis of human behavior, based on REAL experimentation with REAL consequences. Those bettors are obviously doing a valid analysis with their own methods. Successful salesmen and scammers and editors and design selectors and hit-pickers have been analyzing human behavior in great detail for a long time, with valid results. Cult leaders are the best of all. Jesus and Hitler and Hubbard understood human behavior better than a million academics. ¶ 12:10 AM
