Wednesday, July 17, 2013
Genuinely puzzled.
Each day thousands of "social science" studies get published in various journals. Maybe a half dozen of these studies make it into the pop-sci media or the general news each day. EVERY ONE of these studies is invalid. Zero exceptions. Some are purely counterfactual, operating on the nonsensical assumption that humans are identical. Most start with a good and unbiased observation of facts but violate the basic rules of causation.
Today's classic example: "Retiring early increases your chance of Alzheimers."
I don't have any reason to question their data gathering; probably careful and solid. But you simply CAN'T draw the causative inference unless you've done a randomized experiment. It's possible that some people keep their minds lively by working longer. It's also possible that some people sense their cognition is slowing down ... or their employer senses loss of performance ... so they retire early, voluntarily or forcibly. Most likely the situation is half and half. You can't pick either possibility without a randomized experiment, so the whole result is utterly meaningless as personal or policy guidance. It's just a tautology.
Here's what puzzles me: Bad causation is easy to spot, and lots of people can spot it. Even local TV newscasters, who are generally uninformed and shallow, often point out the problem.
If local TV newscasters can see the failure, why can't the researchers see the failure? Or their journal editors?
¶ 11:51 AM 
