How many business leaders would waste money searching for new information that they know will not affect their decision making? Politicians do it all the time. They use taxpayer money to fund studies they hope will be ammunition for positions they already hold. If the results don’t come out the “right” way, they simply ignore the study and continue advocating the same position.
A particularly egregious case came out of Europe last week, as reported by Science Insider. The Netherlands Organization for Scientific Research (NWO), a quasi-governmental agency that dispenses academic grants, commissioned a study to investigate whether it was treating female researchers fairly. The results of the study initially suggested a bias against women: While 17.7 percent of male applicants received a grant, just 14.9 percent of female applicants received one. In response, the NWO promised to overhaul its selection system, including re-training its evaluators and eliminating the use of gender-biased words such as challenging and excellent. (Seriously.)
But it turns out that women tend to apply for grants in subjects with lower acceptance rates. Once broken down by subject area, there were actually no significant differences in acceptance rates for men and women. In other words, there was no evidence of gender bias. So did the NWO cancel its overhaul? No. According to Science Insider, “The organization will go ahead with its reforms because they’re a good idea, regardless of the paper’s quality.”
The NWO’s disregard of the study it paid for led science-blogger Scott Alexander to ask: “Why are we even bothering to do science anymore? Why don’t we just write the only acceptable conclusion on a piece of paper beforehand and save however much it cost to do the study?”
Exactly. In my view, whenever taxpayer money is spent to evaluate a particular policy, politicians should explain in advance how the study’s results will inform their thinking. If the answer is, “My opinion will not change regardless of the results,” then don’t fund the study in the first place! Better yet, we should tie funding (or reforms) of social programs directly to the evaluation results. That will at least generate an honest conversation about what we expect to achieve with public policy.