Phi Beta Cons

Does Academic Work Need More Scrutiny?

I don’t know, but there are two new examples of papers that could have benefited from it.

Immigration economist George Borjas tried to replicate the results of another study, but found he was unable to do it. The problem with the original, it turns out:

The [study’s] data includes currently enrolled high school juniors and seniors. They classify these high school juniors and seniors as part of the “high school dropout” workforce. Their finding [that immigration increases wages for many natives] disappears if the analysis excludes these high school juniors and seniors.

And via John Lott, regarding a famous Journal of Political Economy paper (PDF) finding that illegal downloading doesn’t hurt music sales, comes this Craig Newmark post.
Apparently, the paper was based on data provided by the file-sharing services “MixmasterFlame” and “FlameNap.” This is already a conflict of interest and, what’s more, the agreement through which the researchers procured the data forbade them to share it – they could analyze it themselves for the paper, but no one could double-check their work or run different analyses.
(Fortunately, it seems that since the paper was accepted for publication, JPE has made binding a rule that requires authors to publish data and methods along with results.)
I think both cases stem from a drive for attention-getting results: The more “counterintuitive” a finding is, the more likely journal editors are to see it as “too good to check.” Both of these studies, if true, would basically have repealed the laws of supply and demand – increased competition for jobs increases wages; making something available free won’t affect how many people choose to pay for it.


The Latest