Chris Auld has an excellent piece on his blog regarding interpreting the “competing” Seattle minimum wage studies from the University of Washington and UC Berkeley. It’s long, but very much worth the read. In fact, it’s probably the best short introduction to statistics/econometrics I think I’ve read (another great one is Chapter 1 of Robert Abelson’s Statistics as a Principled Argument. I’m also a big fan of Angrist & Pischke’s Mastering ‘Metrics).
Allow me to highlight two items in particular from this blog:
There is no statistical magic which can fully overcome these fundamental [causal] problems. We will never be able to “prove” what the effect of the minimum wage was: that’s not the way statistics work in general, and in a case study like `what was the effect of the 2015 increase in minimum wages on employment in Seattle?’ the best we can hope for is to bring some suggestive evidence to the table. [Emphasis added]
In other words, what they Berkeley team means when they report “no effect” on employment is not that there is no effect on employment (yes, that is confusing). What they mean, again, is that there is no statistically significant effect on employment, whereas the UW team, using different data and somewhat different statistical methods, finds a statistically significant effect. But the difference between statistically significant and statistically insignificant is often itself not statistically significant.
One team found there were no statistically significant effects on employment, but that result should not be misunderstood as a claim that the study “proves” the effect was actually zero… [original emphasis]
Any additional commentary I add here will only detract. Read Dr. Auld’s post. It’s excellent stuff.
H/T: Michael Enz