There are two seemingly unrelated stories that I’d like to take a moment to compare in order to make a point.
In the world of economics, public policy, and how not to format a spreadsheet, word has come out that a seminal paper arguing that high debt-to-GDP rates are bad for economic growth was based on bad data after the two professors who ran the study made an error in Excel.
Meanwhile, in the world of amusing-but-not-important baseball news, MLB Network’s Brian Kenny ripped into White Sox broadcaster Hawk Harrelson for blaspheming sabermetrics.
The way in which we interpret data is important. For years prior to the Moneyball revolution (which has definitely been related to the success of Nate Silver and Big Data’s popularity), people within the sports stats community were begging for somebody to pay attention to their numbers, and they had a darn good point. Good hypotheses based on solid evidence were ignored for traditional theory in a way that seemed fraternal and anti-scientific.
Since then, statistical analysis certainly has gained ground in front offices and with the greater fan base, but too often it’s presented, much like a lot of modern economic theory, as science.
A lot of blame here is on the media, which like to create false dichotomies to masquerade conversation as conflict. No example of this is better than MLB Network’s over-the-top commercial featuring Kenny and Harold Reynolds, with the former serving as the God of Logic and the latter as the God of Wisdom in an eternal battle to decide who should bat fifth for the Mariners.
The commercial begins with Kenny doing his best Will Hunting impression. (And if we’re gonna get all super nerdy, the best he can mutter is something about OPS? C’mon.) He then looks squarely into the camera and states resolutely, “Stats tell the truth,” which befuddles me. The truth … about what? Reynolds plays opposite as the old-timey baseball coach who learned the game on the diamond, not from a textbook.
This all really started with Michael Lewis’ over-dramatization of the front office divide between scouts and stats guys in Oakland, but it’s been taken to a whole other, dare I say, religious level. On one side is Sabermetrics, represented as a branch of science grounded in Enlightenment values and unyielding objectivity. On the other side is Scouting/Feeling/Traditionalism, represented as dealing with strategy, keen observation, and insightful instinct as a result of experience.
I talked to a random guy about baseball before this season started, and when I attempted to rebut his argument that the Mets would have the worst outfield in the history of baseball, he shook his head, looked at me solemnly and said, “Sabermetrics says so.” I guess I had two options: I could agree with him and trust the numbers or reject the numbers and trust faith. Sabermetrics said so, so I really had no other choice.
In the big data revolution, it’s always important to remember that there are no panaceas. Statistical analysis is a social science, not a physical one. The best anyone can do with a spreadsheet is test some thoughts and get results that mean the thoughts may be true after all. Nerdy 20s-something-looking kids with glasses are not the modern oracles. They’re just using a different tool.
*Note: for another good take that overlaps with this topic, check out Jack Moore’s article