Tuesday, March 23, 2010
Drawing statistical conclusions in baseball
Posted by Mike FastIf I were writing a course syllabus for an aspiring baseball analyst, this article by Phil Birnbaum would be one of the first lessons we'd cover. Bayesian methods and thinking don't get enough use in baseball analysis.
As Phil summarized the situation (in regard to an example dealing with clutch hitting):
A good way to think of it: the result of the statistical test adds to the pile of evidence on whatever issue it's testing. If you have no evidence, take the 70 points (or whatever) at face value. If you DO have evidence, use the 70 point difference to add to the pile, and it will move your conclusion one way or the other.
As Tango says, you are not entitled to assume that the difference is really 70 points just because the 70 is statistically significant. You have to make an argument. And, sure, your argument can be, "I don't know anything about baseball, so 70 points is the most likely difference." But it's perfectly legitimate for Tango to turn around and say, "I DO know something about baseball, and 70 points makes no sense."
Mike Fast is a Royals fan who enjoys investigating baseball questions using data of many sorts. He is a member of Complete Game Consulting. He welcomes comments via e-mail.
While I agree with the general thesis of the article, I do disagree with the Phil’s statement that the actual BA difference between clutch and non-clutch is “greater than 0 and less than 70” (and if I am misinterpreting his meaning, I apologize in advance). I don’t believe that statement is statistically correct, in absence of additional information. In fact as I and others posted over there, the best estimate of the difference based solely on the batting averages is 70, with a 95% confidence interval extending both above and below 70. Presumably it does not extend all the way to 0, else the null hypothesis (i.e., no difference) cannot be eliminated at the 95% confidence level.