Drawing statistical conclusions in baseball

If I were writing a course syllabus for an aspiring baseball analyst, this article by Phil Birnbaum would be one of the first lessons we’d cover. Bayesian methods and thinking don’t get enough use in baseball analysis.

As Phil summarized the situation (in regard to an example dealing with clutch hitting):

A good way to think of it: the result of the statistical test adds to the pile of evidence on whatever issue it’s testing. If you have no evidence, take the 70 points (or whatever) at face value. If you DO have evidence, use the 70 point difference to add to the pile, and it will move your conclusion one way or the other.

As Tango says, you are not entitled to assume that the difference is really 70 points just because the 70 is statistically significant. You have to make an argument. And, sure, your argument can be, “I don’t know anything about baseball, so 70 points is the most likely difference.” But it’s perfectly legitimate for Tango to turn around and say, “I DO know something about baseball, and 70 points makes no sense.”


3 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Mike Fast
14 years ago

Alan, I think Phil was quoting Tango there, and that Phil in fact is in agreement with what you and others said in the comments there.

Nick Steiner
14 years ago

If I were writing a course syllabus for an aspiring statistician, this article by Phil Birnbaum would be one of the first lessons we’d cover.

Fixed.

Alan Nathan
14 years ago

While I agree with the general thesis of the article, I do disagree with the Phil’s statement that the actual BA difference between clutch and non-clutch is “greater than 0 and less than 70” (and if I am misinterpreting his meaning, I apologize in advance).  I don’t believe that statement is statistically correct, in absence of additional information.  In fact as I and others posted over there, the best estimate of the difference based solely on the batting averages is 70, with a 95% confidence interval extending both above and below 70.  Presumably it does not extend all the way to 0, else the null hypothesis (i.e., no difference) cannot be eliminated at the 95% confidence level.