Uncovering DIPS

One of the most important concepts in sabermetrics is DIPS theory, which states, essentially, that pitchers have a lot less control over the outcome of balls-in-play than is often assumed, and that in fact, differences in the ability to prevent hits on balls-in-play are not that great. Why is this so important? Well, more than three-quarters of all plate appearances end with a ball being put into play, meaning that a pitcher’s actual rate of hits on balls-in-play (otherwise known as BABIP) can a have a huge impact on his statistics, while telling us little about what to expect from him in the future. As Voros McCracken, the inventor of DIPS, surmised, “You can better predict ERA without hits allowed than you can with them.”

But Voros’ theory leaves us with a troubling question: How much control do pitchers have over balls in play, and where does it come from? Soon after Voros made his theories public, answers to that question came from a multitude of sources. Voros emphasized that home runs and strikeouts had a weak but significant correlation with BABIP. Tom Tippett found that pitchers that perform exceptionally in any important category tend to have lower batting averages on balls in play. Mitchel Lichtman studied the year-to-year consistency of batted ball information and found that while pitchers deviated little in their batting average on balls in play, they demonstrated substantial control in the types of batted balls they allow.

The latest word on DIPS has come from two sources: JC Bradbury, and the Hardball Times Annual 2007. Almost two years ago, JC published an article here at THT examining the relationship between defense-independent variables and batting average on balls in play. He concluded that, “While pitchers may have some ability to prevent hits on balls in play, the effect is small. And any effect a pitcher does have is reflected within DIPS metrics.” This conclusion, if true, is incredibly important to our understanding of DIPS. While it is widely acknowledged that pitchers do have some control over the outcome of their balls in play, if that control is totally reflected by their fielding-independent statistics, it becomes that much easier to forecast a pitcher’s future results, and easier to separate the pitcher from his defense. However, I believe that some of JC’s tests were somewhat shaky, and I will attempt to recreate them in a more robust manner.

In the 2007 Annual, I went back to Mitchel’s fascinating “DIPS Revisited” tests, and, using data purchased by The Hardball Times between 2003-06, examined both pitcher and hitter control of batted balls, and of the results of those balls in play. What I found was that pitchers do not see much deviation at all in their groundball and fly ball rates from year-to-year, have a good amount of control over the number of pop-ups they allow, and have little control over their line drive rates. In part two of this article, we will look more closely at how much variance in BABIP is explained by batted ball information.

Today, however, let’s concentrate on fielding independent (often known as “peripheral”) statistics. What do a pitcher’s peripherals tell us about his batting average on balls in play? It turns out, a whole lot.

The Pitcher Versus Himself

The first question we can ask is this: How does a pitcher’s batting average on balls in play vary with his peripheral statistics? That is, is a change in a pitcher’s peripheral statistics correlated with a change in his BABIP? It’s an interesting question because of the possible implications: If we indeed find some correlation, we would in fact be discovering a direct relationship between changes in a pitcher’s style and the number of hits he allows on balls in play. That would serve as the most direct link possible between a pitcher’s peripheral statistics and his BABIP. This idea was originally posited by Cy Morong at Beyond the Boxscore.

A few words on the data used for the study before we delve into the results: I looked at all pitcher seasons between 1921-2005, and for each season, I adjusted each pitcher’s batting average on balls in play for his team (which includes an automatic adjustment for league, park, and defense), and I adjusted all other statistics for the league average. I then recomputed each pitcher’s stat line in a perfectly average league.

In this particular study, I looked at all pitcher seasons for pitchers with at least 1,500 career balls in play since 1921, and at least 200 balls in play in that particular season. That gives us a gigantic sample of 12,265 pitcher seasons and well over six million balls in play. For each pitcher season, I compared the pitcher’s adjusted numbers (from now on, I will not refer to the statistics as adjusted, but rest assured, they all are) to his career statistics. So the average in this case is the pitcher’s career average, and each of his seasons is compared to that.

This is a more pertinent way of looking at things than just finding correlations between fielding independent statistics and BABIP. If strikeouts correlate with a low BABIP, for example, we don’t know if that is because high-strikeout pitchers tend to allow easier-to-field balls in play or if striking out more batters somehow helps lower batting average on balls in play. The former method is comparing one pitcher to another—apples and oranges, or Santanas and Moyers—but the latter allows us to hold the most important variable, the pitcher, constant.

So what is the effect of peripheral statistics on batting average on balls in play? Using multiple regression analysis, we can find out:

Constant       .2861
SO/Game       -.0011
BB/Game        .0011
HR/Game       -.0021
HBP/Game       .0015
	
1 = Significant at the 1% level
5 = Significant at the 5% level
10 = Significant at the 10% level

Correlation = .082

Let’s discuss these results for a moment. First, what we see is that every peripheral statistic has a statistically significant impact on BABIP. Strikeouts and home runs are associated with a drop in BABIP, while walks and hit batters show the opposite. The overall correlation is low, but not much lower than the year-to-year correlation of batting average on balls in play. What that means is that most of the observed variance in BABIP is due to the fielding independent coefficients. In other words, a player’s batting average on balls in play is tied to, and mostly explained by his peripheral statistics!

Let’s discuss the specific coefficients for a moment. As has generally been acknowledged (most importantly by Voros himself in introducing DIPS 2.0), strikeouts and home runs serve to drive down BABIP. Why? Well, to start, both are associated with fly ball tendencies, and fly ball pitchers allow less hits on balls in play. Strikeout pitchers may also allow easier to catch batted balls because their pitches are harder to hit (and, so the theory goes, to hit well).

Why would walks and hit batsmen correlate with BABIP? Both are symptoms of control issues, which perhaps correlate with bad pitching overall. As well, pitchers who walk a lot of hitters probably end up behind in the count more often, when batters hit better. Or perhaps, as with strikeouts and home runs, there is a link between the two and batted ball distributions. But we’re getting ahead of ourselves…that question will be addressed next week.

What’s important is the discovery that a change in peripheral statistics can indeed predict a change in BABIP. There is a clear and identifiable relationship between fielding independent numbers and batting average on balls in play. Not only does that confirm pitcher control over his hit-rate on balls in play, but it tells us that fluctuations in that rate move in conjunction with other fluctuations, which goes a long way in explaining what factors determine how a pitcher will perform on balls in play.

Odds and Evens

What the above analysis does not tell us, however, is how much additional information a pitcher’s batting average on balls in play adds to what we already know from fielding independent information. JC’s contention was that once peripheral statistics are accounted for, a pitcher’s BABIP contains no predictive information—something that would be both a radical and extremely helpful fact, if it were true. Essentially, it would mean that once we account for the necessary variables, we could ignore balls in play completely.

JC developed his thesis based on two years of data, which introduces a great deal of potential error into the model. I will take a page from Keith Woolner’s book, and increase the sample size substantially by looking at the career statistics of each pitcher in my sample, and splitting them into two halves. To avoid biases (such as due to age), I will split each career into odd and even seasons. Again, all numbers are properly adjusted for context, and only statistics from 1955 and later were used. The cutoff to be included in the sample is 1,000 balls in play in each career half, which leaves us with 1,077 pitchers and almost 5.5 million BIP. (Note: Some pitchers were removed from the sample because the Lahman Database did not contain height information for them. This does not have any effect on the results, as the sample is obviously more than sufficient anyways.)

What we can then do is regress a pitcher’s peripheral statistics and batting average on balls in play in one career half against his batting average on balls in play in the other half. If the peripheral statistics capture all the predictive value of BABIP, then coefficient for batting average on balls in play will be insignificant. (Note: I have also included controls for handedness and height, two variables that Voros indicated had an effect on BABIP in his article on DIPS 2.0.) Essentially, we’re asking: Can we predict batting average on balls in play without BABIP as well as we can with it?

Constant       .2311
BABIP          .1841
SO/Game       -.0021
BB/Game       -.00110
HR/Game       -.0121
HBP/Game      -.003
Left           .0021
Height         .00010
	
1 = Significant at the 1% level
5 = Significant at the 5% level
10 = Significant at the 10% level

Correlation = .358

As you can see, the coefficient for BABIP is highly significant, meaning that a pitcher’s control over the results of his balls in play extends beyond his fielding independent numbers. However, what is interesting is that is we look at the beta coefficients (which correct for the spread in each variable), we find that BABIP is just 25% of the pie. What that means is that batting average on balls in play is about as important in predicting itself as is just the pitcher’s strikeout rate or home run rate. That’s incredible! While each pitcher does have some control over his BABIP that is unique to him, his hit rate on balls in play is mostly as predictable without knowing his previous demonstrated ability to prevent hits on balls in play as it with it. Even given a minimum of 1,000 balls in play, 75% of a pitcher’s projected BABIP should be based on fielding independent statistics. That’s huge!

Let’s discuss the coefficients for a moment. We’ll start with the ones a pitcher doesn’t have any control over: height and handedness. Height has a marginally significant effect on BABIP, though the listed coefficient is .000. That’s simply because I’m limiting the numbers to three decimal points; in fact, each extra inch is ever-so-slightly associated with an increased BABIP, perhaps because tall pitchers are more awkward fielders (this was Voros’ suggestion, at least). Left-handed pitchers tend to allow a higher average on balls in play than do righties, likely because first basemen convert balls in play at a higher rate than do third basemen.

The home run and strikeout coefficients show the correct (expected) signs, as both drive down BABIP. However, walks and hit-by-pitch also have negative coefficients, which is unexpected. The HBP coefficient, at least, is insignificant, so I won’t spend too much time dwelling on it. Walks, however, are at least marginally significant so perhaps they deserve more attention. Or perhaps not.

The tests I have run in this article are actually measuring two different things. The first looked at how deviations in a pitcher’s peripheral statistics correlated with changes in his hit rate. This test, however, is looking at the predictive power of those peripherals in comparison with the predictive power of BABIP. The first test is backwards-looking; this one is about looking into the future. So it is entirely plausible that an up-tick in a pitcher’s walk rate can indicate an increase in his batting average on balls in play, while overall, a high walk rate corresponds to a low BABIP.

What’s most important, though, is the result itself, and not any specific number or sign. What is important is that while BABIP is a somewhat predictable statistic, much of a pitcher’s rate of hits on balls in play can be predicted from his fielding independent numbers alone. That means that BABIP can be pretty easily projected from stable peripheral numbers and phenotypic attributes, with a pitcher’s actual batting average on balls in play making up only 25% of the equation. That allows for more precise predictions, and also serves as an indicator of the tight limits on a pitcher’s ability to prevent hits on balls in play independent of his other statistics.

Concluding Thoughts

No matter how many traditional statistics we try to control for, batting average on balls in play is a meaningful statistic, and DIPS theory is not completely right. However, BABIP correlates very well with more stable fielding independent statistics, and is only limitedly useful in projecting how a player will fare on balls put into play. A pitcher’s peripheral statistics can be used both to predict and explain batting average on balls put into play, and they can do a lot of both. What remains to be seen is why that is. Can we use new cutting edge data to fully explain away batting average on balls in play, or do pitchers have some amount control that cannot be explained no matter what? For what reasons do peripheral statistics correlate with BABIP? Those questions, we will answer next time.