Introducing: Quality of Opponent adjustments and CAPS (for pitchers)

image
Johan Santana shakes off one of his 33 home runs in 2007. Can the batters he faced explain his out-of-the-ordinary 14.5% HR/FB? (Icon/SMI)

A few weeks ago, I discussed how we need to adjust stats based on players switching leagues. I didn’t explicitly say it at the time, but when analyzing baseball players, it is of the utmost importance to understand the context under which stats are accumulated.

As I’m sure anyone reading this knows, it is far from enough to simply look at ERA. What some of you may not fully realize is that it isn’t even enough to analyze peripheral stats within a vacuum, anymore.

Lots of fantasy players (and websites, for that matter) are starting to realize that stats like K/9 and BB/9 are better indicators of a pitcher’s true skill than ERA or WHIP, so we—as fantasy owners—need to take things a step further to keep our advantage over them. The context in which pitchers post these stats is something I’ve yet to see any other website address (aside from Baseball Prospectus, although all they provide is BA/OBP/SLG—not of much use to fantasy owners).

Are they accumulated in the American League or the National League? A “pitcher’s” ballpark or a “hitter’s” ballpark? Against good batters or poor batters? That last one is the one that I’ll discuss today. If a pitcher faces a disproportionate number of Adam Dunn-type hitters, he is going to strike out and walk more batters than he should be. Because a pitcher has no ability to control the batters he faces, we can’t consider this a repeatable skill and must, therefore, neutralize a pitcher’s stat line based on the opposition he faces.

Method

To calculate the quality of opposition faced, I took the aggregrate year-end Marcels projection of every batter the pitcher faced in a given year. I then compared this to league average to arrive at a ‘quality of opposition index’ for each pitcher. I repeated this process for every stat that we care about for pitchers.

I considered using Sal Baxamusa’s daily Marcels method to estimate true talent, but eventually landed on using year-end values. I think fellow THTer Colin Wyers put it best when he said, “In the vast majority of cases, on the size of a single season, you’re not going to have a lot of cases where a player’s true-talent level drastically changes midseason. And you’d get better results for rookies—a day-by-day Marcels of, say, Evan Longoria is going to be very inaccurate to begin the season.”

That’s really all there is to this method. If you have any questions you think I didn’t address, feel free to let me know and I’d be happy to answer them.

Now let’s check out our quality of opposition adjustments in action.

Johan Santana

+------+---------+-------+-------+--------+-------+-------+
|      |         |       |       | Actual | Adj.  |  Adj. |
| YEAR | LAST    | FIRST | IP    | HR/FB  | HR/FB | Index |
+------+---------+-------+-------+--------+-------+-------+
| 2004 | Santana | Johan | 228.0 |   11.9 |  11.8 |  1.01 |
| 2005 | Santana | Johan | 231.7 |    9.9 |   9.6 |  1.03 |
| 2006 | Santana | Johan | 233.7 |   11.4 |  11.0 |  1.03 |
| 2007 | Santana | Johan | 219.0 |   14.5 |  12.9 |  1.11 |
| 2008 | Santana | Johan | 234.3 |   10.9 |  10.3 |  1.06 |
+------+---------+-------+-------+--------+-------+-------+

One of the concerns some Mets fan had coming into 2008 was that Santana was very prone to the long-ball in 2007. Luckily he bounced back in 2008, as we should have expected based on the unstable nature of HR/FB, but if we had these stats back then, we could have nearly wrote 2007’s HR/FB off completely as bad luck.

Santana was one of the unluckiest pitchers in baseball in 2007 in terms of opposition HR/FB, seeing inflation of 11 percent. If we neutralize his HR/FB, it’s still a little high at 12.9% but that is much closer to his career line and much easier to chalk up to random variation.

Dan Haren

+------+-------+-------+-----------+-------+--------+------+-------+
|      |       |       |           |       | Actual | Adj. | Adj.  |
| YEAR | LAST  | FIRST | TEAM      | IP    | K/9    | K/9  | Index |
+------+-------+-------+-----------+-------+--------+------+-------+
| 2005 | Haren | Dan   | Athletics | 217.0 |    6.8 |  7.0 |  0.97 |
| 2006 | Haren | Dan   | Athletics | 223.0 |    7.1 |  7.5 |  0.94 |
| 2007 | Haren | Dan   | Athletics | 222.7 |    7.8 |  8.3 |  0.93 |
| 2008 | Haren | Dan   | D'Backs   | 216.0 |    8.6 |  8.8 |  0.97 |
+------+-------+-------+-----------+-------+--------+------+-------+

Haren is an incredible example of why it’s important to consider context. Looking solely at his K/9, we would have thought he experienced a huge jump in 2008, and in projecting 2009, would expect a sizable regression. Bill James has him at 7.5, Marcels at 8.1, and Ron Shandler at 8.2.

If we look at his adjusted numbers, though, we see that he’s actually been steadily increasing over the past four years, culminating in an 8.8 adjusted K/9 in his age 27 season. And if we were to apply the 0.57 AL to NL adjustment, his 2007 figure would have exactly matched his 2008 one, and he’d have been over 8.0 three years in a row. A simple three-year weighted average would put his K/9 at 8.6—much more optimistic and accurate than the three projections listed above.

Javier Vazquez

+------+-------+------+------+------+---------+------+-------+-------+
| YEAR | IP    | QERA | K/9  | BB/9 | K/BB RI | xGB% | BABIP | HR/FB |
+------+-------+------+------+------+---------+------+-------+-------+
| 2006 | 202.7 | 3.84 |  8.2 |  2.5 |    0.59 |   40 | 0.311 |  10.7 |
| 2006 | 202.7 | 3.35 |  9.5 |  2.4 |    0.96 |   40 | 0.301 |   8.0 |
+------+-------+------+------+------+---------+------+-------+-------+
| 2007 | 216.7 | 3.34 |  8.8 |  2.1 |    0.84 |   38 | 0.294 |  12.1 |
| 2007 | 216.7 | 2.86 | 10.4 |  2.0 |    1.25 |   38 | 0.286 |   8.8 |
+------+-------+------+------+------+---------+------+-------+-------+
| 2008 | 208.3 | 3.76 |  8.6 |  2.6 |    0.62 |   39 | 0.320 |  11.3 |
| 2008 | 208.3 | 3.25 |  9.9 |  2.5 |    0.99 |   39 | 0.310 |   8.4 |
+------+-------+------+------+------+---------+------+-------+-------+

Note: To read this table, the first line for each year is Vazquez’s actual numbers. The second line is his adjusted line based on quality of opposition, ballpark factors, and the league change.

Vazquez is often talked about as an unlucky pitcher, but very few analysts notice that Vazquez has also been unlucky in the batters he’s faced. His strikeout numbers have been depressed by four, five and three percent, respectively, from 2006 to 2008.

Those numbers aren’t huge in-and-of themselves, but when you consider the five percent swing from U.S. Cellular to Turner Field and the 0.57 K/9 increase from switching leagues, Vazquez’s adjusted numbers are monstrous.

No matter how much bad luck he faces in terms of HR/FB (which will greatly improve moving away from Chicago), BABIP, or LOB%, I can’t see Vazquez’s ERA being held above 4.00 as it has four out of the last five years. In fact, his QERA hasn’t been higher than 3.35 over the past three years, and there’s a good chance his actual ERA ends up there in 2009. Plus, with the strikeout adjustments, he could strike out over 230 batters if he reaches his usual innings total. Huge fantasy value to be had here.

Caveats

As of right now, the quality of opposition indexes are based on overall MLB average. I’ve yet to break it down by league (AL/NL) or division, though I may do that sometime in the future. There’s a few logistical hurdles I need to jump over first, and I’m unsure if division quality is too unstable from year-to-year for use as a predictive stat.

CAPS

Since we’re working with so many adjustments now, and since not all of them will apply to every player, I’m simply going to call our new stat Context Adjusted Pitching Statistics (CAPS). When you see me refer to, say, Context Adjusted K/9 (or CAPS K/9), it means that I’ve included all adjustments that apply. For some pitchers, that might only be the quality of opposition adjustment. For others, it might mean the opposition adjustment and a ballpark adjustment. For others, like Vazquez, it could mean all three.

To present the results of these adjustments, I’m really leaning towards the format I used for Vazquez above. I think it allows me to include all of the important stats in an easy-to-understand format. I’d really appreciate feedback on this, though, so if you guys don’t agree, please let me know. Furthermore, while I’ve still got some cool stuff in the works behind-the-scenes, if you guys have any ideas for future adjustments you’d like me to tackle, don’t hesitate to contact me.

If you have any other questions regarding these new adjustments, please feel free to contact me.

References and resources

The listing of batters each pitcher faced was generated from files from the incomparable Retrosheet.

I also received help from fellow THT-writers Sal Baxamusa, David Gassko, and, in-particular, Colin Wyers. Colin’s willingness to help was incredible for a guy new on the THT scene, so many, many thank you’s Colin.


27 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Mike Podhorzer
15 years ago

For pitchers who stay on the same team the following year, wouldn’t his Quality of Opponent be a lot more repeatable than a team/league switcher? The team’s schedule is similar each year and although the pitcher won’t be facing the exact same hitters, I would think the adjustment would be less than for those pitchers making a switch.

Victor Wang
15 years ago

Great stuff Derek

Derek Carty
15 years ago

Alex,
Yeah, I’d like to be able to do that, but that would require three additional lines for every single year, which I think would really scare a lot of readers.  Perhaps I could put a section at the end of the articles with those numbers for anyone who is interested in them.  What would you think of that?

Mike,
That’s a definite possibility, and you’ve hit on the caveat I mentioned in the article.  A pitcher in the AL East would likely face a tougher set of batters than a pitcher in, say, the NL West.  I’m not sure how large of a difference that would be, but I am thinking of breaking it down by division, which would eliminate this.  There’s still a few things I’d need to figure out first.

Thanks, Victor!

Nick
15 years ago

Outstanding!  This is truly exceptional work.  I’m sure it would be difficult, but I do think that league/divisional adjustments are worth investigating.  Huge difference being in the AL East, I would think.

One other question about park adjustments.  Normally, we look at a player’s raw stats and then adjust them to reflect the fact that they played in Coors, or whatever.  However, when making projections, wouldn’t it be better to adjust a player’s “true talent” prediction not only for his home park (say again, Coors), but also for those parks in his division that he will play a disproportionate amount of games (because of the unbalanced schedule)?  Keeping with the Coors example, a Rockie will play a lot of games in SF, SD, LA, & AZ.  That’s one huge pitcher’s park, two more or less neutral park, and another big hitting park.  Would it make sense to consider these things when trying to formulate a more accurate fantasy projection?

Derek Carty
15 years ago

Thanks, Nick.  I’m definitely going to look into divisional adjustments.

As far as breaking down park factors by division, that’s a very interesting idea and one that probably wouldn’t be too complicated to do.  That’s something I’ll probably do shortly as well.

Thanks!

Alex
15 years ago

I’d like to see the adjustment indexes for each component stat and each ‘factor’ (park, league, opposition) added to the Vazquez format somewhere.  Not sure quite how you’d squeeze it all in though.

Matthew Chelius
15 years ago

I love the stat, however question – are CAPS and other stats such as QERA absolute & independant of each other?  Or when you combine CAPS with QERA, do they become redundant, and make an unlucky pitcher look even unluckier, and visa versa?

Donald
15 years ago

Great Work.  I have been wondering about something like this for a while.
You have just guaranteed that a number of people will be targeting Javier Vaz this year.

Nick
15 years ago

Matthew – The idea is that some of the peripherals that are used to generate things like QERA (K rate, BB rate) are effected by the quality of the opposition.  To use an extreme example, a pitcher’s walk rate would be much higher if he always faced the Red Sox as opposed to the Angels.  So if these peripherals are influenced by opposing batters, a peripheral-based stat like QERA will be influenced as well.  Do I have that right, Derek?

Matthew Chelius
15 years ago

Understood – I tried to locate the definition of the QERA and couldn’t find it.  I was thinking that QERA was possibly based or weighted on stats such as BABIP – and to me if you combined stats such as BABIP (which to me would be affected by the quality of opponent) with CAPS (which is soley based on quality of opponent) – you would get redundancy.

Either way, nice work of “leveling the playing field” so to speak.

Derek Carty
15 years ago

Nick’s got it spot on, Matthew.  What we really care about in fantasy is ERA, but there is so much noise that we can’t look at ERA by itself.  Instead, we look at things a pitcher can better control (like Ks, BBs, GBs, etc.) and from there we can construct an “expected” ERA based on these numbers.  This is what QERA or LIPS ERA does.

What CAPS does is adjust things like Ks, BBs, GBs, etc. based on other factors to make them more accurate.  Once adjusted, I then plugged these numbers into the QERA formula to make it more accurate.

So the answer to your question is *no*, there is no redundancy in these numbers.

Hopefully that clarifies it for you.  If not, let me know and I’ll try my best to put it a different way.

Dalton
15 years ago

This is good stuff Derek.

Derek Carty
15 years ago

Yeah, QERA is a Baseball Prospectus stat.  I prefer LIPS ERA but QERA is easier to calculate with these adjustments and is still a quality stat.

To further explain the BABIP/CAPS relation, BABIP is based on a number of things: pitcher skill, ballpark, defense, complete randomness, and quality of opponent (among a few others).  CAPS, when applied to BABIP, neutralizes the quality of opponent and ballpark factors to try and better represent the pitcher’s skill, or at least eliminate as much of the noise/randomness as possible.

QERA, however, doesn’t take BABIP or HR/FB into account.  It uses only K, BB, and GB and assumes a league average BABIP and HR/FB, which almost all pitchers regress towards (or close to, anyway).

What we need to do is look at QERA in conjunction with CAPS BABIP and CAPS HR/FB.  If a pitcher seems to show a legitimate, long-term trend of above or below average CAPS BABIP or HR/FB, then we can say that he should overperform or underperform his QERA accordingly.

Make sense?

Nick
15 years ago

Looking at pitchers who show significant differences in their K or walk rates according to CAPS obviously looks to make a significant difference.  And I guess you could theoretically make a better prediction by looking at CAPS BABIP or HR/FB, but I’m not sure that the differences would be greater than the normal error margin for a stats like QERA or tRA or LIPS or FIP or whatever.  At that point my guess would be that just regressing things like HR/FB and BABIP would be enough.  You never know though. 

Now, making an adjustment to BABIP according to a reliable DER projection, while still sketchy, I think would make a significant difference when trying to predict fantasy numbers.  I think PECOTA does this, but if they’re using FRAR I think it could be improved upon by using something like a three year average of UZR and converting that to a DER.

Nick
15 years ago

Which reminds me…

I pre-ordered the THT season preview, especially for the player projections.  Do you know much about how these are generated?  Is it a regression-based system like Marcels, or does it try to find like-player matches like PECOTA?  And are there any plans to update the projections online according to projected playing time after spring training?

Derek Carty
15 years ago

Nick, I’d talk to David Gassko about the projections.  He’s the one who puts them together.  I think it’s closer to a Marcels or CHONE than to a PECOTA.  Also, I believe he will be putting out updated projections online, but again, definitely ask him.

Tim
15 years ago

Nick, please let us know if you receive information from Gassko.  It would be easier than bombarding him with additional emails.  Thanks

Great work Derek.

Matthew Chelius
15 years ago

Thank you very much for the explanation, I am very interested in more of this analysis. 

Your last comment regarding legit trends above or below CAPS – if that proves to be true how would you be able to use it to predict stats for players with less data?  Such as rookies, or 2nd yr players?

Not trying to play devils advocate here, just learning.

Thanks, Matt

Mike
15 years ago

This calculation is well done and I’ve been waiting for someone smarter than me to get it done, thanks Derek!  If it’s possible, it would be a great case study to do two pitchers over their career: Greg Maddux and Mike Mussina.  I’m 98% sure Maddux’s numbers will still come out far better, but i’m betting Mussina’s numbers get a bigtime boost.

Mike
15 years ago

Derek,

With all the advanced defensive metrics now freely available, wouldn’t it be fairly easy to include a defensive adjustment to these stats. E.G. a player switching from a bad defensive team to a good one. I would assume the White Sox were not good on D last year, and the Braves were, so what kinda of effect would that have on Vazquez’s line? Even if it not the case, wouldn’t new defenders have an effect on a pitcher. Specifically for a pitcher like Vazquez, wouldn’t have Andruw Jones in his prime help him a lot or an outfield defense like the Mets have?

Nick
15 years ago

Will do.  Figured I would wait until I get the book, though.  Maybe there’s an explanation in there.  Also, didn’t buy the book last year.  Maybe someone who did can let us know if there was an explanation of the methodology?

David Gassko
15 years ago

Hey Nick,

Our system is regression based, though significantly more complicated than Marcel. Our projections will be updated throughout the off-season and available for free for anyone who purchases the Season Preview. Here’s the link to pre-order the book: http://www.actasports.com/detail.html?id=019

Derek Carty
15 years ago

No problem, Matthew.  I’m here to help you guys, so don’t feel bad to ask any questions.

The answer to your question is that it really can’t be used for rookies or second year players or even players entering their third year with some stats.  We simply have too little data on these players to say with any kind of certainty that they are the exception to the rule.

Just based on statistical principles, when dealing with sample sizes that small, we need to regress to the mean more heavily than we would with veteran players.

And to explain regression to the mean, this simply says that the less data you have on a player, the more room for error there is, so the closer we must assume a player’s true talent level is to league average.

Hope that answers your question, but feel free to follow up if you have any more.

Derek Carty
15 years ago

Mike,
I think that would be interesting to look at.  The problem would be creating Marcels for twenty years worth of players.  Not a task I’d revel doing and not one I likely have enough time for.  Sorry.

As far as defense, that is absolutely something that should be taken into consideration (although I’m not sure if it would have much of an effect on K and BB, if it’s significant at all).  The reason I haven’t included it now is that it’s hard to systematically apply a defense adjustment.  I normally just try to make a general note of it in my articles.

There’s a lot that would go into it, and some sort of projection would likely be necessary with a daily projection in-season.  Defense can vary from year-to-year, so using actual defensive numbers in-year could make things less accurate.  Since I don’t have or do defensive projections, I haven’t incorporated it yet.  It’s definitely something I’m looking into, though, so hopefully it will happen at some point.

Nick
15 years ago

Thanks, David.  Already ordered it and awaiting its arrival with baited breath.

Derek, you got linked by Neyer!  You’ve gone big time!

bpasinko
15 years ago

I find Vazquez’s situation to be quite funny.  He more or less loses no skill, but continuously under performs and therefore gets moved around year to year.  And it’s not like he gets shipped off for 50 cents on the dollar either, the Yankees got Randy Johnson, The Dbacks Chris Young, and the Braves a plethora of young guys for him.

It’s like all the GMs know he should turn it around, but they’re too impatient to live with it.

So is 2010 when the Braves trade him to Houston?

Nevin
15 years ago

Hey Derek,

Any chance that you’ve been able to look at at least Vazquez himself, going back a but further than the 4 years?  Would be interesting if he continuously in his career has underperformed his peripheral K and BB walks rates, CAPS adjusted.  If he has, would seem to indicate that there’s something about him that gets away from the math, or that isn’t being captured.

regards,

Nevin