Catcher Defense 2010

Estimating a player’s defensive value is quite difficult.

I know, I know—duh. BUT, I would venture to say that it’s a tad bit easier to estimate a catcher’s defensive value compared to other positions. At least, most of it. There are, of course, certain aspects of a catcher’s impact behind the plate that we can only feebly grasp at—game calling, for example, and framing pitches, amongst other things. But we can still make a reasonable estimate of things that we do know about—the catcher’s caught stealing rate, prevention of wild pitches or passed balls, things like that—and while it is certainly influenced by the pitching staff the catcher works with, there is one piece of information we have for catchers that separates them from the rest of the position players: opportunities. We know the number of plays made by infielders and outfielders, yes, but not the actual amount of chances they had afield. When it comes to catchers, we do know how often a baserunner attempted to swipe a bag; we do know how often a catcher allowed a passed ball with a runner on and a base open for the taking, and so on.* So in some ways, we’re a bit ahead of the curve with catchers than we are with the other positions.

When I first began studying sabermetrics, I was heavily influenced by Justin Inaz’s series of player valuation at his site, Basement-Dwellers.com (it appears to be abandoned, which makes me really sad—it was one of my favorites). In it, Justin presented a method for estimating a catcher’s defensive efficiency based on statistics that were carried here at The Hardball Times: the catcher’s runs saved or cost compared to the league average based on stolen bases allowed, wild pitches and passed balls, and the catcher’s rate of errors (differentiating between throwing and fielding errors). Matt Klaassen of FanGraphs and Beyond the Box Score has brought it back, and I thought I’d revisit his work (the more people that see this, the better) with some slight differences:

• The run values are tailored for the 2010 run environment and were derived from a Base Runs equation, which was derived from empirical linear weights. This is really just for theoretical accuracy more than anything else.
• Catcher pickoffs have been added. So we’re getting a look at not only runners removed from stolen base attempts, but also the guys being caught off guard.
• I use different denominators for the rates—rather than WP or PB per PA, I’m doing it as a function per stolen base opportunity, and errors per defensive chances, not PA or innings.
• There is an estimate for a catcher’s efficiency at handling balls in play.

Naturally, these estimates are nothing more than just that—estimates. They are approximations of a catcher’s defensive value based on freely available data located at Baseball-Reference.com or Sports Illustrated. They are by no means definitive; plenty of adjustments could be made to enhance the quality of the estimate. Think of it as a crude starting point. Commentary on the components are below, and, of course, a spreadsheet containing all of the data can be found at the very bottom.

CS Runs are the catcher’s runs above/below the league average based on their caught stealing rate. I’m only giving catchers credit for times in which they recorded an assist (in other words, I’m excluding times runners were picked off by the pitcher). The league average in 2010 was 22.9%, and the run value of the caught stealing is 0.641 (the run value of the SB in 2010 was .193; CS -.448). The best was, unsurprisingly, Yadier Molina at +9 runs saved (with an astonishing 44.4% CS rate), followed by Miguel Olivo (+7) and Lou Marson (+5). The worst was Ryan Doumit (-9), followed by East Coast powerhouse catchers Victor Martinez (-6) and Jorge Posada (-6). The spread between the best and worst is 17 runs; nearly two wins.

PO Runs are the catcher’s pickoff runs based on their rate of picking off runners per stolen base opportunity. These are relatively rare occurrences—it happened only 60 times in 68,338 opportunities (.08%)—but some players excelled at removing potential stolen base threats. Humberto Quintero led the Majors at +4 runs, with Russell Martin (+3) and Jeff Mathis (+2) directly behind him, and Jason Kendall, Kurt Suzuki and Joe Mauer at the bottom of the list with -1 run. The run value of the pickoff is about .535 runs, and the overall spread was five runs; about half a win.

Glove Runs consist of the catcher’s passed balls and wild pitches allowed per stolen base opportunity. I treat the two separately, although more often than not the two are indistinguishable and the difference between them is purely arbitrary. The run value of the passed ball is .279 runs; the wild pitch .282, so they’re essentially the same in terms of runs. I’m also including the catcher’s runs saved or cost due to catcher’s interference (.365 runs), which makes a very small difference (+ 1 run). The Major League leader was Phillies catcher Carlos Ruiz at +7, followed by Nats backstop Ivan Rodriguez (+6) and Matt Wieters (+5). At the bottom of the list are Jeff Mathis (-6), Adam Moore (-5) and Miguel Olivo (-5). The difference between the best and worst catcher is approximately a win and a half, a bit more than I imagined.

The player’s Error Runs are composed of the player’s rate of throwing errors and fielding errors per defensive chance. Following Justin’s methodology, the run value of a throwing error is about .492 runs and a fielding error .758. The best in the Bigs was Chris Snyder at +2, followed by Joe Mauer (+1) and Yadier Molina (+1), while the worst were Francisco Cervelli (-3), Jason Kendall (-3), and Brian McCann (-3). All in all, we’re looking at about a half win of difference.

Last but not least, the catcher’s Range Runs. This is a trivial addition but I thought it would be fun to throw in to the mix. Catchers don’t see that many ball in play chances—we’re talking about 30 or so chances at the most—but some catchers are a bit more adept at converting attempted bunts or weak hits into outs than others. The run value of an out made above average is .666 (infield singles are about .40 runs; the out value in 2010 is -.266). The best catcher at handling balls in play was Ryan Doumit at +1 run, followed by Ivan Rodriguez and Ramon Hernandez. The worst were Gregg Zaun, Francisco Cervelli and Russell Martin, all at -1.

Putting it all together, we get:

Name	Runs
Yadier Molina	16
Ivan Rodriguez	11
Carlos Ruiz	10
Matt Wieters	9
Humberto Quintero	9
Lou Marson	6
Henry Blanco	5
Yorvit Torrealba	5
Ramon Hernandez	5
Brian McCann	4

Dang, Yadier. Looks like Pudge still has it, and it’s nice to see a kid with so much offensive potential in Wieters score so well in his defensive ratings.

The trailers:

Name	Runs
Chris Iannetta	-4
John Hester	-5
Kevin Cash	-5
Victor Martinez	-6
Mike Napoli	-6
Adam Moore	-8
Jeff Mathis	-8
Ryan Doumit	-9
Jorge Posada	-10
Francisco Cervelli	-10

Mathis is considered to be the “defensive specialist” if I remember right, so it’s a bit odd to see him do so poorly. The Yankees were horrendous behind the dish last season, so switching to Russell Martin (+2) as the primary backstop could give them an extra win based on just the simple categories used. For a relatively small signing, it could have a modest impact for the Yanks.

All in all, there’s a 2.8 win difference between the best and worst catcher, which is pretty large. Of course, that’s not accounting for the intangibles, so it’s pretty apparent that the quality of defense behind the plate—even when you’re not paying attention to framing, pitch sequencing, etc.—is pretty important.

Yeah, yeah, I know…I promised you all a spreadsheet. So here it is– I hope you enjoy.

UPDATE (1/9): I’ve put WP and PB together (averaging the run value as well). The spreadsheet has been updated.

*As I already mentioned, naturally, pitchers can and do affect a catcher’s CS and WP/PB rates. Of course, if one is interested in estimating the catcher’s true talent, you’ll have to regress each component to the mean. I imagine this would help sift out the impact of the pitcher by at least a little bit.


17 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Doug Gray
13 years ago

JT,
Justin from Basement-dwellers is now writing at both http://www.RedReporter.com and http://www.beyondtheboxscore.com if you still want to follow his work.

JT Jordan
13 years ago

Thanks for the heads-up!

Jason Linden
13 years ago

Question: Given how much pitch data there is out there, shouldn’t it be at least somewhat possible to figure out if any catchers are good at framing pitches. If someone is especially good, shouldn’t he get more strike calls on balls outside the zone and vice versa? This should be measurable, right?

JT Jordan
13 years ago

I would certainly think so- and if I remember right, I’ve seen some work using Pitch F/X data to look at framing.  I haven’t seen anything too recently, though.

Mike Fast
13 years ago

Framing has been tackled previously by Dan Turkenkopf:
http://www.beyondtheboxscore.com/2008/4/5/389840/framing-the-debate
Matthew Carruth:
http://www.lookoutlanding.com/2010/2/1/1285412/framing-the-framing-debate
and Bill Letson:
http://www.beyondtheboxscore.com/2010/3/26/1360581/a-first-pass-at-a-catcher-framing
(apologies if I missed anyone)

All of the results are problematic, though.  I think the problem may lie in the fact that we don’t really understand the strike zone well enough yet.  But I’m not sure.  If the problems with measuring catcher framing were obvious, bright folks like Dan, Matthew, and Bill would have identified them.

Bad Bill
13 years ago

Very nice analysis, but I think it may undervalue the defensive value of cannon arms like Y. Molina.  That guy doesn’t just obliterate runners who try to steal; he deters the entire running game practically out of existence.  This year only 68 runners attempted steals on him in 1138 innings caught.  That compares to 1985 attempted steals in 21950 innings versus non-Yadi catchers in the NL, who aggregated a CS percentage of just over 28%.  In essence, about 35 guys didn’t try to run on him at all, but would have against an “average” catcher, and likely would have succeeded.  I think he and the other SB-deterring catchers should get credit for that in your analysis.

JT Jordan
13 years ago

Bill,

I’ve done “reputation runs” in the past and I remember Sean Smith did that quite a while back with his catching runs.  But I also remember there were a few people that were against the idea (MGL?  Can’t remember).  Basically, you’ll want to use a WOWY approach if you’re going to do it- otherwise you’re looking at quite a bit of influence from the pitchers, not just the catchers.  A left-handed heavy staff is going to see less SB attempts, pitchers that are good at holding runners on are going to allow less SB attempts, etc.

Brian Cartwright
13 years ago

I did a similar study about two years ago at FanGraphs http://www.fangraphs.com/blogs/index.php/stealing-on-catchers-not-named-molina/

As for Bad Bill’s comment, I used Tango’s WOWY to account for the influence of pitchers on base stealing, and my runs formula had net SB and net CS, so a catcher would get credit for runners not attempting.

I’m still using the same formula in the catcher’s fielding available here in the THT Forecasts.

Brian Cartwright
13 years ago

Bill, ‘reputation’ can mean looking at the stat sheet and seeing it’s not worth running. I handle it with WOWY by finding, given the identity of the pitchers and the baserunners, the expected number of steals and caught stealing vs each catcher, comparing that with the observed SB & CS, and applying linear weights to get a run value.

MGL is correct in noting that if a catcher allows a league average SB%, the runs saved will be zero regardless fo the attempts. The best thing for an offense to do against Yadier is NOT run, as on average a steal attempt loses runs for the offense – therefor, stealing less vs Yadier gains runs back for the offense, and worsens his defensive runs. Conversely, a catcher such as Piazza would be made better by running on him less.

But I beg to differ with MGL on WP vs catchers. The official scorer has said that a WP is mainly against the pitcher, and a WP mainly against the pitcher, but as in base stealing, their is shared responsibility that can be determined with WOWY. I did another piece at FanGraphs on catcher’s WP & PB http://www.fangraphs.com/blogs/index.php/more-on-catchers-fieldingwppb/

Bad Bill
13 years ago

Fair enough, JT, but Yadier Molina has had a caught-stealing rate of close to 50% for his whole freaking CAREER.  Furthermore, St. Louis during that time hasn’t been heavy on left-handers (although several of the lefties they’ve had have been plenty heavy…); if anything, rather the opposite, Jaime Garcia this year was the first LH starter since Mark %^&*$ Mulder to have thrown many innings for them.  The CS% has been robust against the turnover in pitching staff that they, like any other team, have experienced during Molina’s career.

Other teams aren’t being deterred from running on Molina by a nebulous “reputation.”  They’re being deterred because managers, coaches and baserunners can all look at hard, objective data and see that reliance on the running game would be likely to be a Bad Idea when facing St. Louis.

MGL
13 years ago

I am not necessarily against “reputation runs”. The difference between a league average catcher and the catcher in question has to be accounted for somehow. If the league average catcher allows a net positive value (if the league average stolen base rate is above the break even point, which it probably is now), then a catcher who never allows any SB attempts is obviously worth something.  If the league average SB rate is around equal to the BE rate, then we won’t need reputation runs.

The interesting thing about that framework is that if players cannot do better than break even across the board, then the best a catcher can do should top out at zero runs, if runners were running optimally against them.  Something like that anyway.

I think the reason that Brian (maybe it is Peter) likes using “reputation runs” is that he (correctly, I think) asserts that a stolen base attempt is worth something when the batter puts the ball in play, such that we are not giving enough weight to a stolen base attempt in general.  Of course, this can be taken care of just by adjusting the values of the SB and the CS to take into consideration the value of the attempt, even though it is a roundabout way of doing it.

To me, the easiest and cleanest (and precise, I think) method of accounting for SB/CS runs for catchers is to simply assign a proper value to SB and CS for all catchers.  Then you adjust everyone’s value for the league average number so that you can represent each catcher’s value as “above or below average.”  That takes care of everything.  Absolutely no need to do reputation runs.

One more unrelated comment. I do not like the idea of including WP in a catcher rating.  In fact, I find it ludicrous.  Yes, when a ball is in the dirt, catchers have some (maybe a lot) skill at preventing the WP or not, but the majority (or at least a significant part) of the blame (or not) is with the pitcher.  For PB, it is exactly the opposite.

So, if you are going to include WP in a catcher’s rating, you must do one or both of the following:  One, some kind of adjustment (a WOWY) for the staff of pitchers he catches, and/or two, use some percentage of WP, probably at most .5 (IOW split the responsibility with the pitcher).  Of this, I am quite certain.  Combining PB and WP is horrible.  A catcher who catches a pitcher who throws hard and is wild (like a Marmol) is going to get unfairly brutalized, whereas a catcher who catches someone like Maddux is going to come out smelling like a rose, which tells you everything you need to know.  The only pitchers I can foresee a similar problem with for PB are knuckleballers.

MGL
13 years ago

And of course, as JT mentions, pitchers also can and do greatly influence a catchers SB and CS numbers, so you really have to either do some kind of adjustment, some kind of split responsibility or some kind of regression…

JT Jordan
13 years ago

Bill, you’re right- but keep in mind:

“(…) plenty of adjustments could be made to enhance the quality of the estimate. Think of it as a crude starting point.”

WOWY or anything like that would be a bit beyond the scope of what we’re going for here (and if I was good with databases and the like, this would certainly be more comprehensive).  So yes, some catchers will be undervalued due to their “reputation”- just like some will be undervalued while being outstanding at blocking the plate or working well with the pitching staff.

MGL, I’m not confident enough in the official scorers to say that we can really differentiate between wild pitches and passed balls with a great deal of certainty.

That said, I believe I’ve made a mistake in separating between the two (and the spreadsheet has been updated).

Brian Cartwright
13 years ago

The rules state that if ball bounces in the dirt and then gets away, it must be ruled a wild pitch. But of course some catchers will be better at blocking those bouncing pitches than others.

JT Jordan
13 years ago

I imagine that would be entirely due to rounding.

Paul
13 years ago

Don’t know if its too late for this, but is the Runs Saved column just a summation of the other 6 columns? If so, there’s a handful of errors in there…

jinaz
13 years ago

JT, thanks for the kind words & mention above.  As Doug mentioned, I’m still around, just not at that site typically.  I just don’t have the time to keep my own blog active, plus I get a lot more eyeballs/comments by posting on the sbnation blogs.  Sometimes I even send stuff to studes to see if he’s interested in publishing it here! smile

Anyway, sorry I missed this at first, but nice work!  The catcher stuff was fun to put together (even if I just ended up replicating what Rally had already done), and I’ve been surprised at how long and well it’s held up.  There are better ways, but until they’re accessible this is a pretty good starting place.
-j