Search THT:

Or you can search by:

January 07, 2011

Catcher Defense 2010

Estimating a player’s defensive value is quite difficult.

I know, I know—duh. BUT, I would venture to say that it’s a tad bit easier to estimate a catcher’s defensive value compared to other positions. At least, most of it. There are, of course, certain aspects of a catcher’s impact behind the plate that we can only feebly grasp at—game calling, for example, and framing pitches, amongst other things. But we can still make a reasonable estimate of things that we do know about—the catcher’s caught stealing rate, prevention of wild pitches or passed balls, things like that—and while it is certainly influenced by the pitching staff the catcher works with, there is one piece of information we have for catchers that separates them from the rest of the position players: opportunities. We know the number of plays made by infielders and outfielders, yes, but not the actual amount of chances they had afield. When it comes to catchers, we do know how often a baserunner attempted to swipe a bag; we do know how often a catcher allowed a passed ball with a runner on and a base open for the taking, and so on.* So in some ways, we’re a bit ahead of the curve with catchers than we are with the other positions.

When I first began studying sabermetrics, I was heavily influenced by Justin Inaz’s series of player valuation at his site, Basement-Dwellers.com (it appears to be abandoned, which makes me really sad—it was one of my favorites). In it, Justin presented a method for estimating a catcher’s defensive efficiency based on statistics that were carried here at The Hardball Times: the catcher’s runs saved or cost compared to the league average based on stolen bases allowed, wild pitches and passed balls, and the catcher’s rate of errors (differentiating between throwing and fielding errors). Matt Klaassen of FanGraphs and Beyond the Box Score has brought it back, and I thought I’d revisit his work (the more people that see this, the better) with some slight differences:

• The run values are tailored for the 2010 run environment and were derived from a Base Runs equation, which was derived from empirical linear weights. This is really just for theoretical accuracy more than anything else.
• Catcher pickoffs have been added. So we’re getting a look at not only runners removed from stolen base attempts, but also the guys being caught off guard.
• I use different denominators for the rates—rather than WP or PB per PA, I’m doing it as a function per stolen base opportunity, and errors per defensive chances, not PA or innings.
• There is an estimate for a catcher’s efficiency at handling balls in play.

Naturally, these estimates are nothing more than just that—estimates. They are approximations of a catcher’s defensive value based on freely available data located at Baseball-Reference.com or Sports Illustrated. They are by no means definitive; plenty of adjustments could be made to enhance the quality of the estimate. Think of it as a crude starting point. Commentary on the components are below, and, of course, a spreadsheet containing all of the data can be found at the very bottom.

CS Runs are the catcher’s runs above/below the league average based on their caught stealing rate. I’m only giving catchers credit for times in which they recorded an assist (in other words, I’m excluding times runners were picked off by the pitcher). The league average in 2010 was 22.9%, and the run value of the caught stealing is 0.641 (the run value of the SB in 2010 was .193; CS -.448). The best was, unsurprisingly, Yadier Molina at +9 runs saved (with an astonishing 44.4% CS rate), followed by Miguel Olivo (+7) and Lou Marson (+5). The worst was Ryan Doumit (-9), followed by East Coast powerhouse catchers Victor Martinez (-6) and Jorge Posada (-6). The spread between the best and worst is 17 runs; nearly two wins.

PO Runs are the catcher’s pickoff runs based on their rate of picking off runners per stolen base opportunity. These are relatively rare occurrences—it happened only 60 times in 68,338 opportunities (.08%)—but some players excelled at removing potential stolen base threats. Humberto Quintero led the Majors at +4 runs, with Russell Martin (+3) and Jeff Mathis (+2) directly behind him, and Jason Kendall, Kurt Suzuki and Joe Mauer at the bottom of the list with -1 run. The run value of the pickoff is about .535 runs, and the overall spread was five runs; about half a win.

Glove Runs consist of the catcher’s passed balls and wild pitches allowed per stolen base opportunity. I treat the two separately, although more often than not the two are indistinguishable and the difference between them is purely arbitrary. The run value of the passed ball is .279 runs; the wild pitch .282, so they’re essentially the same in terms of runs. I’m also including the catcher’s runs saved or cost due to catcher’s interference (.365 runs), which makes a very small difference (+ 1 run). The Major League leader was Phillies catcher Carlos Ruiz at +7, followed by Nats backstop Ivan Rodriguez (+6) and Matt Wieters (+5). At the bottom of the list are Jeff Mathis (-6), Adam Moore (-5) and Miguel Olivo (-5). The difference between the best and worst catcher is approximately a win and a half, a bit more than I imagined.

The player’s Error Runs are composed of the player’s rate of throwing errors and fielding errors per defensive chance. Following Justin’s methodology, the run value of a throwing error is about .492 runs and a fielding error .758. The best in the Bigs was Chris Snyder at +2, followed by Joe Mauer (+1) and Yadier Molina (+1), while the worst were Francisco Cervelli (-3), Jason Kendall (-3), and Brian McCann (-3). All in all, we’re looking at about a half win of difference.

Last but not least, the catcher’s Range Runs. This is a trivial addition but I thought it would be fun to throw in to the mix. Catchers don’t see that many ball in play chances—we’re talking about 30 or so chances at the most—but some catchers are a bit more adept at converting attempted bunts or weak hits into outs than others. The run value of an out made above average is .666 (infield singles are about .40 runs; the out value in 2010 is -.266). The best catcher at handling balls in play was Ryan Doumit at +1 run, followed by Ivan Rodriguez and Ramon Hernandez. The worst were Gregg Zaun, Francisco Cervelli and Russell Martin, all at -1.

Putting it all together, we get:

Name	Runs
Yadier Molina	16
Ivan Rodriguez	11
Carlos Ruiz	10
Matt Wieters	9
Humberto Quintero	9
Lou Marson	6
Henry Blanco	5
Yorvit Torrealba	5
Ramon Hernandez	5
Brian McCann	4

Dang, Yadier. Looks like Pudge still has it, and it's nice to see a kid with so much offensive potential in Wieters score so well in his defensive ratings.

The trailers:

Name	Runs
Chris Iannetta	-4
John Hester	-5
Kevin Cash	-5
Victor Martinez	-6
Mike Napoli	-6
Adam Moore	-8
Jeff Mathis	-8
Ryan Doumit	-9
Jorge Posada	-10
Francisco Cervelli	-10

Mathis is considered to be the "defensive specialist" if I remember right, so it's a bit odd to see him do so poorly. The Yankees were horrendous behind the dish last season, so switching to Russell Martin (+2) as the primary backstop could give them an extra win based on just the simple categories used. For a relatively small signing, it could have a modest impact for the Yanks.

All in all, there’s a 2.8 win difference between the best and worst catcher, which is pretty large. Of course, that’s not accounting for the intangibles, so it’s pretty apparent that the quality of defense behind the plate—even when you’re not paying attention to framing, pitch sequencing, etc.—is pretty important.

Yeah, yeah, I know...I promised you all a spreadsheet. So here it is- I hope you enjoy.

UPDATE (1/9): I've put WP and PB together (averaging the run value as well). The spreadsheet has been updated.


*As I already mentioned, naturally, pitchers can and do affect a catcher’s CS and WP/PB rates. Of course, if one is interested in estimating the catcher’s true talent, you’ll have to regress each component to the mean. I imagine this would help sift out the impact of the pitcher by at least a little bit.

Posted by: JT Jordan

July 28, 2010

A very important article on fielding

Baseball Prospectus has published an article by Colin Wyers today that may be one of the most important pieces written about fielding measurement in the last decade. The full piece is available only to BP subscribers, but let me briefly recap some of the topics Colin covers.

Colin reiterates the point that uncertainty in fielding measurements is something that can be tackled with bigger sample sizes, i.e., more season of data. Bias, on the other hand, is persistent. It does not decrease with larger sample sizes of fielding data. He mentions two types of bias: that related to park/scorer and that related to the fielder's range.

He then outlines a clever method for using data like putouts and assists in order to develop a fielding metric for infielders that should be much less subject to those two sources of bias than our current advanced metrics like Ultimate Zone Rating (UZR), Plus/Minus, and TotalZone. His metric very likely has greater uncertainty than the advanced fielding metrics that use ball-in-play data to determine which fielder had the best chance to field a batted ball. However, at some point, larger sample sizes should decrease the effect of the uncertainty, such that the reduction in bias using Colin's method will actually produce more accurate measures of fielding. Is Colin's method better after two seasons? Three seasons? Five seasons? Because we don't yet know the size of the park-scorer bias or range bias, we don't know exactly at what point that occurs.

Colin gives some fielding numbers from his system for shortstops, and with them, margins of error! That in itself is a very important advancement. He also shows that the advanced fielding metrics appear to compress the measure of Ozzie Smith's fielding value by about 25% over his career.

As I mentioned in the comments to Colin's article:

Colin, as I mentioned on Twitter, can you use these numbers to estimate the magnitude of range bias for various advanced fielding systems (and at various positions)? Over a large sample of players, the park-scorer bias should become much less important.

If the ~70 run difference for Ozzie Smith is due to range bias, and 1 play = 0.8 runs, and Ozzie played about the equivalent of 17 seasons, then 70 / 0.8 / 17 = about 5 runs per season due to range bias.

If we apply the same method to a large group of players, we ought to be able to estimate the range bias.

Colin showed that the margin of error in his system for a full season of fielding by a shortstop was around 20 runs. Since random errors add quadratically, that means that the margin of error for three seasons of shortstop data would be around 35 runs, or 12 runs per season.

If we guess that advanced metrics can cut this uncertainty in half, that puts them at six runs of uncertainty per season on three seasons worth of data, plus whatever bias they may have. At what sample size does that bias become bigger than the improvement in the uncertainty from using subjective ball-in-play data? It varies according to the player, for one thing, but my crude guesses suggest that maybe for anything in the neighborhood of more than two to four seasons, Colin's method could be superior.

Not to go too far down that road, because there's a lot of work to be done yet, but hopefully that shows why I am excited about what Colin has published today.
Posted by: Mike Fast

July 16, 2010

Can we objectively evaluate advanced fielding data?

Colin has an article at Baseball Prospectus today that is required Sabermetrics 101 reading. It's been linked already in the THTLinks Twitter feed this morning, but it deserves more prominence and more discussion. This is an article of fundamental importance for the baseball analysis community. Anyone who is evaluating fielding, which is almost everyone in these heady days of Wins Above Replacement (WAR) statistics, needs to read and understand what Colin is saying.

He states an approach that should have been tackled by the analytical community before advanced defensive metrics started to gain such widespread acceptance. How did we ever come to accept such statistics without ever objectively testing them?

Now that they are being tested objectively, it should not surprise us that problems are being found. That does not invalidate the metrics. It is the path to knowledge. We are, after all, on a search for objective knowledge about baseball, are we not? Openly and objectively testing defensive metrics is not the quest of those who want to destroy baseball knowledge, as some will tell you when this topic is broached. It is a path well-worn by sabermetric pioneers, though "small is the gate and narrow the road that leads to life, and only a few find it."

We want to know what we know and why we know, when we trust it and when we don't trust it. We want to know what the sources and ranges of the errors are. This way lies improved fielding metrics and the ability to silence critics with facts that can be demonstrated in a way that is convincing, not one that demands blind faith that sabermetricians know what they are doing.

This also does not mean that we should stop using advanced fielding data today or that it has zero utility in objective sabermetrics. First and foremost, this is a clarion call to the community to turn its research efforts toward cracking this problem. Secondly, it's a wake-up call to understand and quantify the uncertainty in our measurements related to fielding, and to the derivative statistics like WAR, not a call to abandon them altogether.

Scientific inquiry has always operated in an environment of measurements made with uncertainty. This had led scientists to devote great effort to estimating the bounds of that uncertainty in order to determine their confidence in their measurements, and thus their confidence in the conclusions based on those measurements. There is no need to abandon the "science" of fielding measurement. Far from it. There is a need for the application of the time-tested sabermetric approach.

Doubt is not something to be feared. When its source is based on facts, doubt is healthy. Colin's doubt, which I share, is healthy. Let's take this opportunity as an analytical community and turn doubt into growth.
Posted by: Mike Fast

July 04, 2010

David Berner and the game-ending triple play

The bases were loaded in the bottom of the ninth inning. There were no outs.

David Berner, the Jethawks closer, leaned in for the sign from his catcher. He got it, and gave the tiny nod of acknowledgment. Then the tall, broad-shouldered southpaw went to his set stretch position. He checked the runners, he kicked into his motion, and he delivered the 1-0 pitch.

It was a fastball.

The Giants batter, big, strong right-handed-hitting James Simmons, wasn’t fooled. He didn’t try to pull the low-and-away location, but instead calmly stepped into the pitch and hit it squarely to the opposite field.

Had Simmons hit the ball any more squarely, the result would certainly have been a home run, a grand slam which would tie the score at 4-4. However, though Simmons hit the pitch very well, he got just slightly underneath it. He loudly smacked a towering fly ball, arcing high, tremendously high into the black midsummer night sky, and slicing toward the right field corner.
Click for more...

Posted by: Steve Treder

June 26, 2010

Red Sox opposition running game

Early in the year, Boston was being eaten alive by the opponents' base stealers. The Red Sox allowed 36 stolen bases in their first 16 games, with only one runner caught stealing. This was capped by the Texas Rangers stealing nine bases off of Tim Wakefield and Victor Martinez in the game on April 20.

After the Rangers nabbed five more bags in the remaining two games of the series, the Red Sox were on a pace to allow a record 365 stolen bases for 2010. That would have obliterated the old Retrosheet-era team record of 223 stolen bases allowed by the Red Sox in 2001, when Scott Hatteberg and Doug Mirabelli filled in at catcher for an injured Jason Varitek.

However, after that series in which the Rangers stole 14 bases in three games without getting caught, the Red Sox defensive effort against the running game has had much improved results. In the 60 games since April 22, the Red Sox have allowed 40 stolen bases, caught 18 runners stealing, and picked off an additional three baserunners. The contrast is impressive. I wish I knew what the Red Sox changed defensively.



The change does not seem to be associated with any one catcher or pitcher. It seems to have occurred across the team, suddenly and dramatically. Let's look at the catchers.

Victor Martinez  GS    SB    CS
Apr 4-22         12    25     1
Apr 23 - Jun 26  42    30     8

Jason Varitek    GS    SB    CS
Apr 4-22          4    11     0
Apr 23 - Jun 26  18    10     5

And the pitchers. (The line for the bullpen includes spot starters Scott Atchison and Felix Doubront.)

             April 4 - April 22        April 23 - June 26
Pitcher       G     SB    CS+PO         G     SB    CS+PO
Wakefield     3     10      0          10      7      2
Beckett       4      9      0           4      0      1
Lester        3      5      0          12      3      3
Buchholz      3      4      0           7      3      4
Lackey        3      2      1          12      9      4
Matsuzaka     0      0      0           8      8      4
Bullpen      16      6      0          60     10      3

I'd love to hear from any Boston fans who know whether the pitchers have been paying more attention to the runners at first base, throwing over more often, or coming quicker to the plate, and whether the catchers have changed anything.
Posted by: Mike Fast


Click here for more THT Notes.