Measuring defense for players back to 1956by Sean Smith
January 10, 2008
Last year, I figured out how to use Retrosheet’s files and build a play-by-play database. Soon after, I came up with a new defensive system that utilized only that play-by-play data: Knowing who ultimately fielded each hit, and whether the hit came on a fly ball, line drive, ground ball, or pop-up, I decided which fielder was truly responsible for the hit. I then combined that number with the fielder's plays made and errors to produce a zone-rating-type estimate that I called TotalZone.
At the time, the data that I needed for the ratings was available only for the years 2003 to 2006. Since then, I have run the ratings for 2007 as well. Unfortunately, for earlier years, Retrosheet often neglects to mention either the batted-ball type (usually for hits) or the fielder.
This was disappointing. I had created a pretty simple system that correlates well with Mitchel Lichtman's Ultimate Zone Rating (UZR), which in my opinion is the best system for evaluating defense. However, I could use my system only for 2003 on. Mitchel already has publicly posted UZR data for 2003 to 2006, plus John Dewan has a pretty good system in the Fielding Bible and Bill James Handbooks, and David Pinto has a good system of his own, PMR, on his site, Baseballmusings.com. When you own a Town Car, a Crown Victoria, and a Mark VIII, why would you drive a Ford Taurus?
What I really wanted was a system that works even when the data are incomplete. Tom Tango’s article in this year’s THT Annual, With or Without Derek Jeter, got me thinking. It's true—I occasionally don't know where a ball was hit, or whether it was on the ground or in the air. However, I do know certain other things. I know who threw the pitch. I know who hit the ball. And I know where the game was played. Could I use this information to fill in the blanks?
Covering the gaps
Let's lay the foundation. For infielders, I define "plays made" as ground balls that the infielder fields and turns into outs. Note that I consider only ground balls. Pop-ups are close to automatic outs and tell us little about fielding ability. Line drives are also excluded; although it takes more skill to catch a liner than to catch a pop-up, infield line-outs are rare, and it is difficult to determine the number of opportunities that the fielder had.
For outfielders, "plays made" equals put-outs. I do not include assists because I am attempting to measure the outfielder's range, not his arm. For both infielders and outfielders, I also count errors, in those cases where the "1st error player" in Retrosheet's database matches the position that I'm evaluating. I ignore plays made by pitchers and catchers.
Then comes the tough part: charging a hit to a fielder when we are missing the hit location or ball type. This is when I look at the batter. For each batter, I run a query showing the percentage of his outs that were recorded by each of the seven positions that I'm tracking.
For example, Rod Carew in his career made 4,288 outs that meet my definition of a play made by one of the seven fielders. Of these outs, 28 percent were made by the second baseman, 18 percent by the shortstop, and 3.5 percent by the right fielder. Thus, whenever Carew gets a hit but I don't know where, I charge 0.28 hits to the second baseman, 0.18 hits to the shortstop, and 0.035 hits to the right fielder. I repeat this routine for each such case.
I also factor in the pitcher who gave up the ball that became the hit. For each pitcher, I compute his groundball/flyball rate and compare it to the overall average. Thus, if Robin Roberts gets fly outs at a 20 percent higher rate than average, I pump up the outfielder fielding rate for the batters whom Roberts faces by 20 percent, and I reduce the fielding rate for the infielders behind him by the same number.
Tables of the reconstruction
Next, for each fielder, I combine his plays made, errors, and estimated hits allowed to get his total chances. I then divide plays made by total chances to get his rate of success. I compare each rate to the season average for the fielder's position to determine the number of plays made above or below average. Finally, I translate this difference to runs allowed, using figures of 0.75 runs per play for middle infielders, 0.80 runs for corner infielders, and 0.84 runs for outfielders.
For outfielders, I also apply a park factor. When I compiled the 2003-2007 ratings, I found that park factors make only a small difference among infielders; for outfielders, though, the park can make a huge difference. To get outfield park factors, I calculate the TotalZone for each combination of park and outfield position for both home and visiting players and then compare the number to the same rating for a team's games on the road. Assume that the rating for LF Fenway park is .61, and for Red Sox and their opponents on the road it is .67, and the league average is .65. An outfielder in Fenway has his individual rating compared to a positional average of .65 +(.61-.67) = 0.59. His road games are compared to the specific factor of the parks in which he played his road games.
(Note that, instead of single-year park factors, I calculated factors for the life of the team for these years. I guess that I really shouldn't call them "park factors"—they're closer to franchise factors. An improvement would be to reset the park factor when a team moves into a new stadium; that said, the most extreme factors are for long-standing parks, such as Fenway Park, Wrigley Field, and Yankee Stadium.)
With that work done, I have a rating that measures each fielder's range, from any year covered by Retrosheet. Certainly, this rating is not a complete measure of a player's defensive value—some important things are not included, such as an infielder's ability to start and turn double plays, a first baseman's ability to scoop bad throws, and an outfielder's throwing arm.
Nevertheless, after I crunched the numbers, I was pleasantly surprised. I ran the numbers not only for years 1956 to 1992 but also for 2000 to 2007, with the intent to compare the latter to the values from some of the better rating systems of today. I can also compare this version of TotalZone to the original version, which uses the type of batted ball and the fielder and which I will retroactively label as "TotalZone+" (TZ+).
Spot-checking the 2000-2007 data, I wanted to see how the players who are considered great or poor fielders by today’s systems fared in TZ, without the type or fielder data.
Darin Erstad, when he was able to play center field every day, was a UZR monster. His TotalZone ratings:
2000 CF +6, LF +18 2001 CF +11 2002 CF +36That last figure is very close to what I remember Erstad having for UZR. Looks good so far.
2003 +13 2004 +7 2005 +11 2006 +22 2007 +6 (missed half the season)Everett was closer to +40 in UZR for 2006, but my system does rate him as a great defensive shortstop.
2004 +17 2005 +10 2006 +9 2007 +16Another good sign.
2003 +12 2005 +10 2006 -7 2007 +23UZR has Ellis at +45 from 2003 to 2006. But the 2007 rating is a good sign. I'm not sure what happened in 2006.
2005 +2 2006 -12 2007 +5Here is the first miss. UZR has Utley at +30 for 2003-2006. For what it's worth, TZ+ has Utley at +16.
2003 +12 2004 +23 2005 +4 2006 +11 2007 -1For 2003-2006, UZR has Beltre at +53, almost a perfect match.
2000 -1 2001 +13 2002 -1 2003 +27 2004 +4 2005 0 2006 +10 2007 -1That’s +39 for TZ from 2003-2006 and +43 for UZR.
Now for some of the poorly rated fielders:
2000 -21 2001 -16 2002 -17 2003 -16 2004 -8 2005 +1 2006 -9 2007 -16That’s -32 for TZ for 2003-06, -44 for UZR.
2003 -2 2004 -1 2005 -9 2006 -8 2007 +3That’s -20 for 2003-06. TZ+ has Ramirez at -22, UZR at -97, and Zone Rating (which includes no park adjustment) at -107. My numbers are not even close; let’s just say that I don’t think we’re using the same park adjustment.
2003 -6 (missed most of season) 2004 -9 2005 -22 2006 -16 2007 -7 (merciful move to RF)UZR has a much more extreme value, -113, but we agree that Griffey was really, really bad in center and not so great in right, either.
2005 2B -10 3B -9 2006 2B -14UZR has Cantu at -38 2B, -21 3B.
There are some weird results too (all numbers again are from 2003 to 2006). TZ has Richie Sexson slightly ahead of Doug Mientkiewicz; UZR puts them at -29 and +37 respectively. TZ has Gary Matthews Jr., Vernon Wells, and Juan Pierre at -16, -20, and -29, whereas TZ+ rates them all close to average and UZR has Matthews +33, Wells +46, and Pierre +32.
When the infielders are off, I assume that it’s just the error of an imprecise system and that UZR has the right value. For outfielders, though, I’m not so sure. Although UZR is the better system, the discrepancy could also be due to differences in data sources. Some of the outfielders, such as Grady Sizemore, Ichiro, Andruw Jones, and Matthews, rate drastically different depending on whether you are looking at UZR (which uses STATS as its source) or John Dewan’s plus/minus or David Pinto’s PMR, both of which use Baseball Info Solutions.
Now let's look at all players with 500 or more chances:
|Position||Average Chances||TotalZone||TotalZone+||STATS Zone Rating|
The first thing I notice is the close fit of Zone Rating to UZR. TotalZone+, using batted ball type and simple hit location, is almost as good as Zone Rating, but not quite, and regular TotalZone is a little worse, though still a fairly strong correlation for infielders.
For outfielders, it's surprising that TZ is better than TZ+. Perhaps using hitter/pitcher match-ups better estimates outfielder chances than does using hit type and general hit location. A future project is to explore if combining batter/pitcher information with batted ball type can offer even better outfield ratings.
One problem with the above evaluation is that I'm assuming that UZR is truth and then judging a system on how well it compares to UZR. My opinion is that UZR is the best; however, I would not suggest that UZR is perfect, and it would be useful to compare all these systems to one that is independent of the fielding metrics.
Tango Tiger's scouting report by the fans seems to fit the bill. For each position, I compared the four defensive systems to his overall score from the 2006 scouting report.
|Position||Ultimate Zone Rating||TotalZone||TotalZone+||STATS Zone Rating|
Overall, the new TotalZone does just as well as UZR and Zone Rating in recognizing the fielders whom fans think are the best and worst. I'm not claiming that TZ should become the new standard, but I think it's pretty good. Let's see what TZ says about players from the past.
Pulling back the tarp
Without further ado, here are the highlights for years 1956 to 1986:
1st base: Keith Hernandez +120 (11 gold gloves, was the MVP in '79, he can do what he wants) George "Boomer" Scott +95 (8 GG) Vic Power +54 (7 GG in 11 years as a starter) ... Willie McCovey -51 (HoF career with 0 GG) Willie Stargell -55 (ditto) Dick Stuart -65 (Dr. Strangeglove) 2nd Base: Lou Whitaker +76 (3 GG) Frank White +71 (8 GG) Bill Mazeroski +70 (8 GG) Dick Green +70 ... Jorge Orta -69 Tony Taylor -71 3rd Base: Brooks Robinson +299 (+18 per year, 16 GG) Buddy Bell +181 (6 GG) Graig Nettles +154 (only 2 GG, bad timing) Clete Boyer +141 (1 GG, also bad timing) Mike Schmidt +139 (10 GG) ... Butch Hobson -67 Richie Hebner -74 Bill Madlock -86 Shortstop: Mark Belanger +251 (8 GG) Luis Aparicio +142 (9 GG) Ozzie Smith +133 (only half of Ozzie's career. 13 GG) ... Don Buddin -53 Frank Taveras -59 Left field: Carl Yastrzemski +115 (7 GG) Jose Cruz Sr. +83 Roy White +78 Willie Wilson +69 ... Sarge Matthews -83 Greg Luzinski -94 Center Field: Paul Blair +157 (8 GG) Willie Mays +117 (12 GG) Curt Flood +112 (7 GG) ... Rick Monday -78 Matty Alou -85 Right Field: Al Kaline +104 (10 GG) Tony Oliva +80 (1 GG) Roberto Clemente +59 (12 GG) ... Jeff Burroughs -57 Claudell Washington -58 Rusty Staub -64——
References and Resources
The information used here was obtained free of charge from and is copyrighted by Retrosheet. Interested parties may contact Retrosheet at 20 Sunset Rd., Newark, DE 19711.
Mitchell Lichtman has made UZR for years 2003 to mid 2007 available on the web. You can download the files from here.
Zone rating is updated daily during the baseball season and is reported on several websites. I usually use CNN/Sports Illustrated.
The Fan's Scouting Report is run by Tango Tiger; you can read more about it on his site. If you haven't already been doing so, visit the site next year and cast a ballot for your favorite team.
Click here to download the career and single-season ratings for all players from 1956 to 1986.
Sean Smith is a lifelong Angels fan despite never visiting the west coast until April 2006. His work can also be found at baseballprojection.com and Anaheim Angels All the Way and he can be contacted by email.