Even though baseball’s general managers voted last week overwhelmingly in favor of instant replay, the human element is still alive and well in baseball officiating. Home plate umpires make about 150 decisions a game, about which no argument is even allowed. They do a pretty good job—according to Mike Port, the vice president of umpiring, 95.4 of all balls and strikes this season were called correctly. However, the zone they call is significantly different from the rulebook definition and therefore open to interpretation. Which raises the question: Is there a big difference between one umpire and the next, and how much of an effect can that have on a game?
To look into this, I used Pitch f/x data to call every pitch (about half of the pitches thrown in 2007 were tracked) over the season either a ball or a strike, using the strike zone as calculated by John Walsh in his articles, The Strike Zone: Fact vs. Fiction and The Eye of the Umpire. I then compared these calls to the actual calls made by each umpire. I’m not trying to determine how many “mistakes” umpires are making because having a different strike zone isn’t necessarily a problem as long as it’s called consistently.
Instead, we can compare each umpire to the league average to get an idea of his tendencies. This also has the advantage of eliminating any errors the pitch f/x might be making, because it’s just comparing umpires to each other rather than assuming that the system is perfect. Over the thousands of calls measured last season, any errors the system made will be evenly distributed and drop out.
I define an extra strike as when an umpire calls a strike that the Pitch f/x system would have called a ball, and vice-versa for an extra ball. Subtract the extra balls from the extra strikes that an umpire makes, divide by the total number of calls he made over the season, and the result is how much (and which way) each umpire is weighted.
Then to put the number in a form that is easily digestible, I’ve expressed each umpires score in terms of the league average and multiplied by 150 (calls/game). The result is a +/- value for how many more strikes a particular umpire would call compared over a complete game as compared to the league average (strikes above average or SAA). Negative numbers mean that the umpire would call that many more balls than strikes.
Enough definitions—here are the 10 largest and smallest zones in baseball: (minimum of 1000 pitches)
Smallest Strike Zone
Name of Umpire SAA Gerry Davis -5.20 Paul Schrieber -5.02 Dana DeMuth -4.03 Randy Marsh -3.57 Chuck Meriwether -3.25 Sam Holbrook -3.10 Chad Fairchild -2.89 Larry Poncino -2.79 Greg Gibson -2.73 Larry Young -2.69
Largest Strike Zone
Name SAA Jeff Nelson 4.81 Doug Eddings 4.70 Charlie Reliford 4.05 Phil Cuzzi 3.98 Paul Nauert 3.59 Jim Wolf 3.36 Bill Welke 2.86 Mark Wegner 2.77 Brian Runge 2.70 Tom Hallion 2.64
Note that there is a big difference between the two extremes: Jeff Nelson calls about 10 more strikes than Gerry Davis in an average game. That’s even taking into account that hitters are no doubt aware of this (or figure it out during the game) and adjust by hacking away or being more selective, as the case may be.
But are these numbers relatively consistent from one game to the next or just caused by random fluctuations? Here’s a graph of five umpires from the largest and smallest 10 zones showing their SAA’s in each quarter of the season (anything smaller than 4-5 games is too small of a sample).
There is a fair amount of fluctuation in each umpire from one period to the next (over a relatively small sample size). However, there is absolutely no crossover between the two groups, implying that the difference between both sides of the spectrum is consistent. Don’t count on catching Gerry Davis on a good day!
OK, big deal. We already could look at strikeout and walk rates to get a reasonable idea of which umpires have generous or stingy strike zones. But now we can also use this method to break things down based on the four directions of the strike zone, and figure out who calls the high/low/left/right strike.
For the directional breakdown that follows, I’m only considering pitches that are not within four inches of the edge of the strike zone in each direction, because there’s no real way of knowing if something is being called “high” or “outside” if it’s on a corner. So the numbers for the high strike consider all pitches that are on the upper half of the plate and that no umpire in their right mind would call a ball wide. Again, this is the number of extra strikes over 150 pitches, not the number of pitches that will be called in that particular location per game (which of course will differ greatly from pitcher to pitcher).
Right Side of Plate (From Catcher’s perspective)
Smallest Right Side Strike Zone
Name of Umpire SAA Chad Fairchild -3.87 Mike Everitt -3.44 Paul Schrieber -3.05 Bill Miller -2.78 Ron Kulpa -2.22 Gerry Davis -2.17 Jerry Meals -2.09 Marty Foster -2.08 Angel Hernandez -2.04 Hunter Wendelstedt -1.98
Largest Right Side Strike Zone
Name of Umpire SAA Jeff Nelson 4.01 Rob Drake 3.61 Mike Winters 2.98 James Hoye 2.94 Tim Tschida 2.82 Bruce Dreckman 2.77 Brian Runge 2.55 Wally Bell 2.20 Ed Hickox 2.04 Rick Reed 2.00
Left off the list due to a small sample size is a number over 10 for Travis Reininger on the right side. There was only data collected this season for about 4 of his games this season. Still, over that time, he gave twice as many extra strikes to the right side than any other umpire, while he was average on the other side of the plate.
Left Side of Plate (From Catcher’s Perspective)
Smallest Left Side Strike Zone
Name of Umpire SAA Larry Vanover -5.86 Randy Marsh -5.57 Greg Gibson -5.36 Fieldin Culbreth -4.83 Dana DeMuth -3.78 Angel Hernandez -3.69 Alfonso Marquez -3.20 Bruce Froemming -2.98 Tim McClelland -2.90 Sam Holbrook -2.83
Largest Left Side Strike Zone
Name of Umpire SAA Ed Hickox 4.09 Doug Eddings 3.97 Laz Diaz 3.86 Bill Miller 3.84 Bill Welke 3.66 Lance Barksdale 3.56 Paul Nauert 3.00 Tom Hallion 2.85 Mark Carlson 2.84 Wally Bell 2.74
Note that for both small and large strike zones, there is a greater spread in the numbers to the left side, which means there is a greater range in how umpires call that edge of the plate. Considering the nature of the real strike zone as calculated by Walsh (the zone is larger on the outside half side by several inches for lefties, but symmetrical for righties), it would make sense that umpires disagree a lot over whether or not to give the extra outside calls to lefties (and looking at the splits confirms this).
One reason for this could be that most umpires today set up in the “slot” position between the batter and the catcher. The only weakness of this position is that it is hard to see a pitch low and away. One of the things that can help an umpire when he can’t see a pitch is how much the catcher’s arm moves, or his elbow pops out to catch a pitch. But since catchers are almost universally right-handed, with a left-handed hitter up the umpire is set-up on the other side of the catcher’s body, which would eliminate that cue for the outside pitch for umpires that rely on it.
Smallest Low Strike Zone
Name of Umpire SAA Larry Young -8.88 Chuck Meriwether -8.08 Sam Holbrook -7.83 Larry Poncino -7.26 Dana DeMuth -6.79 Dan Iassogna -6.23 James Hoye -6.05 Wally Bell -4.61 Mike Winters -4.52 Derryl Cousins -4.35
Largest Low Strike Zone
Name of Umpire SAA Charlie Reliford 8.73 Phil Cuzzi 7.05 Paul Nauert 6.63 Tim McClelland 5.66 Lance Barksdale 5.39 Gary Darling 5.17 Jeff Nelson 5.08 Jim Wolf 4.89 Jim Reynolds 4.46 CB Bucknor 4.43
Smallest High Strike Zone
Name of Umpire SAA Chuck Meriwether -8.53 Ed Rapuano -7.05 Tim Timmons -7.02 Jerry Layne -6.79 Larry Vanover -6.35 Brian Runge -5.68 Marty Foster -5.21 Chad Fairchild -5.04 Ed Montague -4.53 Mark Carlson -4.52
Largest High Strike Zone
Name of Umpire SAA Gary Darling 8.34 Dan Iassogna 8.22 Eric Cooper 7.52 Jeff Nelson 7.44 Sam Holbrook 5.61 Jim Wolf 3.72 Gary Cederstrom 3.64 Brian Knight 3.51 John Hirschbeck 3.42 Ted Barrett 3.33
Again, the higher numbers show that there’s a lot more disagreement in the vertical strike zone among umpires than there is horizontally. That could be because it’s a lot harder to be consistent when the vertical height of the strike zone is always changing, as opposed to the edges of the plate. However, some of this effect could also be due to the inconsistency of the measurements the pitch f/x operators are taking when measuring the top and bottom of the batter’s strike zone.
Jeff Nelson: The most pitcher-friendly umpire in 2007. He calls the vertical strike zone much closer to the rulebook definition. He also is willing to give the outside corner against right-handed hitters, the classic “Glavine” call a couple inches off the plate.
Paul Schrieber: Favors hitters second-most in the league, but has a normal sized zone on the left side of the plate. Right-handed hitters must love him—he is very stingy on both the high or low strike (under -3 for both) and doesn’t call the outer half of the plate on them. When he’s behind the plate, pitchers are going to have to come inside and keep it closer to the belt.
Gerry Davis: Had the smallest zone in the league. His strike zone is universally small, although more normal when calling the high strike.
Charlie Reliford: Has one of the largest strike zones in the league because he calls the left hand side of the plate extremely generously, but is average on the right side of the plate. Also calls the low strike.
Chuck Meriwether: Is known for a tight zone. That’s because it’s incredibly small from top to bottom—even further than the couple of inches that most umpires take off the rulebook definition. Although he is similar to Reliford from side-to-side (he very slightly prefers the left side of the plate), he has the smallest high zone and the second-smallest low zone in the league.
Ed Hickox: Has the largest horizontal strike zone in the league. He compensates by having a small upper zone and a normal lower one. This favors control pitchers if they can spot the ball on the outside corner, but any pitcher that needs to throw high heat is in trouble.
Joe West: Although I’ve tried to avoid casting judgment on the quality of umpires for now, Joe gets my top vote. He is very close to the league average in every direction, but doesn’t give the pitch off the left side like most of the league. He also had the fewest number of extra balls and strikes (as opposed to having a bunch that cancel out), which is a sign of consistency.
Roger Clemens was well known for taking rigorous notes and studying umpires—when they can differ as much as this, you’d think that every pitcher would. Umpires don’t just have big or small zones, they have very specific preferences for the four edges of the strike zone and are rarely simply “big” or “small”. Often an ump expands the zone one way but plays it by the book in the other directions.
As I was watching the playoffs and tinkering with this data, I was amazed at how consistent each umpire’s tendencies were. Remember when Victor Martinez started yelling at the home plate umpire over calls to Fausto Carmona? No wonder: Dana Demuth has the fourth smallest zone in the league where those pitches were called, on the left side of the plate.
I used to think that an umpire’s zone was largely negotiated by the pitcher during the game and could differ greatly from one day to the next. Now I’m sure there are some very consistent tendencies that each umpire sticks to (although they may not come up in every game). The next step is to look at is which umps are the most consistent and if they are affected when dealing with particular situations, teams, or players.
References & Resources
You can download the entire dataset in this Excel file.