A zone of their own

Even though baseball’s general managers voted last week overwhelmingly in favor of instant replay, the human element is still alive and well in baseball officiating. Home plate umpires make about 150 decisions a game, about which no argument is even allowed. They do a pretty good job—according to Mike Port, the vice president of umpiring, 95.4 of all balls and strikes this season were called correctly. However, the zone they call is significantly different from the rulebook definition and therefore open to interpretation. Which raises the question: Is there a big difference between one umpire and the next, and how much of an effect can that have on a game?

To look into this, I used Pitch f/x data to call every pitch (about half of the pitches thrown in 2007 were tracked) over the season either a ball or a strike, using the strike zone as calculated by John Walsh in his articles, The Strike Zone: Fact vs. Fiction and The Eye of the Umpire. I then compared these calls to the actual calls made by each umpire. I’m not trying to determine how many “mistakes” umpires are making because having a different strike zone isn’t necessarily a problem as long as it’s called consistently.

Instead, we can compare each umpire to the league average to get an idea of his tendencies. This also has the advantage of eliminating any errors the pitch f/x might be making, because it’s just comparing umpires to each other rather than assuming that the system is perfect. Over the thousands of calls measured last season, any errors the system made will be evenly distributed and drop out.

“Extra” Strikes/Balls

I define an extra strike as when an umpire calls a strike that the Pitch f/x system would have called a ball, and vice-versa for an extra ball. Subtract the extra balls from the extra strikes that an umpire makes, divide by the total number of calls he made over the season, and the result is how much (and which way) each umpire is weighted.

Then to put the number in a form that is easily digestible, I’ve expressed each umpires score in terms of the league average and multiplied by 150 (calls/game). The result is a +/- value for how many more strikes a particular umpire would call compared over a complete game as compared to the league average (strikes above average or SAA). Negative numbers mean that the umpire would call that many more balls than strikes.

Enough definitions—here are the 10 largest and smallest zones in baseball: (minimum of 1000 pitches)

Smallest Strike Zone

Name of Umpire            SAA
Gerry    Davis           -5.20
Paul     Schrieber       -5.02
Dana     DeMuth          -4.03
Randy    Marsh           -3.57
Chuck    Meriwether      -3.25
Sam      Holbrook        -3.10
Chad     Fairchild       -2.89
Larry    Poncino         -2.79
Greg     Gibson          -2.73
Larry    Young           -2.69

Largest Strike Zone

Name                      SAA
Jeff     Nelson           4.81
Doug     Eddings          4.70
Charlie  Reliford         4.05
Phil     Cuzzi            3.98
Paul     Nauert           3.59
Jim      Wolf             3.36
Bill     Welke            2.86
Mark     Wegner           2.77
Brian    Runge            2.70
Tom      Hallion          2.64

Note that there is a big difference between the two extremes: Jeff Nelson calls about 10 more strikes than Gerry Davis in an average game. That’s even taking into account that hitters are no doubt aware of this (or figure it out during the game) and adjust by hacking away or being more selective, as the case may be.

But are these numbers relatively consistent from one game to the next or just caused by random fluctuations? Here’s a graph of five umpires from the largest and smallest 10 zones showing their SAA’s in each quarter of the season (anything smaller than 4-5 games is too small of a sample).

image

There is a fair amount of fluctuation in each umpire from one period to the next (over a relatively small sample size). However, there is absolutely no crossover between the two groups, implying that the difference between both sides of the spectrum is consistent. Don’t count on catching Gerry Davis on a good day!

OK, big deal. We already could look at strikeout and walk rates to get a reasonable idea of which umpires have generous or stingy strike zones. But now we can also use this method to break things down based on the four directions of the strike zone, and figure out who calls the high/low/left/right strike.

For the directional breakdown that follows, I’m only considering pitches that are not within four inches of the edge of the strike zone in each direction, because there’s no real way of knowing if something is being called “high” or “outside” if it’s on a corner. So the numbers for the high strike consider all pitches that are on the upper half of the plate and that no umpire in their right mind would call a ball wide. Again, this is the number of extra strikes over 150 pitches, not the number of pitches that will be called in that particular location per game (which of course will differ greatly from pitcher to pitcher).

Right Side of Plate (From Catcher’s perspective)

Smallest Right Side Strike Zone

Name of Umpire            SAA
Chad     Fairchild       -3.87
Mike     Everitt         -3.44
Paul     Schrieber       -3.05
Bill     Miller          -2.78
Ron      Kulpa           -2.22
Gerry    Davis           -2.17
Jerry    Meals           -2.09
Marty    Foster          -2.08
Angel    Hernandez       -2.04
Hunter   Wendelstedt     -1.98

Largest Right Side Strike Zone

Name of Umpire            SAA
Jeff     Nelson           4.01
Rob      Drake            3.61
Mike     Winters          2.98
James    Hoye             2.94
Tim      Tschida          2.82
Bruce    Dreckman         2.77
Brian    Runge            2.55
Wally    Bell             2.20
Ed       Hickox           2.04
Rick     Reed             2.00

Left off the list due to a small sample size is a number over 10 for Travis Reininger on the right side. There was only data collected this season for about 4 of his games this season. Still, over that time, he gave twice as many extra strikes to the right side than any other umpire, while he was average on the other side of the plate.

A Hardball Times Update
Goodbye for now.
Left Side of Plate (From Catcher’s Perspective)

Smallest Left Side Strike Zone

Name of Umpire            SAA
Larry    Vanover         -5.86
Randy    Marsh           -5.57
Greg     Gibson          -5.36
Fieldin  Culbreth        -4.83
Dana     DeMuth          -3.78
Angel    Hernandez       -3.69
Alfonso  Marquez         -3.20
Bruce    Froemming       -2.98
Tim      McClelland      -2.90
Sam      Holbrook        -2.83

Largest Left Side Strike Zone

Name of Umpire            SAA
Ed       Hickox           4.09
Doug     Eddings          3.97
Laz      Diaz             3.86
Bill     Miller           3.84
Bill     Welke            3.66
Lance    Barksdale        3.56
Paul     Nauert           3.00
Tom      Hallion          2.85
Mark     Carlson          2.84
Wally    Bell             2.74

Note that for both small and large strike zones, there is a greater spread in the numbers to the left side, which means there is a greater range in how umpires call that edge of the plate. Considering the nature of the real strike zone as calculated by Walsh (the zone is larger on the outside half side by several inches for lefties, but symmetrical for righties), it would make sense that umpires disagree a lot over whether or not to give the extra outside calls to lefties (and looking at the splits confirms this).

One reason for this could be that most umpires today set up in the “slot” position between the batter and the catcher. The only weakness of this position is that it is hard to see a pitch low and away. One of the things that can help an umpire when he can’t see a pitch is how much the catcher’s arm moves, or his elbow pops out to catch a pitch. But since catchers are almost universally right-handed, with a left-handed hitter up the umpire is set-up on the other side of the catcher’s body, which would eliminate that cue for the outside pitch for umpires that rely on it.

Bottom Edge

Smallest Low Strike Zone

Name of Umpire            SAA
Larry    Young           -8.88
Chuck    Meriwether      -8.08
Sam      Holbrook        -7.83
Larry    Poncino         -7.26
Dana     DeMuth          -6.79
Dan      Iassogna        -6.23
James    Hoye            -6.05
Wally    Bell            -4.61
Mike     Winters         -4.52
Derryl   Cousins         -4.35

Largest Low Strike Zone

Name of Umpire            SAA
Charlie  Reliford         8.73
Phil     Cuzzi            7.05
Paul     Nauert           6.63
Tim      McClelland       5.66
Lance    Barksdale        5.39
Gary     Darling          5.17
Jeff     Nelson           5.08
Jim      Wolf             4.89
Jim      Reynolds         4.46
CB       Bucknor          4.43

Upper Edge

Smallest High Strike Zone

Name of Umpire          SAA
Chuck  Meriwether      -8.53
Ed     Rapuano         -7.05
Tim    Timmons         -7.02
Jerry  Layne           -6.79
Larry  Vanover         -6.35
Brian  Runge           -5.68
Marty  Foster          -5.21
Chad   Fairchild       -5.04
Ed     Montague        -4.53
Mark   Carlson         -4.52

Largest High Strike Zone

Name of Umpire          SAA
Gary   Darling          8.34
Dan    Iassogna         8.22
Eric   Cooper           7.52
Jeff   Nelson           7.44
Sam    Holbrook         5.61
Jim    Wolf             3.72
Gary   Cederstrom       3.64
Brian  Knight           3.51
John   Hirschbeck       3.42
Ted    Barrett          3.33

Again, the higher numbers show that there’s a lot more disagreement in the vertical strike zone among umpires than there is horizontally. That could be because it’s a lot harder to be consistent when the vertical height of the strike zone is always changing, as opposed to the edges of the plate. However, some of this effect could also be due to the inconsistency of the measurements the pitch f/x operators are taking when measuring the top and bottom of the batter’s strike zone.

Highlights

Jeff Nelson: The most pitcher-friendly umpire in 2007. He calls the vertical strike zone much closer to the rulebook definition. He also is willing to give the outside corner against right-handed hitters, the classic “Glavine” call a couple inches off the plate.

Paul Schrieber: Favors hitters second-most in the league, but has a normal sized zone on the left side of the plate. Right-handed hitters must love him—he is very stingy on both the high or low strike (under -3 for both) and doesn’t call the outer half of the plate on them. When he’s behind the plate, pitchers are going to have to come inside and keep it closer to the belt.

Gerry Davis: Had the smallest zone in the league. His strike zone is universally small, although more normal when calling the high strike.

Charlie Reliford: Has one of the largest strike zones in the league because he calls the left hand side of the plate extremely generously, but is average on the right side of the plate. Also calls the low strike.

Chuck Meriwether: Is known for a tight zone. That’s because it’s incredibly small from top to bottom—even further than the couple of inches that most umpires take off the rulebook definition. Although he is similar to Reliford from side-to-side (he very slightly prefers the left side of the plate), he has the smallest high zone and the second-smallest low zone in the league.

Ed Hickox: Has the largest horizontal strike zone in the league. He compensates by having a small upper zone and a normal lower one. This favors control pitchers if they can spot the ball on the outside corner, but any pitcher that needs to throw high heat is in trouble.

Joe West: Although I’ve tried to avoid casting judgment on the quality of umpires for now, Joe gets my top vote. He is very close to the league average in every direction, but doesn’t give the pitch off the left side like most of the league. He also had the fewest number of extra balls and strikes (as opposed to having a bunch that cancel out), which is a sign of consistency.

Final Thoughts

Roger Clemens was well known for taking rigorous notes and studying umpires—when they can differ as much as this, you’d think that every pitcher would. Umpires don’t just have big or small zones, they have very specific preferences for the four edges of the strike zone and are rarely simply “big” or “small”. Often an ump expands the zone one way but plays it by the book in the other directions.

As I was watching the playoffs and tinkering with this data, I was amazed at how consistent each umpire’s tendencies were. Remember when Victor Martinez started yelling at the home plate umpire over calls to Fausto Carmona? No wonder: Dana Demuth has the fourth smallest zone in the league where those pitches were called, on the left side of the plate.

I used to think that an umpire’s zone was largely negotiated by the pitcher during the game and could differ greatly from one day to the next. Now I’m sure there are some very consistent tendencies that each umpire sticks to (although they may not come up in every game). The next step is to look at is which umps are the most consistent and if they are affected when dealing with particular situations, teams, or players.

References & Resources
You can download the entire dataset in this Excel file.


Comments are closed.