The Commissioner Speaks: Imagining a Redefined Strike Zone

Commissioner Rob Manfred has been keeping an eye on the strike zone this year. (via Arturo Pardavila III)

Commissioner Rob Manfred has been keeping an eye on the strike zone this year. (via Arturo Pardavila III)

Before the season, Yahoo’s Jeff Passan reported that the strike zone would be monitored this year by the MLB Playing Rules Committee. With run scoring falling to its lowest level since 1981, the committee was thought to have the potential to recommend a change to the rulebook strike zone as early as 2016 if it deemed such a move prudent for the welfare of the game.

Eno Sarris recently had the opportunity to ask new Commissioner Rob Manfred about the strike zone, and this is what he had to say:

It’s a work in progress (referring to fact that the zone got smaller and then bigger over the course of the first two months). First of all, all of these changes you’re talking about are minuscule in terms of magnitude. Having said that, there has been absolutely no direction given to the umpires this year that is different from what we’ve given them the last few years, which is to call the strike zone consistent to the rules. Our focus with respect to the strike zone has not been on changing it in any way; to the contrary, it has been on making it more consistent across umpires. We think we’ve made progress in that regard.

Conceptually — that was the facts, what’s out there — conceptually, I would be reluctant, even if someone said you have X problem, and the way to fix it is to give instructions on changing the strike zone, I would be reluctant to do that. Because when you talk to baseball people, baseball people you respect, and you say to them what happens if I do this with the strike zone, you don’t get any consistent response. I am very disciplined to make changes with respect to the play of the game on the field where we don’t have a pretty good idea of what the outcome is going to be.”

There is a lot of information in this quote from the commissioner, so let’s examine his statements piece by piece.

Magnitude of Changes

It’s a work in progress (referring to fact that the zone got smaller and then bigger over the course of the first two months). First of all, all of these changes you’re talking about are minuscule in terms of magnitude.”

When I first measured the 2015 season strike zone near the end of April, both its overall size and the band at the bottom of the zone where expansion clearly has been visible had both contracted ever so slightly. By the time the end of May had rolled around, the strike zone had filled out to narrowly exceed the zone measured last season.

Measuring now at the midway point in the season shows results similar to last check.

Strike Zone Data, 2009-2015
Year Strike Zone Size (sq. in) Strike Zone Size Below 21” (sq. in) K% R/G
2009 435 0 18.0% 4.61
2010 436 6 18.5% 4.38
2011 448 11 18.6% 4.28
2012 456 19 19.8% 4.32
2013 459 30 19.9% 4.17
2014 475 47 20.4% 4.07
2015* 475 49 20.2% 4.12
* Games until the end of June, Source: PITCHf/x via Baseball Heat Maps

I would agree that the magnitude of the changes seen in these measurements is small when placed up against the 2014 numbers. The overall size is identical, with the size of the zone below 21 inches only 2 square inches larger. What I would claim to be the larger potential issue is that the size and shape of the strike zone in today’s game is drastically different from where it was five or six years ago. I’ll return to this subject a little later. Let’s continue parsing commissioner Manfred’s quote.

Direction Given to Umpires

Having said that, there has been absolutely no direction given to the umpires this year that is different from what we’ve given them the last few years, which is to call the strike zone consistent to the rules.”

This statement matches the sentiment reported by Jeff Passan (linked above), namely that the MLB Playing Rules Committee “will pay close attention to the size of the strike zone in 2015 with an eye on change as early as 2016.” There is no reason to think umpires have been asked to call the strike zone any differently in 2015 than they were in 2014, and this season’s zone as I measure it looks very similar to last season’s zone.

Consistency Across Umpires

Our focus with respect to the strike zone has not been on changing it in any way, to the contrary, it has been on making it more consistent across umpires. We think we’ve made progress in that regard.”

Fellow strike zone researcher Brian Mills has done excellent work in the area of home plate umpire ball/strike calling accuracy and consistency. In the introduction of his peer-reviewed journal article, “Expert Workers, Performance Standards, and On-the-Job Training: Evaluating Major League Baseball Umpires,” he writes this about the state of umpiring accuracy during the PITCHf/x era.

Additionally, umpires have shown substantial improvement in their ball-strike call accuracy. In 2007, umpires correctly called only 76.8% of pitches within the rulebook zone and 87.4% of pitches outside the rulebook zone based on the Sportvision data. However, by 2013, umpires were correctly calling 85.7% and 90.0% of pitches within and outside the zone, respectively.”

Using the zone definitions provided in the Gameday data, I compared umpire pitch calling accuracy over our period of study.

Umpire Pitch Calling Accuracy, 2009-2015
Year % Balls called Strikes % Strikes called Balls
2009 14.9% 15.6%
2010 14.9% 12.3%
2011 14.5% 12.2%
2012 14.4% 11.3%
2013 14.1% 10.1%
2014 14.3% 9.5%
2015* 14.3% 9.6%
* Games until the end of June, Source: PITCHf/x via Baseball Savant

The results agree with Mills’ assessment that pitch calling accuracy is trending in the right direction.

If you’re surprised at how high the percentages are for incorrect calls, I believe the main culprit at play is the way that the zone is called differently depending on game state. While I believe Gameday uses constant zone definitions for all pitches in a given season, many researchers (including myself) have shown that the zone expands and contracts based on game state, in particular by count. As a quick example, if I look at only pitches on 0-0 counts from 2014, the percentage of balls called strikes jumps to 20.2 percent, but the percentage of strikes called balls drops to 7.5 percent. This is consistent with the previous research which shows a 0-0 count is pitcher-friendly as far as the size of the strike zone is concerned.

In Table 2 of his paper, Mills calculates a Weighted Coefficient of Variation (wCV), his measure of strike variability between umpires by season. A lower wCV is indicative of lower variation among umpires. As his paper was written in 2014, he calculated this measure for seasons up to and including 2013. After reproducing his 2013 value, I calculated the same measure for 2014.

Umpires’ Weighted Coefficient of Variation (wCV), 2009-2014
Year wCV % of Pitches to New Umpires
2009 0.01232 2.6%
2010 0.01183 1.1%
2011 0.01195 0.1%
2012 0.01171 0.9%
2013 0.01014 4.5%
2014 0.01132 8.0%
Sources: Mills’ paper, Umpires Report via Baseball Prospectus

Where Mills left off, there had been a clear trend toward better overall consistency between umpires across the league that actually extended back to 1999, when wCV was much higher at 0.01554. The measure of variability from 2014, while not another improvement, appears to be in line with recent seasons.

Out of interest, I calculated the percentage of pitches thrown when a “new” umpire was behind the plate, where “new” here is defined as umpires who did not call any major league games at all in the previous season. The results shows variability across the league held quite steady despite a relatively large influx of new umpires last season. I would suggest this speaks well of the Zone Evaluation system implemented to assess umpires across the league.

To compare umpire consistency fairly for 2015, I measured the same values for previous seasons, but only for all games up until the end of June.

Umpires’ Weighted Coefficient of Variation (wCV), 2009-2015, Start of Season-End of June Only
Year wCV % of Pitches to New Umpires
2009 0.01571 3.2%
2010 0.01390 0.2%
2011 0.01349 0.0%
2012 0.01519 0.1%
2013 0.01181 3.2%
2014 0.01416 10.6%
2015 0.01419 4.0%
Source: PITCHf/x via Baseball Heat Maps

Notice the values are higher than for full seasons, when the pools of pitchers that each umpire calls from behind the plate have time to distribute more evenly. While variability this season is certainly no better than recent years, it does not look like there is any significant degradation here.

For one additional viewpoint, I calculated the same measure across the seasons of study using only called strikes as a percentage of called pitches (wCVCP), as opposed to counting all non-balls as strikes in the previous metric (wCV).

Umpires’ Weighted Coefficient of Variation of Called Pitches (wCVCP), 2009-2015, Start of Season-End of June Only
Year wCVCP % of Pitches Called By New Umpires
2009 0.04367 3.1%
2010 0.04408 0.2%
2011 0.03595 0.0%
2012 0.04339 0.1%
2013 0.03900 3.3%
2014 0.03945 10.6%
2015 0.03733 4.0%
Source: PITCHf/x via Baseball Heat Maps

The scale changes when looking only at called pitches, but again it would appear the last several years have produced less variable results than earlier on in the Zone Evaluation era.

One observation I’ve made in my research with respect to umpire consistency is that there is an example of umpire cohorts adjusting to a change in the strike zone at different rates. In looking at the “lefty strike,” the outside edge of the strike zone off the plate for left-handed hitters, I noticed a quicker pace of contraction made by newer umpires compared to their colleagues who debuted in the league prior to the 2001 strike zone re-focusing effort. For this table I have included the 2007 and 2008 seasons as well, as they provide some context to the more equal starting point for the two cohorts prior to the introduction of Zone Evaluation in 2009, as well as the magnitude of the contraction in this area during the PITCHf/x era.

Outside Left-handed Strike Percentage, 2007-2015
Year Umpire debut prior to 2001 Umpire debut 2001-present Difference
2007 31.7% 31.0% 0.7%
2008 29.1% 27.8% 1.3%
2009 29.7% 25.1% 4.5%
2010 28.5% 23.1% 5.3%
2011 26.1% 21.6% 4.4%
2012 25.6% 20.2% 5.3%
2013 18.9% 15.5% 3.4%
2014 19.3% 12.8% 6.5%
2015* 15.9% 12.3% 3.6%
* Games until the end of June, Source: PITCHf/x via Baseball Heat Maps

There is a definite convergence happening here for all umpires toward eliminating the “lefty strike.” What can be seen, however, is that senior umpires have been contracting this region more slowly than their junior brethren. The difference is not staggering, but it is certainly present and quite consistent in magnitude since 2009.

The difference between the two umpire groups on the right-handed hitter outside edge has been much more muted, and there has been virtually no difference between the rates of change in the rapidly expanding bottom of the zone. This observation about the outside region of the left-handed hitter strike zone is my sole example of a place where umpire-to-umpire consistency may have room to improve.

So on the whole, the numbers back the commissioner’s claim that there’s been progress in regard to more consistency between umpires.

Effect of Changing the Strike Zone Definition

Conceptually – that was the facts, what’s out there – conceptually, I would be reluctant, even if someone said you have X problem, and the way to fix it is to give instructions on changing the strike zone, I would be reluctant to do that. Because when you talk to baseball people, baseball people you respect, and you say to them what happens if I do this with the strike zone, you don’t get any consistent response. I am very disciplined to make changes with respect to the play of the game on the field where we don’t have a pretty good idea of what the outcome is going to be.

Finally we come to the most open-ended aspect of the quote, thinking about what effects may be seen in the game should the strike zone definition be altered. Manfred says that people around the game have given him a variety of responses when prompted to imagine baseball with a new zone. Let’s try to talk through some potential outcomes, in as quantitative a manner as possible.

For the purposes of this exercise, the change to the strike zone that will be considered is elevating the bottom of the zone from its current level of around 18 inches off the ground (roughly the hollow of the knee) back up to its former level (in the 2009 timeframe) of 21 inches above home plate (more or less the top of the knee). The remainder of the strike zone will remain as it is being called today.

Umpire Accuracy and Consistency

If the official strike zone definition were to change, it would require ratification by the World Umpires Association. So we can realistically assume most members would be okay with the idea of a new definition. The question is, what could we expect as far as accuracy and consistency among major league umpires?

I submit that recent history tells us the Zone Evaluation system employed by MLB to assess home plate umpires works quite well, even in the face of change. Earlier we saw that as the zone has been changing and expanding at the bottom, both umpire accuracy and umpire-to-umpire consistency have been improving in recent years.

The past two and a half seasons also have witnessed much higher rates of new umpires calling games than in the previous stretch of the same length, yet umpire performance has not notably been affected by the turnover. This suggests the standard evaluation system is effective enough to ensure conformance to the desired called strike zone.

The observation about umpire cohorts adjusting to the left-handed hitter outside edge region at different rates has not translated to rates of change at the bottom of the zone. So the proposed zone change for this exercise should not introduce further umpire consistency issues along this avenue.

These facts make me believe that if the rulebook definition were to be altered, major league umpires as a group would adjust quite well, with the league’s Zone Evaluation and feedback process the key to success.

Strikeout and Walk Rates

When I first analyzed recent changes to the strike zone, I identified three regions where the majority of the changes were taking place: the outside corner for both left-handed hitters and right-handed hitters, and the bottom of the strike zone.

The outside corners for both left-handed and right-handed batters have been contracted in recent years. This has been especially prudent for left-handed hitters, who have had to deal with a strike zone shifted two to three inches off the outside edge of the plate.

The rule change proposed here only raises the bottom of the zone that has fallen so dramatically; the improvements to the zone made on the outside edges will be assumed to remain constant.

Consider first the direct changes in called strikeout and walk rates due to the way the bottom of the strike zone has been altered in recent years.

Called Strikeouts and Walks, Pitches 18-21” Above Home Plate, by Year
Year Called strikeouts Walks Difference
2009 334 1,563 -1,229
2014 1,134 1,004 130
Net Change 800 -559 1,359
Source: PITCHf/x via Baseball Heat Maps

In addition to this direct impact, changes to the zone drive adaptations to batter swing rates, so there are secondary effects such as swinging strikeout rates.

Swinging Strikeouts, Pitches 18-21” Above Home Plate, by Year
Year Swinging strikeouts
2009 2,218
2014 2,542
Net Change 324
Source: PITCHf/x via Baseball Heat Maps

Through previous research, I have shown that both pitchers and hitters learn where the zone is contracting and expanding relatively quickly. This is visible in the swinging strikeout totals on pitches in the expanded bottom of the strike zone.

I would expect the number of strikeouts and walks on pitches to the bottom of the zone to regress back toward their totals for the 2009 season, given that the strike zone definition that year had its bottom established at the same place the rulebook would define it in this exercise.

Summing up the tables gives an estimate of the change in strikeouts and walks that may arise from this zone adjustment. If we apply these changes to the 2014 totals, it would reduce the strikeout rate from 20.4 percent to 19.7 percent and increase the walk rate from 7.6 percent to 7.9 percent. The resulting (K-BB)% of 11.8 percent matches the 2012 rate, when about 4.3 runs per team per game were being scored in the league as opposed to about 4.1 runs today.

Run Scoring

If the rulebook strike zone definition is amended, it is with increased run scoring in mind as an effect. In my previous strike zone research, I estimated changes to the run environment caused by the morphing zone by calculating expected run differences for each pitch in a given season.

The idea behind this is that starting in a given count, if the next pitch is called a ball, the count moves more into the hitter’s favor, and the expected number of runs that will score from that point forward increases by a small fraction of a run. Conversely, if the next pitch is a strike, the pitcher improves his position, and the expected number of runs scored slides slightly lower.

To determine the expected number of runs for each count, I calculated the weighted on-base percentage (wOBA) for all plate appearances in a season in which a particular count was reached at any point, then divided the change in wOBA when the next pitch was a ball or strike by the yearly wOBA constant to arrive at an expected run difference in each of these cases.

Expected Run Difference, by Count, by Year
Count 2009 Ball 2009 Strike 2014 Ball 2014 Strike
0-0 0.037 -0.043 0.031 -0.036
1-0 0.062 -0.050 0.053 -0.044
0-1 0.031 -0.059 0.023 -0.051
2-0 0.097 -0.062 0.093 -0.052
1-1 0.050 -0.064 0.045 -0.053
0-2 0.026 -0.169 0.021 -0.150
3-0 0.117 -0.058 0.115 -0.055
2-1 0.101 -0.073 0.090 -0.063
1-2 0.040 -0.195 0.035 -0.171
3-1 0.175 -0.074 0.170 -0.066
2-2 0.100 -0.235 0.087 -0.206
3-2 0.249 -0.335 0.236 -0.293
Source: PITCHf/x via Baseball Heat Maps

With the run values calculated, I can sum up the small values over all pitches to our 18-21 inch band of interest for this exercise in both 2009 and 2014.

Expected Runs on Pitches 18-21” Above Home Plate, by Year
Year Expected Runs
2009 1,238
2014 235
Source: PITCHf/x via Baseball Heat Maps

If the bottom of the strike zone reverted to its 2009 height, this analysis estimates about 1,000 additional runs would be scored over the course of the season. If we augment the 2014 run total by this difference, it brings the runs scored per team per game up from 4.07 to 4.27. Once again, this closely matches the run environment experienced in the league in 2012.

To this point I have considered effects that such a strike zone re-definition may have on the major leagues as a whole. In addition, we can imagine impact felt at the individual and team levels.

Pitchers Who Target the Expanded Region of the Strike Zone

Intuitively, it seems that the pitchers who would have the most to lose if the bottom of the strike zone were lifted up again would be those who pitch most often to the area being contracted. If pitches in a particular region are suddenly being called balls, my earlier research has shown, hitters learn to swing less often at such pitches in response. Concurrently, pitchers adapt and throw less frequently to the suddenly unfriendly area.

For this portion of the analysis, I’m not so concerned with who was throwing to this 18-21 inch band several years ago; what is relevant is to consider who is doing so now to get an understanding of who would be forced to adapt the most to the rule change.

Here are the 2015 leader boards at the end of June for relievers and starters:

% of Pitches Between 18-21” Above Home Plate, 2015 Relief Leaders
Relief Pitcher % of pitches between 18-21” above home plate
Sergio Romo 17.9%
Joe Smith 17.0%
Brad Ziegler 15.4%
Jared Hughes 14.1%
Darren O’Day 13.5%
Minimum 400 pitches thrown, Source: PITCHf/x via Baseball Heat Maps
% of Pitches Between 18-21” Above Home Plate, 2015 Starters Leaders
Starting Pitcher Percentage of pitches
Tim Hudson 12.6%
Kendall Graveman 12.6%
Jerome Williams 12.2%
Mike Leake 11.6%
Kyle Gibson 11.3%
Minimum 1,000 pitches thrown, Source: PITCHf/x via Baseball Heat Maps

Clearly the sinkerball relief specialist who targets the bottom of the zone is more common than a starting pitcher who does so, given that relievers are facing hitters only once per outing. These leaderboards are full of guys who don’t throw very hard, make their living off command, keep the ball down with sinker/slider combinations, and nibble around the outskirts of the zone where it’s harder for hitters to do real damage.

These are the types of pitchers who would suffer the most with such a change to the rulebook.

Interaction with Infield Shifts

In his book Big Data Baseball, Travis Sawchik described a synergistic combination of changes the Pittsburgh Pirates implemented to improve their run prevention. This included converting from four-seam fastballs to two-seam fastballs, which tend to sink when thrown. Intuitively, we would expect sinkers to be thrown down in the zone, with the hope that hitters will swing over top of them and beat them into the ground. The final piece of the run prevention plan was an abundance of infield shifts, so that players were situated in the areas where all these ground balls most likely would be hit.

If the bottom of the strike zone were raised, two-seam fastballs would need to be thrown higher on average, which could lead to fewer ground balls and a reduced effectiveness of defensive efficiencies gained via shifting on the infield. With the Pirates’ story as an example, I looked to see at the team level if there is a positive relationship between the number of infield shifts implemented on defense and the percentage of pitches thrown to the bottom of the current strike zone that would be contracted under such a rule change.

2015 Team Statistics
Team % of pitches between 18-21” above home plate Number of Infield Shifts
Diamondbacks 9.7% 334
Pirates 9.7% 443
Giants 9.5% 253
Twins 9.3% 355
Phillies 9.1% 239
White Sox 9.1% 180
Athletics 9.0% 334
Yankees 9.0% 570
Brewers 8.9% 219
Rangers 8.8% 215
Mariners 8.8% 234
Marlins 8.7% 194
Padres 8.5% 358
Rockies 8.5% 497
Reds 8.5% 199
Astros 8.4% 878
Cubs 8.2% 165
Cardinals 8.2% 124
Red Sox 8.2% 255
Nationals 8.2% 148
Mets 8.1% 116
Angels 8.1% 219
Royals 8.1% 378
Rays 8.0% 869
Dodgers 8.0% 162
Tigers 8.0% 375
Orioles 7.7% 506
Braves 7.6% 95
Blue Jays 7.5% 467
Indians 7.4% 311
Through the end of June, Source: PITCHf/x via Baseball Heat Maps, Inside Edge via FanGraphs

At first glance, I realized that infield shifting appears to be performed at very different levels between the American League and National League.

2015 League Statistics
League Number of Infield Shifts
AL 6,146
NL 3,546
Through the end of June, Inside Edge via FanGraphs

The AL has shifted 73 percent more often to date in 2015 than the NL. This is an interesting phenomenon on its own. I could see two potential reasons for this. First, the AL features the designated hitter, whom I envision as one of the hitters most likely to be shifted to the pull side. Certainly he would be shifted more regularly than an NL pitcher. Second, perhaps there is a bit of follow-the-leader happening in the AL. The Rays and Astros are the shifting kings, and I could believe as teams face division rivals and see hits taken away by cleverly shifted infielders, it may spur them to invest in shifting research as well. The more teams that come on board, the more pressure there is to keep up among direct competitors.

Looking at potential correlations between throwing to the bottom of the zone and employing infield shifts, the stories are very different between the two leagues. In the National League, the data suggest a relationship between pitching down and shifting on the infield.

NL_18-21_tall
The combination is especially present for the Pirates. The American League, much like in the 2015 standings, looks chaotic. Several teams that shift heavily have been targeting the expanded area of the strike zone infrequently relative to the rest of the league.

AL_18-21

Perhaps a study with more resolution that looked at the number of infield shifts employed per pitcher or per pitch type might yield more consistent results. While certain teams may employ this combination of strategies, in general, there are many infield shifting plans used around the league.

As another related test, let’s consider potential relationships between teams that keep the ball down in general and teams that shift on the infield.

NL_lt_21_tall
AL_lt_21
When using the percentage of pitches fewer than 21 inches above home plate rather than specifically the contracted area, the data show an even stronger relationship in the NL but still nothing at all in the AL. Really, it is a handful of AL teams that shift a lot despite not keeping the ball down — like the Blue Jays, Orioles, and Tigers — that cause the scatter plot with such few data points to become jumbled in the AL.

In any event, I suspect teams like the Pirates, Yankees and Diamondbacks, which shift reasonably higher than their league average and also throw to the low part of the zone more frequently than other teams, may have the hardest time adapting to a zone re-definition such as this one.

Summary

In parsing the commissioner’s quote about the strike zone, I have found evidence to support most of his claims. As far as imagining what effect a new rulebook strike zone definition may have on the game, my analysis has estimated that if the only change were to move the bottom of the strike zone back up from the hollow of the knee to the top of the knee, the game as a whole would resemble the 2012 game. I found this when looking at strikeout rates, walk rates and the run-scoring environment.

The most complicated aspect of predicting the outcome of making such a change is identifying particular teams or individuals who would be especially hurt by this specific strike zone change. I suggested some individual pitchers and teams that may need to make more adjustments than others if a change to the bottom of the strike zone was enforced. One other potential side effect that could be conceived at the team and individual level would be whether (usually shorter) catchers who are particularly good at framing low pitches at the bottom of the strike zone may lose some of their competitive advantage with this type of rulebook change imposed.

I have confidence in the capabilities of (most) teams’ analytics departments to adapt to a rule change like this and find new ways to gain advantages. Much like the availability of Statcast data, a change to the strike zone definition would be a new development being received by all teams at the same time, and it is up to the organizations to make best use of the data as quickly as possible to gain an upper hand.

Can you think of other potential side effects? Please leave your ideas in the comments!

References & Resources


Jon Roegele is a baseball analyst and writer for The Hardball Times. He was nominated for a SABR Analytics Conference Research Award in 2014 and 2015. Follow him on Twitter @MLBPlayerAnalys.
26 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Gman
8 years ago

You don’t have to watch much baseball to know that the biggest problem with the strike zone is inconsistency from umpire-to-umpire and inconsistency from inning to inning. There is no question it is the most difficult job in umpiring. Given that fact, I fail to see why the umpires union treats this job as a “generalist” job instead of a job requiring specialists.

Rotating umpires behind the plate shows me that the Umpires Union has no concern for truly improving the quality of performance for that function. The Union now has a few years worth of statistics to show which umpires are on the “excellent” side of the performance bell curve (there’s always a performance bell curve) and which fall on the poor side. They should take best plate umpires and designate them as Primary and Secondary plate umpires for each crew. The combination of taking the best plate umpires and giving them greatly increased repetitions at the job will solve the long standing problem of performance mediocrity.

Specialization is an age-old proven strategy for improving the performance of complex jobs. It should be tried before before baseball goes to a robot ump.

BMMills
8 years ago
Reply to  Gman

Gman,

Very good points, and something I’ve thought about (very recently) as well. However, I suspect some of it is to reduce the demands of the job. While the umpire doesn’t squat behind the plate as much as the catcher, being back there with all that gear in mid-summer is a pretty demanding job. I’m guessing this is why there aren’t HP specialists. But definitely an interesting consideration.

In terms of the variability in strike zones, this is being addressed rather strongly through training ever since the league redefined the zone in 1996. If you read my paper (linked in the article), you’ll see that variation in strike rates across umpires has dropped dramatically–though gradually–over time.

Now, this doesn’t mean we’re at an optimal point of variation in umpire strike zones, but still something to keep in mind.

Further, there is not any guarantee that we’ll see a bell curve for a given behavior/performance measure. There are plenty of examples of outcomes that do not follow a normal distribution or anything that looks like one. While it is common across populations in many measures, it’s just not the case for many, many measurements you can think of. This is an unfortunate misunderstanding of the distribution of data that seems to come out of Intro stats courses and misunderstanding of the Central Limit Theorem. It turns out that accuracy rates themselves are somewhat left censored, as you might expect (minimum performance level expected in MLB). But that’s beside the point here.

Gman
8 years ago
Reply to  BMMills

Your points are well taken Brian. I was assuming the +/- performance variance based on my work experiences. We always found that jobs that were complex and required higher levels of specific skill always had better and worse performers and were therefore good targets for specialization.

I think the real key to my argument for greater specialization here is the improvements that can be driven by increasing the frequency that they work behind the plate. From what I can see umpires only get to do it once or twice a week tops. To me it’d difficult to achieve a high level of excellence at a difficult job like that with such limited frequency.

Jon Roegele
8 years ago
Reply to  BMMills

Great points, both of you. Thanks for replying Brian, you know the umpiring side better than anyone! I would think too a change to home plate specialist umps would require ratification by the World Umpires Association, and that may not pass depending on the impact it would have on the majority of umpires. It’s a good thought worth considering, though.

BMMills
8 years ago
Reply to  BMMills

As Jon notes, this would definitely require significant bargaining with the WUA, and would likely mean more umpires hired to be specialists at each position (so that umps are behind plate more if they’re a plate specialist, but not getting overworked). The union might be resistant to bringing up more umpires unless they are guaranteed pay at or above their current rates.

The other possible result could be that by widening the umpire pool, we end up lowering the talent level even for specialists. There must be an interesting optimal dip into the labor pool that would both increase performance because of specialization, and avoid hiring too many low-skilled umpires.

Sounds like a fun empirical question (and lots of important implications in the training/specialization literature in labor econ, where my interest in the issues arise).

francis
8 years ago
Reply to  Gman

Although you are right about the fact that having one ump specialize behind home plate would be better, there is no reason not to have what you call “robot umps”.

I don’t see why people are so reluctant to embrace automated balls and strikes. It didn’t take any hard thinking to put up foul poles or paint the foul lines. Once a solution existed to set a boundary that didn’t need umps and didn’t interfere with the game, it was put in the game.

The technology exists, it ought to be used.

And to me there’s nothing worse than broadcasters who will say in passing that an ump “has a generous strike zone”, or is known to give veterans a pitch on the corner because he knows the guy.

This is insane ! I heard one broadcaster say something like “he’s been throwing strikes all day, so the umpire’s going to give him that pitch on the corner”. What ?! How is that acceptable ?

Imagine if that happened with field goals. “It’s up and it’s wide right … but wait, it’s called good ! Well, he’s been making those kicks from 50 yards out all season, so the refs are going to give him that one.”

Gman
8 years ago
Reply to  francis

Glad I came back to these comments. I agree that a computerized plate umpire would be a vast improvement immediately, even if it wasn’t perfect. That’s mainly because it would likely be consistently imperfect in the pitches it calls improperly.

Maybe the best way for MLB to deal with a historically stubborn Umpires Union is to offer them only two alternatives, automation or specialization. The commissioner always has the “best interest of baseball” card to play in order to force them to take one of the 2 alternatives.

If the Umpires strike, MLB would go to automation for balls-and-strikes and replacement umps for the bases. Instant replay is a great safety net to minimize the risk of bad calls in the field by field umps. And, MLB could always expand the challenge rule to allow more challenges and increase the things that can be challenged in order to prop up the quality of replacement field umps.

BMMills
8 years ago

As Jon notes, this would definitely require significant bargaining with the WUA, and would likely mean more umpires hired to be specialists at each position (so that umps are behind plate more if they’re a plate specialist, but not getting overworked). The union might be resistant to bringing up more umpires unless they are guaranteed pay at or above their current rates.

The other possible result could be that by widening the umpire pool, we end up lowering the talent level even for specialists. There must be an interesting optimal dip into the labor pool that would both increase performance because of specialization, and avoid hiring too many low-skilled umpires.

Sounds like a fun empirical question (and lots of important implications in the training/specialization literature in labor econ, where my interest in the issues arise).

Cyril Morong
8 years ago

Great work!

Gman
8 years ago

Brian, I’m not sure I understand this statement “this would definitely require significant bargaining with the WUA, and would likely mean more umpires hired to be specialists at each position (so that umps are behind plate more if they’re a plate specialist, but not getting overworked)”.

Here’s where I’m confused. If the prevailing opinion is that there are enough umpires per game based on how they perform their functions today, then why would they need more to specialize? They still would only use umps per game.

To stick a broad number on it, let’s say they use the statistics to identify the top 50% plate umpires. Of that group the top 25% would be designated Primary Plate Umpires and the other 25% of the top performers would be designated as alternates. Instead of my example of 50%, the % used would just have to cover having one primary and one secondary per umpiring crew. The secondary ump would still ump the bases when the primary is behind the plate .

One other question for either of you. Do you know if umpires get “premium pay” for games they umpire behind the plate? If that’s true, I can see the Union fighting against specialization. Umpires with seniority who don’t make the grade as plate umpires won’t accept losing that perk or having some younger union member getting higher pay.

Gman
8 years ago
Reply to  Gman

should read “They still would only use 4 umps per game”.

Jon Roegele
8 years ago
Reply to  Gman

Gman, I’m curious about your question about pay as well. Even if they aren’t paid at a premium now for those games because umpires rotate through, you would think home plate umpiring specialists would have to be paid at a premium under a more specialized system.

I think Brian’s point is that home plate umpiring is much more taxing than umpiring on the bases, and even if highly-rated umpires agreed to specialize and work the plate exclusively, they may not be able to handle the rigors of that over such a long season throughout a hot summer. Or if they could, their performance may drop accordingly, which defeats the purpose. So the WUA may have to hire extra home plate umpire specialists to give enough breaks to everyone, which is where costs can rise.

BMMills
8 years ago
Reply to  Gman

My point simply being that an umpire is not going to work 162 games behind the plate. Even if you have 4 umps per game, they’ll need days off, in which case other home plate specialists will have to take their place. Let’s say they can do double their current load. Then they’ll need twice as many HP specialists for the days off the guys have. But I see where you’re going with this now.

Perhaps you’re saying only 2 guys rotate as HP specialists in the 4-man crew. That seems to be your implication, and that makes more sense to me.

One thing to note is that there are likely status issues with not being behind the plate that would still need to be handled by the WUA. We’re talking about egos that fought against calling strikes according to the rules for years.

I don’t think there’s any premium pay for behind the plate (though there is, I think, for crew chief–which could be correlated with plate duty), but that’s also because they rotate pretty evenly. I would envision that changing if umpires are required to do the plate more often, which would result in more pay differences across the labor pool of umpires. This would require significant bargaining, if history tells us anything about the union.

There is additional bonus for playoffs, and if there are specialists, it could impact who is chosen to work the playoffs. Again, this would require bargaining.

francis
8 years ago
Reply to  BMMills

Pay them more and they will be happy to work 162 games behind the plate. Besides, for most of these guys, they love the power to dictate the course of the game and running guys who question their “judgment”. Honestly, they should automate balls and strikes and home plate umps should just be there for pace of game and plays at the plate.

Gman
8 years ago
Reply to  BMMills

Sorry for the delayed response BM. Usually these comment sections die out in a day. This one seems to have some legs! Anyway, yes what I envisioned was every crew having 2 umpires who were designated as specialist plate umpires but still rotating those two in the field. This would maximize their repetitions behind the plate without changing the number of umpires required. It would also allow for more focused evaluation and training of those two individuals. This practice should help elevate their skill even further than what they demonstrated in being selected as the top X% of what is currently a pool of generalist umpires.

As for your reference to the issues this creates with the Union, check out the reply I just made today on that very topic to Francis. It would be interesting to hear your take if you’re still following this comment thread.

Robots
8 years ago

If you want a surge in offense and to speed up the game why on earth is the league still not pursuing automation?

I would think a strike zone of consistency would help hitters a lot more since they do not need multiple plate appearances to learn how specific umpires call and would have a major variable in hitting eliminated.

Dave Kingman
8 years ago

There isn’t enough depth and thoroughness to this analysis.

Just kidding! Good Lord, man….what a nice piece of work. Take the rest of the day off.

Much better than the ranting and raving by other (nameless) writers with inarticulate nonsense the “business” side of baseball, and how owners are bad men doing mean things to good people. Or maybe it’s agents. I forget.

Articles like this, as well as the Baseball Card stories, are why I love this site. Keep them coming.

bucdaddy
8 years ago

I remember the 1960s. I’d be verrrrrrrrrrrry careful about making even minute changes in the strike zone. MLB thought it was only making the zone a couple inches bigger. The result was the second deadball era.

To the Wayback Machine!

http://www.hardballtimes.com/re-imagining-the-big-zone-sixties-part-1-1963-1965/

mark
8 years ago

Perhaps this is just pie in the sky, but: why are we still relying on ump calls at all? Why can’t balls/strikes be called by computer? … Well, it’s probably the umpires who don’t want to give up their power to make decisions, even if they are bad ones. Human nature and all that.

But what if we could convince the umps that their job is not to call the strike zone, but to adjudicate other calls such as: foul tips, foul balls, balks, HBPs, catcher interference, home plate safe/out calls, popups caught/not-caught, infield fly, judging when injuries necessitate time off from the mandated time between pitches, etc. Isn’t it possible that umpires would actually ENJOY taking the decision on 250 pitches a game off their plate? (Pun somewhat intended…) In fact there is a well-defined psychological condition called “Decision Fatigue” which might mean that umpires make hastier, less-well-grounded decisions as games go on. (Getting hungry? Want to get home? Widen the strike zone!!)

Logically speaking, the only reason a person wants to continue doing something that a machine can do better is they fear replacement. But those other elements of umpiring are not trivial. And it’s time to think through how to improve the game. By tradition, those men in blue are supposed to be invisible. When they’re doing their job right, they blend in and are incredibly good supporters of the game. When they aren’t, they detract– sometimes brutally– from the proceedings. Read some of the best autobiographies of umpires to see if it isn’t the case. (I recommend “Jocko” by Jocko Conlan as the best umpire book of all time.)

Do people fear that computers would be tweaked by clubs to help home teams? Well, that’s certainly possible, but monitoring the equipment could be the bailiwick of the umpiring crew, and MLB. MLB could ask for no-access rooms at each ballpark. And even if teams tampered, will it be at a level that helps the home team more than the natural home-team advantage (a noisy, partisan crowd voicing pleasure or displeasure at every tense pitch call, generally)?

To me, it’s fascinating to think thru what would happen if balls/strikes were called exclusively by machine. I posit a few:

1. Newcomer (rookie) pitchers and batters would see increased production at the expense of veteran players: it’s reasonable to expect that borderline calls would go against rookie batters just up from minors when facing veteran pitchers, and vice-versa– even if only from comfort with the pitchers’ motions and pitch sequencing.

2. Home teams would have their home-field advantaged neutralized to some degree: as noted above, home crowds seem to subtly influence umpires’ (and NBA referees’) decisions on close-call plays. The constant feedback from a partisan crowd unconsciously influences all humans, even those who would say they are trained not to have this interfere with their judgement. There are all sorts of psychology tests that can prove unconscious bias.

3. The lessening of tensions between certain players and the umpires would result in a fairer game. How many times do you hear of umpires holding grudges, giving make-up calls, letting their humanity enter the frame. Really– I don’t know if you agree, but to me it seems inevitable. And just as you’re more likely to give way on the street to a person who smiles at you nicely, umpires are more likely to give calls to players they like– for familiarity, for unconscious affiliation reasons. You think Dick Allen, the nasty African American player in the 1960s got better calls than Mickey Mantle? Hmmm…. Or think about this: what would Jackie Robinson want when he first got to the league?

All of these, and more, would be a terrific data study. We KNOW that of 250 pitches in any one game, about 12% are called wrong– that’s something like 30 pitches– one for every other batter! What are the ramifications of this error rate? For whom are they wrong? Whom do they disadvantage or give advantage? Are they really random?

We now have the ability to see every pitch on TV broadcasts to see how pitches match up with umpires’ calls. The imprecision is incredible. When Cyclops and the line-watching machines started to monitor tennis lines, the USTA had the same reaction: pride that their (amateur) human line-monitors got 90% of their calls correct. Well, 10% is a big number for an acceptable error rate. And we can see in tennis how often the calls are overturned.

Baseball needn’t even go slower to do so. Just have the computer add to the ball/strike scoreboard count. Flash a K or a B. Flash a red or green light. There’d be less jawboning, less enmity, more equity between umps, batters and catchers, between teams.

In a word, baseball should deeply explore the process of turning over the calling of balls and strikes to machines. It would make a fairer, faster and more perfect version of the game. The only thing stopping it? Probably men in blue who feel threatened. It’s too bad they do. They’d be helping the cause of truth, justice and the American way.

Jon Roegele
8 years ago
Reply to  mark

Thanks Mark for the lengthy comment.

You’re right that there is a strike zone advantage for the home team, and no doubt this as well as “make up” calls and such could be eliminated with some form of automated ball and strike system.

The best article that I have seen on the challenges of implementing this at the MLB level is still this piece from Ben Lindbergh from 2013:

http://grantland.com/features/ben-lindbergh-possibility-machines-replacing-umpires/

B Mac
8 years ago

Balls put in play from the bottom of the expanded strike zone rarely result in extra base hits. Thus eliminating this part of the strike zone would likely have a greater effect on improving run scoring than merely contracting the strike zone in general. If you force the pitchers to bring their pitches up, you will increase scoring.

This stuff about changing the definition of the strike zone is fairly over dramati; they just need to enforce the existing definition.

Ed
8 years ago

Wow. Amazing piece. Thank you.

The larger question I have about the strike zone though, is why it isn’t called, and isn’t close to being called according to the rules on the high side.

“Rule 2.00: The Strike Zone

The STRIKE ZONE is that area over home plate the upper limit of which is a horizontal line at the midpoint between the top of the shoulders and the top of the uniform pants, and the lower level is a line at the hollow beneath the kneecap. The Strike Zone shall be determined from the batter’s stance as the batter is prepared to swing at a pitched ball.”

Watching baseball every day, it appears to me that in practice the top end of the strike zone is effectively the belt….or maybe an inch or two higher. As I read the rule though, it should extend at least another 6 inches higher….maybe more. And as I recall from my youth, it used to be called something like that. Somehow it slid over the years.

BMMillsy
8 years ago
Reply to  Ed

Good point, and one that likely dates back to Sandy Alderson in the late 1990s. He sent out a memo that the league wished to have the strike zone called 2 inches above the uniform pants, and this was shortly after the redefinition of the zone happened in 1996 to the one you quote.

http://amarillo.com/stories/1999/02/23/spo_LS0403.001.shtml#.Va07IfnIYn9

Why it’s not set in as the rulebook definition, I’m not sure.

Rick Swanson
8 years ago

Great story. You should look at Baseball Savant website. They list each umpire and their % of wrong calls since 2008.

Your story tells the answer when you see the pitches outside the zone only changed from 14.9% to 14.3%

Those inside the zone went from 15.6 to 9.6

Umpire mistakes favor pitchers by a large margin now.

Bilbo161
8 years ago

How meaningful can strike zone size be considering that depending on a batter’s size and stance the size of the zone changes too. Is this even accounted for in the electronic pitch calling methods?

rbl
8 years ago

Simple answer to fix the poor ball/strikes calling by the umps is to releave them of that job since they dont get it right. it poorly. The 99.9 % accurate electronic system of calling the balls/strikes will make it fair to pitcher and batter and speed up the game by eliminating all that ego arguing by those kids!! This is the national pastime a nd is not about the umps but only the team players. Its past due. Go back and correct all those terrible calls that changed the outcome of the game. Joyce and the others ego’s got in the way!!