Defining the Pitch Sequencing Question

Charlie Morton was streaky when it came to mixing his fastball. (via Matthew Roth)

Charlie Morton was streaky when it came to mixing his fastball. (via Matthew Roth)

“It is a familiar and significant saying that a problem well put is half solved.” –John Dewey

Pitch sequencing is a hot topic in the sabermetric world. It is also extremely complicated. It has attracted the attention of some very smart people who have attacked it with varied analyses containing a high level of mathematical sophistication. Just over the last 12 months, we have seen (among others) from THT: The Effects of Pitch Sequencing and Finding Value in Fastball Mixing; and from BP: The Art and Science of Pitch Sequencing: Sabermetrics’ Undiscovered Country, Moonshot: Cracking the Location Code, Baseball Therapy: The Power of Changing Speeds. That’s a great thing for baseball—with so much data available today and so many big brains focused on the topic I have no doubt that it will be “answered” someday.

What is not so great about the current state of research on sequencing is that while lots of smart people are looking for an answer, I’m not sure anyone really agrees on the question. What follows, in this and subsequent articles, is my attempt to try to better frame the problem by exploring what pitch sequencing really means.

To that end, my goals are modest: (a) explore some of the challenges in pitch sequence analysis, (b) look for evidence that pitch sequencing could actually be improved in some small way, and (c) consider what we actually mean by “improvement” in this context. To try to accomplish those goals, this series will ask the simplest possible questions about pitch sequencing and attempt to answer them in an unbiased manner. I have no prior ideas about where this will lead beyond the first few questions I’ll be analyzing.

Now that you understand my purpose, let’s get on to the fun part!

Question 1: What effect does changing pitch speeds have on hitter performance?

This should be a breeze. All we need to do is find all two-pitch sequences where velocity changed by more than some amount, and compare what happened to all the two-pitch sequences where velocity didn’t change by more than that same amount.

Except… how do you make sure the samples are truly comparable? Are pitchers more likely to change speeds in some circumstances than others, like with the bases empty or in favorable counts? Are they more likely to do so against certain hitters than others? What about following a strike as opposed to a ball? If any or all of these concerns are valid, will that bias the results to make changing speeds look more or less effective than it actually is?

On an even more basic level, do you consider all speed changes of a certain magnitude (say 5 mph) to be equal? In other words, do you include both fast-then-slow and slow-then-fast pairs in the sample? If you do, how do you ensure that the control sample has the right mix of fast-then-fast and slow-then-slow pairs? How do you even define a “slow” pitch in the absence of a specific “fast” pitch to compare it to? Do you look at the pitcher’s whole body of work or just how he was throwing on that day?

Once you’ve controlled for everything you can possibly control for, what is the measure of performance you want to compare? Throwing a strike is good, so maybe you can consider swinging strike rate on the second pitch. But if an increase in whiff rate comes at the expense of an even larger increase in called balls then is the gain worth the cost? And contact has to be factored in as well—if there was a magic pitch sequence that led to a pop-up with high likelihood, is that better than getting a strike or even two?

Fortunately, these issues are addressable. To limit the possible variations in game state (runners on, stolen base attempt likely or unlikely, favorable or unfavorable count, etc.) we’ll look only at plate appearances with the bases empty, and only at 0-0 counts. To address the “fast vs. slow” problem we’ll  consider only plate appearances where the first pitch was classified as a fastball variant by PITCHf/x. To control for other variables (platoon, hitter strengths and weaknesses, pitcher’s pitch arsenal, etc.) we will use the “delta method.”

As an example, we’ll take all the plate appearances in 2014 in which Mike Trout faced Felix Hernandez with the bases empty that began with a fastball and went for two or more pitches. We’ll compare the outcomes when the second pitch was at least 5 mph slower to the outcomes when the second pitch wasn’t at least 5 mph slower to get our “delta.” We’ll then weight the Felix vs. Trout delta by the smaller of the two samples; for example if they had five qualifying match-ups where Felix changed speeds and three where he didn’t, we’d use three as the weight. Then we repeat this for all batter-pitcher match-ups (excluding pitchers-as-batters) that meet our criteria in 2014, and then separately in all other years (to try to control for aging effects), and compute a weighted average to get an overall delta.

We still need to choose a metric to… well, measure. Fortunately for us, some very smart people have already figured out how to compare all possible baseball outcomes on a uniform basis: linear weights. The only wrinkle in our case is that we need to incorporate the linear weights values of balls and strikes in various counts, while most linear weights analyses are done on the outcomes that end a plate appearance (out, walk, home run, etc.). Joe Sheehan wrote about the run values of individual pitch outcomes in 2008. This analysis uses pitch-level linear weights computed from GameDay data spanning 2008 to 2014.

So now we finally have a methodology: find all the match-ups that meet our criteria, compare the linear weights value of the second pitch when it is more than 5 mph slower to when it isn’t, and compute a weighted average.

There are still two more wrinkles. First, a pitcher may be more likely to change speeds after getting a strike on the first pitch than after getting a ball (and the numbers do bear this out), and hitter behavior may be different in a count of 1-0 than 0-1. To control for this bias we’ll separate both of those first-pitch outcomes. Second, we’re completely ignoring all plate appearances where the ball was put in play on the first pitch. In this particular case I would argue that it’s fine—we’re looking at the effect of changing speeds conditioned on getting a ball/strike on a first-pitch fastball. For other questions that I’ll explore in later articles this will come into play.

A Hardball Times Update
Goodbye for now.

Enough words! Let’s (finally!) get to some numbers.

Linear weights value (per 100 pitches) of second pitch speed decrease >5 mph (negative is favorable to pitcher)
First pitch outcome Sample Quantity Runs Per 100 pitches
S 21,974 -.12
B 10,117 -.52

This table shows that on average, after throwing a first-pitch fastball for a strike, pitchers did a little bit better when they dropped speeds by more than 5 mph than when they did not. The magnitude of the advantage was 0.12 runs per 100 pitches, or about 0.15 RA/9 points for an average pitcher. The effect was much larger when reducing speed after a ball: 0.52 runs per 100 pitches is about 0.7 RA/9 points for an average pitcher.

Of course there are still some reasons to not get too excited by these results. Most obviously the samples are surprisingly small: 10,000 or 20,000 plate appearances are a lot more than the hundreds we are used to dealing with for individual player analyses, but considering there were millions of pitches thrown between 2008 and 2014, it’s clear we have really whittled away at things by narrowing our focus so much. (As an aside, note that the “Sample quantity” is the sum of the weights across all matched pairs in the delta method, i.e. the sum of the smaller of the two samples in each case.)

But the goal was limited: to find evidence that changing pitch speeds has an effect on hitter performance. The results indicate that there is indeed an effect, and a future article will look at what may be driving it.

Parting thoughts

This was a very long-winded journey to what may seem to some like a very obvious conclusion. One of the goals was to give some flavor for the difficulty of analyzing pitch sequence outcomes, so the long-windedness was partly a feature, not a bug. The good news is that going forward we can skip some of that preamble to just apply similar methods to other pitch sequencing questions and talk about the results. I hope you’re looking forward to it as much as I am.

But is the conclusion really that obvious? Of course changing speeds is a valuable pitching tactic! The ability to do so effectively and with command is part of what separates prospects from pros. And even if we found that the run value of dropping speeds on the second pitch was zero (or even positive) it doesn’t mean that pitchers’ overall performance would be the same or better if they never did it. To draw that conclusion we’ll need to either run controlled experiments (any GMs out there want to volunteer their staff to just throw fastballs every other day for a year or so?) or come up with a more clever way of slicing the data. What this analysis shows is that pitchers who already do change speeds would on average benefit from doing it even more than they already do, all else being equal.

So far, I’ve laid out a methodology for analyzing some basic questions, tackled the first of them and found some kind of a result. With the kind indulgence of the folks at The Hardball Times, I will keep plugging away at this and hope it will all lead somewhere useful.

I welcome everyone’s feedback — please leave your thoughts, suggestions, complaints, insults and devastating critiques in the comments.

Coming next: What effect does changing pitch location have on hitter performance?


Peter is co-founder and CEO of Vendorful.com, a reformed investment manager, a sometimes programmer and data analyst, and an amateur saberist.
11 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
scrappypunto
9 years ago

Would large differences in the amount of data points in the change speeds/not change speeds bucket bias the delta?
I.E. There are 30 instances where felix went same speed/same speed on Trout, and 2 where he changed up. If you weight by same speed instances, this would put a lot of weight on a delta value that is subject to a lot of variance, due to the tiny sample of data where felix changed speeds.

Am I wrong in my thinking? Are large differences between data sizes like this actually an issue?

Andy
9 years ago

One of these smart people, MGL, I think his moniker is, claims that equilibrium is reached very quickly when it comes to a batter’s weakness. If there is a certain kind of pitch speed/location that a batter has trouble with, he will soon see an increasingly larger proportion of those pitches. Eventually, according to mgl, a plateau will be reached, the outcome of an equilibrium between the difficulty the batter has in hitting that pitch, on the one hand, and the greater ease in anticipating it (because it’s being thrown more often) on the other. At this equilibrium, every pitch is equal in its difficulty to hit, with pitches that otherwise would be harder to hit being made easier by their greater frequency.

So my question is, could this happen with pitch sequencing, too? Suppose word gets out that changing speeds results in some significant effect in reducing run value. So pitchers start doing it more often. Then won’t batters come to anticipate this change more often?

It’s the old arms race game, with an improvement in offense countered by a change in defense, and vice-verse. Taking a very general view, I think analysts like you have to be considered part of the game. You are not sitting afar and observing the game objectively. I think you’re actually involved in changing it, even playing it. It’s no longer being played just on the field.

Peter B
9 years ago
Reply to  Andy

MGL’s thoughts on pitching are very much on my mind when it comes to this problem. His “equilibrium” view should be intuitive to anyone with a basic understanding of game theory. What I don’t know is how near or far that equilibrium is from the current state for most pitchers. My prior is that most pitchers are close to that equilibrium state, but testing that hypothesis has proved maddeningly difficult (at least for me). Hence my desire to start from basics.

One huge benefit of the current wealth of data we have on baseball is that we can actually see whether and how analysts are having the kinds of impact you describe! The full record of the PITCHf/x era is available to us, and if the arms race you describe has already played out we should be able to see it in how pitching has changed over recent years.

You described the very thought that led me down this line of research: what impact has PITCHf/x had on how pitchers pitch? Really, when I talk about framing the question I’m talking about framing that question.

RM
9 years ago

Wouldn’t you say that the pitch location also does have an affect on the probability of a pitcher switching speeds? Changing a hitters eye-level with pitches changes the amount of time a hitter has to react to the pitch. for instance a high fast ball is going to “look” faster than a lower one.

Andy
9 years ago
Reply to  RM

RM, I think it’s more of an inside-outside thing, with batters having less time to react to an inside pitch than an outside one thrown at the same speed. There is some guy, I think he was written about here last year, who has developed a system for pitchers based on this.

MGL
9 years ago

I love the way you framed the question and the basic way you set up your methodology. In the past, many researchers have not controlled for count, batter, pitcher, etc. which renders the results useless. As well, they often do not use linear weights by pitch, including balls and strikes, which is critical.

Yes, it is true that if you control for the situation, the value of every single pitch selection should be exactly equal if batter and pitcher are acting optimally (or close to optimally).

One other thing that must be controlled for is game situation and that could be significantly affecting the results. For example, when the pitching team is ahead, especially way ahead, later in the game, the pitcher is more likely to throw a fastball on all pitches, more likely to throw a strike, etc. The batting team is more likely to be taking more pitches, etc.

Base runners must be controlled for too. For example with a base open, the pitcher is more likely to throw off-speed pitches, pitch around, etc.

Remember that pitch sequencing and game theory, wrt, the the Nash equilibrium, apply only to identical situations (count, batter/pitcher, score, inning, runners), so while it was good that you controlled for many of those things, you did not control for nearly all of them, which is problematic in terms of game theory and evaluating strategies. Of course, once you start controlling for all these variables, your sample sizes probably become too small for the data to be that reliable.

Peter B
9 years ago
Reply to  MGL

Thanks MGL. That is extremely helpful feedback. I attempted to control for baserunners by limiting myself to bases-empty situations, but I hadn’t thought much about the effect that the score might have on things. It certainly needs to be considered.

As you observe, adding controls very quickly shrinks the sample sizes available. In your opinion, is there any way I could loosen up with respect to baserunners to try and offset additional game situation limitations? The reason I settled on “bases empty” over simply matching base states in each sample was to avoid having to worry about runner speed and associated tactical considerations (pitchouts being an obvious example). But perhaps that is like swatting a fly with a sledgehammer, if there are base states that can be grouped together from a tactical perspective.

Peter B
9 years ago
Reply to  MGL

Just to follow up on this, I did some analysis of fastball rates based on game situation, and (unsurprisingly) you’re exactly right.

I computed fastball rates (including all PITCHf/x fastball variants) for every pitcher from 2008 through 2014, broken out by platoon, inning, count, pitch index (i.e. 1st pitch, 2nd, 3rd, etc.), and the run differential between the two teams. I then used the delta method to compare the rates for each pitcher from the 7th inning onward when (a) their team is ahead by 4 or more runs vs. (b) it’s tie or one-run (in either direction) game. Here are some results:

* On the first pitch, pitchers threw fastballs at a 5.2% higher rate when far ahead
* On the second pitch (ignoring count), pitchers threw fastballs at a 1.5% higher rate when far ahead
* Irrespective of count or pitch index, pitchers threw fastballs at a 3.6% higher rate overall when far ahead

In fact the effect shows up whenever the pitcher’s team is far ahead, not just late in games. In innings 4 through 6 pitchers threw fastballs at a 4.1% higher rate on the first pitch, and a 2.3.% higher rate overall, when far ahead than in close games.

For avoidance of doubt, the rate differentials here are arithmetic differences (i.e. an increase from 50% to 55% would be 55-50=5%, not 55/50-1=10%).

Dennis
9 years ago

I guess now we might wonder how these variations look between LHP & RHP. And starting pitchers and RP? But you have enough homework as it is…

Dennis
9 years ago

I guess too we might wonder what the results would be using LHP & RHP? And SP and RP? But you have enough homework already…