In Game One of the NLCS in St. Louis, Dodgers manager Don Mattingly decided to keep his best reliever, closer Kenley Jansen, on the bench in favor of lesser relievers until the 13th inning of the game, with runners on first and second with no outs. The reasoning behind this decision was, presumably, the conventional idea that a team’s best reliever should close out the game, that holding a lead is more important than protecting a tie.

Unlike many of my saber-inclined peers on Twitter, I was not particularly bothered by Mattingly’s decision to hold Jansen until the 13th. The reasons for these thoughts were twofold: one, the leverage of the situation in which Jansen came in (4.3 Leverage Index) was higher than any other in the game for Dodgers pitching. Mattingly brought his best pitcher into the game in the most important situation of the game, on the surface a rational-sounding decision.

The other reason was because I do not believe that, in theory, an inning in a tie game is more important than an inning with a one-run lead. My reasoning is as follows:

Assuming for the moment that only one run can be scored in an inning, then in order for the Dodgers to win they had to pitch a scoreless inning whenever the game was tied, and pitch a scoreless inning to end the game. In that case, why would it matter when Jansen pitched, if he would have to pitch for the Dodgers to win regardless?

Consider the win expectancy of both situations, before and after a scoreless and one-run inning:

Score | WE before | WE, 1 RA | WE, 0 RA |
---|---|---|---|

Tie | 0.37 | 0 | 0.5 |

Up one | 0.8 | 0.5 | 1 |

As one would expect, the cost of allowing a run in a tie game is greater than the cost of allowing a run when you are up by one. However, the benefit of pitching a scoreless inning is greater with a run lead than with a tie. In simpler terms, allowing a run in a tie game loses the game, and not allowing a run with the lead wins the game. Everything else just leads to another inning. Therefore, both situations have equal importance…right?

Wrong. As it turns out, given the assumptions made above—that only one run can be scored in an inning, and that relievers can pitch only one full inning—it is not only true, but necessary, that it is better to pitch the best reliever in the first extra inning. I don’t want to bore you with the math, but I think the math is interesting. So, I’ll explain it anyway, as simply as I can.

Take an extreme example: say the closer for a team is guaranteed to pitch a scoreless inning, and that all other relievers will give up a run exactly half of the time.

**Situation A: Closer is brought in at the beginning of extra innings**

In this situation, it’s fairly easy to see, poorly made flowchart aside, that the final win probability for the away team will be 75 percent. In the first inning, there is a 50 percent chance that the away team will score a run, in which case it will win. In the other 50 percent, the game will automatically go to the 10th, since the closer came in regardless. Then, in the rest of the game, since both teams have the same chance of scoring, there is a 50 percent chance that the away team will win. Add that 50 percent chance in the first inning to 50 percent*50 percent in the rest of the innings, and you get 75 percent.

**Situation B: Closer is brought in only after the away team takes a lead**

This situation is somewhat more complicated. In the first extra inning, if the away team scores, it wins, just like Situation A. However, if it does not score, then because the closer is not coming into the game, the home team has a 50 percent chance of scoring, or 25 percent overall in the inning. If the game remains scoreless, than the same situation will repeat itself in the next inning: 50 percent chance of the away team winning, 25 percent chance of the home team winning, and 25 percent of the inning continuing.

There are two approaches to finding the overall probability of the away team winning in this situation. The first is just logic: in each inning,* if the game ends*, the away team is twice as likely as the home team to win, or a 67 percent chance. And because each inning is exactly the same, this probability will be the same. The away team will always have a 67 percent chance of winning if the game ends, and because this happens in every inning, the overall probability of the away team winning is 67 percent.

Very quickly, because I think it’s cool, the other way to prove this is through math. The expression for the probability of winning for the away team in the second situation is as follows (if I remember my pre-calculus correctly):

This is a geometric series, and the nice thing about geometric series is that it’s really easy to solve them. If *a* is the value when *n* is zero, and *r* is the value when *n* is one, then the sum of the series is simply a/(1-r). Since *a* is 1/2, and *r* is 1/4, the sum is (1/2)/(3/4), or 2/3, or 67%. Math! It’s fun!

The hypothetical situation above is fun and interesting, at least to me, but it’s not particularly realistic. No closer prevents runs at a 100 percent rate, and the gap between closers and other relievers is not nearly that great. Even if we stick with the assumptions that relievers must pitch complete innings and they can give up a maximum of one run, the numbers that we found above will be very different with more realistic probabilities. However, the above examples are clear and logical evidence that *within these restrictions*, it is always better to bring the closer in earlier rather than later. The difference between the two situations may grow smaller with the difference in quality between closers and other relievers, but the difference will be there nonetheless.

But what happens when we not only use realistic run prevention probabilities, but include the possibility of allowing (or scoring) more than one run in an inning? To do so, we must move away from flowcharts and simple math and logic, and move into the wonderful world of simulations.

I’m no expert in sims, but I was able to write a simple one, which took the probability that the away closer would give up zero, one, two and three runs, the probability that all other relievers would do the same, and then compared the two situations described above: bringing in the closer in the first extra inning, and bringing in the closer only after a lead. For the closer, I took the career numbers of Mariano Rivera, who gave up zero runs approximately 86 percent of the time, one run 10 percent, two runs three percent, and three runs one percent of the time. For all other relievers, I took the average 2013 reliever numbers, which were 80, 13, five, and two percent for zero, one, two and three runs respectively.

When I stuck these numbers into my sim and ran 100,000 “games” (essentially tie games starting at the 10th inning), I got the following numbers:

Closer at the beginning: 53.1 percent

Closer at the end: 52.2 percent

Well, there’s a difference. It’s not a large one, but it’s there. Is one percent of a win large? Not really. But few managerial decision make much of a difference, so 1 percent is nothing to scoff at.

Of course, we have not considered all of the variables here, the most important of which is the fact that relievers don’t have to only pitch one inning, and they don’t have to come into the game only at the beginning of the inning. This is, I believe, a crucial factor at play. If there is really only a 1 percent difference between bringing the closer in at the beginning and the end of the game, then could that difference be made up for by the benefit of pitching the closer in a high-leverage situation later on?

The dilemma here, as I see it, is this: Each inning that you keep the closer sitting in the bullpen, you risk the home team scoring a run, thus wasting the closer entirely. But holding the closer on the bench to start the inning also gives you the opportunity of bringing in the closer during a high-leverage situation, thus utilizing his value more so than you would if he just pitched a whole inning from the beginning. If the closer is going to pitch three outs, why not make him get more important outs than just a one-two-three inning?

Nevertheless, it is not clear to me how the positive value of holding the closer and waiting for a high-leverage situation compares to the negative value of risking the game ending before the closer comes in. And unfortunately, my sim is not sophisticated enough to make this comparison.

The conclusion that I can still draw from this is that the question of when to bring in the closer as an away team in a tie game is not an obvious one, despite the claims of many. A team with Mariano Rivera and a bunch of average relievers wins about 1 percent more if it brings in Mariano early rather than late, but the assumptions that come with that number are not ones to take lightly. The answer, realistically, is probably different depending on the situation, the quality of the bullpen, the quality of the offense, and so on.

Kenley Jansen came into the game in the highest leverage situation for Dodgers pitching, but in doing so, he did not come in during the four previous innings, each of which could have ended the game without him touching the ball, not to mention that there were plenty of high leverage situations previously. It seems fairly clear that Jansen was not used ideally in the game, but it is not clear what the most ideal usage would have been, especially once the ninth inning had passed. Maybe it’s a question that can be answered, but it’s not one that I will, or can, answer here. I’ll leave that up to you.

Bill said...

There’s a mistaken assumption here, namely that reserving the closer for the save situation “wastes” the top arm in the bullpen by not using it (him) in an even higher-leverage role. The reason it’s mistaken is that in almost exactly half of all major-league bullpens, the closer is NOT the top arm. The fraction varies according to which metric you use to judge how good a reliever is (and I’m not going to get into that particular holy war here), but depending on the metric, anywhere from 14 to 17 teams have at least one reliever who is clearly better at recording relief outs than the closer. Some teams have several. Many of the other 13 to 16 have closers who are just a tiny bit better than their other top relief options; the “Mariano Rivera and a bunch of average relievers” situation you cite as a straw man is extremely unusual.

Ironic, to see this column not too many years after most sabermetrically-aligned pundits were raking teams over the coals for not using their best relievers in the high-leverage situations rather than as closers. Now that many of them actually do so, there is STILL pervasive grumbling about closer use.

Professor Longnose said...

Very interesting. I had assumed that holding the closer increased the chances of losing the game before the best reliever pitched, but hadn’t thought about higher-leverage situations later in the game.

Of course, that’s not generally taken into consideration. Mattingly’s bringing in his close with two on is the exception, not the rule. I think in most cases your start-an-inning-and-pitch-one-inning restriction reflects what managers do.

Down with closers, up with firemen!

Tom said...

To Bill (above):

In this case, Jansen is clearly the best reliever on the team and is the closer. He doesn’t make a complete blanket statement that saber-friendley people think every team should pitch their closer in early than 9th inning situations depending on the leverage index. He was saying Mattingly’s logic revolved around that false premise, but that doesn’t mean the opposite has to be true for it to be false. I would suggest saber-friendly people want the best reliever at the highest leverage index innings (as the author stated), and that some teams’s best reliever is their closer and they are saved solely for the 9th inning. Basically, your argument is not logically sound.

BenMarkham said...

The reason it was a bad idea to save Jansen is that it makes it more likely that you’re team will lose as you continue to use subpar relievers. You cover up how bad it is if you have good bullpen depth, but the Dodgers didn’t. Each inning you choose a subpar reliever rather than Jansen, the more likely you either don’t even use him because you lost, or you have to bring him in a situation with runners on.

Yeah, you saved Jansen for a higher leverage situation, but the higher leverage situation was created by having a worse pitcher starting the inning. By pitching Jansen earlier, on average you push back a situation that you might get into later.

Otherwise, it’s quite simple. After the ninth, no inning is guaranteed. Saving your best reliever for a inning you may never get to (especially when you’re sending weaker arms out there rather than your best reliever) just isn’t a good idea. Jansen could have pitched the 9th and 10th, and then pushed back all the mediocre pitchers later in the game where they were less likely to create a high leverage game in the first place.

Same thing happened last night to Boston. Farell sends Workman, a poor reliever back out for the 9th after pitching an uneasy 8th and lets a runner on. Again, the best reliever is brought it in a high leverage situation, but there was much less of a chance a high leverage situation would have occurred had Uehara started the 9th himself. Again, Farell could have used Uehara for the 9th and 10th and went from there. Perhaps his bullpen would have been weak going into the 11th, but you never get to the 11th if you keep throwing your middle relievers rather than your best guy.

Being a Cardinals fan, I was very happy the manager made the wrong decision in both situations.

Marc Schneider said...

The analysis is based on the assumption that the closer is the best relief pitcher. Every team operates on that assumption although, like any other assumption, it could be wrong. I disagree with the idea that most teams have other pitchers as good as the closer; as a Braves fan, I guarantee they do not have another relief pitcher as good as Craig Kimbrel.

I have never understood the idea that only the closer can save games and, therefore, you cannot bring him into tie games on the road. Apparently, managers believe that not using the closing guarantees that the lead will be blown. I don’t see the logic in that. Once you get a lead, it’s quite possible to patch together 3 outs without giving up a run; in addition, it’s possible that the team scores a bunch of runs so that the bottom of the inning is easy. On the other hand, if you wait for the save situation, you obviously risk losing the game with your ostensibly best pitcher never coming into the game.