Productive Outs And The People Who Love Themby Larry Mahnken
May 12, 2004
Last week, while others were taking a critical look at Productive Out Percentage using run expectancy and other tools, I took on the statistic on it's own terms. I researched the past two postseasons to see whether productive outs were playing a significant role in the success of the past two World Champs, the Angels and Marlins. What I found was that while both teams had high POPs, it wasn't a crucial element in their success, and correlated very poorly with postseason success.
I don't know if Buster Olney knows of my research, but I don't expect it would have changed his mind, or that of any fans who think about the game like him. The target for this study is the unbiased fan, one who didn't have an opinion going in.
In the past week, I've gotten a great many e-mails about my article. While I couldn't respond to many of you, I did read them all. This week, I responded to a few of my readers' questions about the value of productive outs, when they should be used, and how the Yankees really won in the late 90s.
You mention that it does not indicate the runs it creates. Wouldn't a more telling stat be something more along the lines of a "productive at-bat"? Simply, an at-bat that contributes to a run scoring. Sacrifices only count when the runner scores as a result of being in scoring position. Singles and walks (non-out at-bats that don't immediately score a run) that later score a runner as a result of being in scoring position would be included. I would also include the non-out at-bats in which the subsequent RBI producer appeared as a direct result of the previous non-out at-bat. Ex: Two outs, man on third; Batter A walks, Batter B hits a double that scores runner on third. Batter A gets a "productive at-bat."
Any thoughts on this?
- Jarad Smith
While better than Olney's idea, it suffers from some of the same flaws as productive outs -- primarily, that it treats all events as binary outcomes: good or bad. It's not that simple.
The events of a baseball game do not happen in a vaccum, their value fluctuates depending on what happened before and what happens after. Because baseball teams bat in order, the events of one inning have an impact on the innings that follow, too.
For example, Batter A's walk allowed Batter B to come to the plate and double home the run, but what if Batter C had walked the inning before and been stranded? Had he instead made an out, Batter A would never have come to the plate in the inning and the run wouldn't have scored. Do you give him a productive at-bat?
A better idea is to assign a weighted value to each event, which is what Linear Weights and Run Expectancy Charts do. It's not perfect, because, as I said, the value of each event is different depending on the situation that surrounds it. But the values are close enough.
The value of a productive out is minimized by the fact that it's production is accompanied by an out. According to Run Expectancy Charts, there are only three situations where a productive out increases your expected runs scored: bases loaded with nobody out, runner on third with one out, and first and third with one out. Those are the only three situations in which a productive out is a good thing. In any other situation, it can only be positively described as "less bad."
Of course, depending on the batter at the plate, and the lineup he plays in, there may be other situations where a productive out is a good thing. The next letter brings that issue up:
First, I loved your example of the Twins/A’s game from 2002. That was an excellent illustration of where a bunt would not have been better in that situation. The fact that AJ did hit the homer off Koch provided the, as it proved to be, insurance necessary.
I do understand the point you are making. But I have to ask you this. What if Hocking had been batting in the exact same situation, runner on 1st, no one out? I think if Hocking had been batting with AJ on deck, it would have been very likely that Gardy would have had Hocking bunt to get Mohr into scoring position for AJ.
Or, what if AJ had swung at the first pitch, a low riding fastball near the dirt (which with AJ was always possible) and hit into a 6-4-3 double play, leaving Hocking up with no one on.
Or, what if Rivas was up in that situation? (I know, I know, I’d pinch hit for him too, but for the sake of argument, let’s pretend Rivas was actually a good defensive 2B and with the 1-run lead, Gardy decides he wants his best defensive lineup out there for the bottom of the 9th.) Rivas has a propensity to ground into double plays. I would certainly take my chances with him bunting and sacrificing the out, just to keep out of that situation.
I guess what I’m saying is this, I think there are a lot of ways to win baseball games, regular season or playoff. Managers have a lot of decisions to make, and they have to make them on the fly. Personally, I would hope that a manager would at least have read through some statistics before the game. However, I have no problem with a manager just looking at a situation and going with a gut feeling. As I’ve said before, someone like Neifi Perez could some day get a big single in a “clutch” situation against someone like Eric Gagne to win the game (this is an extreme example, I realize) and that is what is so fun about the game of baseball. You just don’t know!
I guess that’s my only negative with the whole Sabermetric thing right now. It makes it sound like there is only one way that a person (GM, manager, media person, or fan) should think.
- Seth Stohs
Well, there is only one way a person running a baseball team should think: how can we best win ballgames? (Media people should seek truth and entertainment, fans should seek fun; those are their roles.)
There is only one best way. Statistical analysis is not the best way.
However, the best way must include statistical analysis, and it must play a major role.
Baseball is not predictable in small samples, and statistics are never going to be able to tell you what's going to happen in each at-bat. But neither is gut feel, or whatever method you want to use. But just because gut feel and statistical analysis are imperfect doesn't mean they're equal.
Over the long haul, events will happen at a fairly predictable rate, which is what statistics tell you. Gut feel, on the other hand, has no predictive qualities whatsoever. That doesn't mean it won't end up working, it can, even over large samples. But -- and I know I'm going to get more e-mails from this -- when it comes to judging a decision, results aren't important. Nobody knows what is going to happen in the future, so the decisions that are made need to be judged on the information available at the time of the decision. Statistics provide information, gut feeling provides none.
There are times when a sacrifice bunt is not only a good play, but statistically the best play. But in order for a manager to always make the right call, he needs to both have available to him and the ability to understand vast amounts of data. Considering that the sac bunt usually isn't a good play, and that the advantage of doing it isn't great, a team is better off finding the best leader of men to run the team (a far more important attribute in a manager), and tell him "don't bunt."
In the situation you described, where the Twins lead by one run in the top of the ninth with the leadoff batter on base and no out, and a rift in the space-time continuum has wiped out the Twins' pinch-hitting options, leaving Luis "Oh-For-Three-Vas" to bat, I would not bunt. Not because I think the Twins' chances of scoring would be better with Rivas swinging away, but because one run is not that important in that situation.
Had Rivas bunted, the Twins' chances of scoring the runner on first would have gone up, but their expected number of runs scored would go down. Already holding the lead, scoring one run should no longer have been a priority, scoring as many runs as possible should be. Rivas hits into a lot of double plays, but even as bad as he is, he's still more likely to get on base than hit into a DP.
Could you please explain to me how one judges in the likeliness is higher if a person will get on base or record an out in a situation?
If the probability is higher that someone will get an out than get on base, wouldn't it be smart to at least make your out a productive one?!
Secondly, you should know as a Yankee fan that last year the Yankees' productive outs were reduced to sac flies for the most part. Look back at the teams, see who struck out more, last year's team or the 98-2000 team, and see who got the ball in play. That makes a difference. Don't go "Stat Mining."
113 Sacrifice Flies and Sacrifice Bunts for the 1996 Yankees
104 for the 1997 Yankees
91 for the 1998 team
75 for the 1999 team
66 for the 2000 team
73 for the 2001 team
64 for the 2002 team
60 for the 2003 team
The oddity in this is that the team's wins from 1998 to 2001 seem to flucuate with their amount of sac flies and hits. While the OBP for the team from 1996 to 2003 flucuate a bit, it stays around the relative same. Difference = Strikeouts have increased almost every year.
The success of the Yankees has little to do with getting on base, it has to do with making contact with the ball once you do.
Ask Aaron Boone. (World Series reference).
- Matt Stewart
1996 Yankees: 92 wins
1997 Yankees: 96 Wins
1998 Yankees: 114 Wins
1999 Yankees: 98 Wins
2000 Yankees: 87 Wins
2001 Yankees: 95 Wins
2002 Yankees: 103 Wins
2003 Yankees: 101 Wins
If you want to talk about stat mining, take a look at your own research. The correlation between wins and sacrifices from 1996-2003 is
-.033, almost perfectly random. The correlation between wins and sacrifices from 1998-2001 is .999, almost exact correlation. You pointed out the data that supported your claim, choosing to cast aside data that completely refuted it. That is a textbook definition of data mining.
Back to your first question, unless the batter is Barry Bonds, he's more likely to make an out than get on base. But that doesn't mean that you should try to make a productive out in lieu of trying to get on base.
You're more likely to lose the lottery than win it, so why not spend your dollar on a candy bar? At least you get something for your buck. Now what if the odds of winning $5 were 1 in 3? Would you buy a ticket then? Of course you would, because even though you're more likely to lose than win, you'll end up with more money than you started with if you keep on playing.
Using Tangotiger's Run Expectancy Chart from 1999-2000, we can see what the actual value of events was in sacrifice bunt situations during those years (assuming runners don't take the extra base -- so hits really have a higher value):
1st/0 Out Value Unproductive Out -.380 Productive Out -.228 Double Play -.836 Walk/Single .620 Double 1.099 Triple 1.529 Home Run 1.597 2nd/0 Out Value Unproductive Out -.464 Productive Out -.206 Double Play -1.072 Walk .384 Single .715 Double 1.000 Triple 1.293 Home Run 1.361A productive out is, of course, a better outcome than an unproductive out or a double play, but every positive outcome other than a walk with a runner on second and no outs has a greater run value than the difference between a productive out and the worst possible outcome, a double play (and it's not very common to hit into a double play with a runner only on second).
In other words, the reward is worth the price.
As I said earlier, there are times when the sacrifice bunt is the best play. If you trail by one run in the ninth, or are tied in the bottom of the ninth or the bottom of an extra inning, your main priority should be to score one run. If you don't score the one run in the former situation, you lose, and if you score one run in the latter situation, you win.
According to the play-by-play data from 1974-1990, using your first out to move a runner up increases your chances of scoring, by 3% if the runner moves from first to second, 7% if he moves from second to third. In those situations, the bunt is the wise move, and depending on the batters at the plate, may be a wise move in the ninth.
But this isn't particularly relevant to what Olney wrote about. Productive Out Percentage doesn't differentiate between productive outs in the ninth inning and productive outs in the third. It also doesn't differentiate between productive outs in a close game or a blowout, or a sac fly and a runner-advancing grounder, or a pitcher's sac bunt.
Like I said, if the stat was calculated differently, it could have some value, but as it's currently constructed, it's worthless. Getting productive outs in the situations that call for them helps you win, getting productive outs whenever you have the opportunity doesn't.
Finally, going back to the Yankees, here are their POPs and OBP in Productive Out Opportunities broken down by type and year that I researched last week (I left out opportunities for the pitcher to sac bunt with one out, because that's stupid):
RUNNER ON/NO OUT RUNNER ON 3RD/ONE OUT TEAM OPP POP OBP OPP POP OBP 1998 52 .278 .308 27 .185 .333 1999 42 .259 .357 23 .043 .522 2000 57 .195 .281 27 .259 .481 2002 14 .300 .286 9 .111 .444 2003 57 .341 .281 21 .190 .286It doesn't look to me like their productive outs were reduced to sac flies. And it seems pretty obvious that the championship teams were getting on base at a better rate, too.
My faith in Jim Kaat and Paul O'Neill is shaken.
Larry Mahnken is a staff writer for The Hardball Times, and co-editor of the Replacement Level Yankees Weblog. You can contact him with your comments, questions, romantic propositions and incoherent rantings at DLMahnken@hardballtimes.com.