Relievers in the roughby Jeff Sackmann
January 21, 2010
What makes a good reliever? It's a complicated question, and if you're putting together a bullpen, it may not even be the right question to ask.
Most relievers didn't start that way. At some point, perhaps in college, the minors, or even midway through a major league career, they were shifted out of a starting role. Sometimes it's due to a 98-mile-per-hour fastball that screams, "Make me a closer!" More often, it's due to ineffectiveness as a starter.
But not all failed starters make it as relievers. Some, like Eric Gagne, become far more valuable in the late innings than they ever would've been in the rotation. Others, like a large percentage of Triple-A relievers, aren't good enough as starters or relievers to contribute at the big league level.
Until you stick a mediocre starter in the bullpen and give him a try, how do you know which category he falls into? Rather than asking what makes a good reliever, let's ask: What makes a mediocre starter a potentially good reliever?
That adjustment both narrows and widens the question. Generally, good pitchers make good relievers. If Tim Lincecum or Felix Hernandez suddenly started pitching out of the bullpen, we could reasonable assume they would pitch well. So really, identifying good relievers is similar to identifying good pitchers.
The other question forces us to look deeper. What makes relief pitching different, allowing some pitchers to successfully specialize? Here are three ideas:
- They pitch well in short bursts. Perhaps they have the stuff (one or two great pitches) to retire batters the first time through the order, but not enough to fool them again.
- They have a large platoon differential. Relievers, even those who aren't known as platoon specialists, can be deployed in a way that protects their weak side. By contrast, an opposing manager can stack his lineup against a starter with a large differential.
- They pitch well out of the stretch. This is part of the job. With the exception of closers and some setup men, relievers are frequently called on to clean up others' messes. Many relievers, in fact, always pitch from the stretch.
- We can look at a starter's numbers his first time through the lineup compared to subsequent times through the lineup. This isn't a great way to identify future relievers, since starters learn to pace themselves. In a relief role, they might throw fastballs harder, for instance. It's still worth looking at, though. Even with less fastball velocity, a starter with a great fastball-change-up combination may be able to cruise the first time through the lineup, but suffer the second and third times as he exposes weaker pitches.
- Platoon differentials are easy to express statistically.
- Assuming that a pitcher nearly always throws from the stretch when runners are on base, we can look at a player's splits with and without runners on base.
As is usually the case with attempts to model hypotheticals like these with statistics, there are limitations. As noted, looking at performance the first time through the lineup doesn't tell us how a starter would throw given only one inning of work. And the third stat presents the most problems: Usually if a pitcher is throwing with men on base, he's already had some problems. But if a reliever comes in with men on base, the fact that he's pitching from the stretch doesn't reflect his own performance.
Despite these caveats, let's press forward and see what we can come up with.
As we approach spring training and teams start to decide whom to invite to camp, let's look at a pool of players who are generally on the cusp: Triple-A starters. Last year, 104 pitchers threw 100 or more innings for their Triple-A team.
First, let's look at how they did each time through the lineup. The average starter in this group posted an FIP of 4.37 the first time through the order, 4.62 after that. About 60 percent of them had better results against the first nine batters they faced.
Here are the 10 who suffered the most in the second time through the order and beyond, suggesting that in many of these cases, two innings might be as much as they should work:
Name Team IP FIP 1st time FIP 2nd+ FIP ratio Brad Salmon Salt Lake 111.7 3.81 6.49 1.70 Clay Buchholz Pawtucket 102.0 2.43 4.02 1.65 Dana Eveland Sacramento 125.7 3.37 5.49 1.63 Gustavo Chacin Lehigh Valley 103.7 3.47 5.24 1.51 Tobi Stoner Buffalo 100.0 3.31 4.86 1.47 Jason Jones Rochester 137.7 4.00 5.85 1.46 Rodrigo Lopez Lehigh Valley 103.3 2.77 4.03 1.45 Carlos Carrasco Lehigh Valley 116.3 3.24 4.64 1.43 Lenny DiNardo Omaha 156.7 2.62 3.74 1.43 Carlos Torres Charlotte 132.7 2.48 3.51 1.42At least at the Triple-A level, some of these guys were solid pitchers regardless of time through the order. But the performance of someone like Jason Jones suggests that a shift to the 'pen may immediately make him look like a much better pitcher.
At the other end of the list is knuckleballer Charlie Haeger, who posted a 7.21 FIP the first time through the order and a 4.41 FIP thereafter. Lest you think it might be a feature of the knuckler, Charlie Zink was a much more expected 5.28/6.72.
Let's do the same for the 104 starters, considering their FIP against lefties and righties. The average starter from this group posted a FIP about 22 percent worse against their weaker side. Here are the ten whose differentials were the largest:
Name Team IP FIP vLeft FIP vRight FIP Ratio Clay Buchholz Pawtucket 102.0 4.63 2.44 1.90 Kei Igawa Scranton/WB 155.3 2.87 5.24 1.83 Douglas Fister Tacoma 111.3 4.96 2.93 1.69 Matt Kinney Fresno 158.3 7.29 4.43 1.65 Charlie Zink Pawtucket 140.7 4.58 7.45 1.63 Jerome Williams Sacramento 119.0 6.99 4.34 1.61 Chuck Lofgren Columbus 100.3 3.50 5.61 1.60 Rodrigo Lopez Lehigh Valley 103.3 4.46 2.83 1.58 Enrique Gonzalez Pawtucket 143.3 6.25 3.98 1.57 Jack Egbert Charlotte 115.0 5.63 3.61 1.56There's Buchholz again! Given that 20 percent of the pitchers in this sample had a ratio below 1.1, it's striking how big some of these gaps are. Maybe, just maybe, Kei Igawa could be successful in a limited major league role.
These pitchers were a bit better with runners on than with the bases empty: a 4.48 FIP compared with a 4.57 mark. There's no clear bias either way: About half of the starters were better with runners on; the other half were better with the bases empty, presumably pitching out of the windup.
Here are the 10 who had the most relative success from the stretch:
Player Team IP FIP None On FIP Men On FIP Ratio Josh Towers Scranton/WB 104.7 5.90 3.53 1.67 Scot Drucker Toledo 114.3 5.75 3.60 1.60 Kris Johnson Pawtucket 103.3 5.62 3.66 1.54 Chris Seddon Tacoma 142.3 6.58 4.37 1.51 Steven Hammond Fresno 163.7 7.35 5.15 1.43 JR Mathes Iowa 136.7 5.12 3.73 1.37 Kevin Mulvey Rochester 153.7 4.68 3.44 1.36 James Simmons Sacramento 117.7 5.09 3.75 1.36 Brandon Hynick Colorado Springs 159.7 5.51 4.14 1.33 Charlie Haeger Albuquerque 145.3 5.89 4.51 1.31For some reason, I have a hard time imagining Josh Towers as a reliever, but at least, he was successful out of the stretch last year.
One odd tidbit at the other end of the list: Chuck Lofgren, chosen by the Brewers in the Rule 5 draft, was by far the most extreme in the other direction, amassing a 3.59 FIP with the bases empty and a 7.59 FIP with runners on. If that tendency lasts, he'll have an even tougher road ahead of him to make the Milwaukee bullpen this spring.
To combine these three qualities, I divided the pool into quintiles and assigned each pitcher a number from 1 through 5 for each category. For instance, the 10 players listed above as the strongest out of the stretch were given a 5 for that category, while Lofgren, at the other end of the spectrum out of the stretch, was given a 1. Thus, the maximum overall score is 15 (top quintile in all three categories) and the minimum is 3 (bottom quintile in all three categories).
No one scored a perfect 15, but there is a 14 and a handful of 13s. Here are the 17 pitchers who scored 12 or above, along with their quintile scores for each category:
Name Team IP Lineup Rank Platoon Rank Stretch Rank Total Rank Chad Reineke Sacramento 140.0 5 4 5 14 James Simmons Sacramento 117.7 3 5 5 13 Kei Igawa Scranton/WB 155.3 3 5 5 13 Tobi Stoner Buffalo 100.0 5 3 5 13 Willie Collazo New Orleans 132.3 3 5 5 13 Wade Davis Durham 165.0 3 5 5 13 Ty Taubenheim Indianapolis 106.0 5 4 4 13 Rodrigo Lopez Lehigh Valley 103.3 5 5 3 13 Charlie Zink Pawtucket 140.7 5 5 3 13 Josh Towers Scranton/WB 104.7 2 5 5 12 Name Team IP Lineup Rank Platoon Rank Stretch Rank Total Rank Scot Drucker Toledo 114.3 3 4 5 12 Andy Mitchell Norfolk 119.3 4 4 4 12 Ramon Ramirez Louisville 130.7 4 4 4 12 Carlos Hernandez Durham 115.0 4 5 3 12 Kevin Pucetas Fresno 165.3 5 4 3 12 Travis Blackley Reno 115.0 5 4 3 12 Gustavo Chacin Lehigh Valley 103.7 5 5 2 12Clay Buchholz was one of the few pitchers to show up on two of the top-10 lists, but because he was considerably better last year with the bases empty, his total was only 11.
Will any of these guys pan out as successful big league relievers? Your guess is as good as mine. But except for a couple of guys here with some prospect sheen, there's a good chance we'll get to find out—or, at the very least, we can expect they won't get many more chances in the big leagues as starters.
We could easily extend this (or a similar) study to the lower minors. Perhaps looking at some of these components for pitchers in High-A and Double-A would be a good way of singling out players worth a pick in the Rule 5 draft.
In fact, this approach may be more valuable the lower the level. By Triple-A, many pitchers who will be relievers are already relievers. At the college level, for instance, the vast majority of pitchers with professional futures are still starters. Perhaps some starters who could excel in relief are currently overlooked.
Most useful would be to do this study for some of the earlier years for which I have splits. As much fun as it is to intuitively pick the characteristics that make for good relievers, it would be much more valuable to know whether a list like this one from 2006 had any predictive value of relief success in the following seasons. We'll examine that in a future article.
References and Resources
Thanks to Kent Bonham for the idea for this article.
Jeff Sackmann is the creator of MinorLeagueSplits.com. With Kent Bonham, he founded CollegeSplits.com. Jeff and Kent blog about college baseball and the draft, and you can follow them on Twitter for bite-sized snacks of minor league and college stats. Jeff also has an email address.