Those fickle pitchers!

We all know ERA is largely bunk as a stat (well, maybe not really…), but what is the best metric for measuring a pitcher’s true talent and baseline? Is it FIP? xFIP? What about tERA? Or SIERRA (which I think stands for “unnecessarily convoluted statistic which has no greater predictive power”)?

Each has their virtues and limitations. FIP, for example, measures a pitcher’s three most controllable stats: strikeouts, walks and home runs, attempting to find the context neutral talent line. FIP, of course, ignores the batted ball rates of pitchers, which affects the home run data.

xFIP tries to deal with that problem by adjusting the home run rate, which studies show to be a regression function of flyballs, to a regressed home run rate (11.5 percent). xFIP, however, ignores clearly relevant elements such as park factors and others, primarily infield flyball rates (well, at least the traditional formulation of xFIP as it appears on Fangraphs does). tERA tries to deal with this by looking at a pitcher’s batted ball distribution to predict ERA, but that’s hardly context neutral and batted ball distributions require larger sample sizes to offer reliability.

Advantages and limitations abound among the most popular metrics. A good fantasy baseball nerd player always looks skeptically at all of the numbers. While each is theoretically scaled to reflect ERA (and hence be more accessible/powerful), it is important to note that FIP tends to have the lowest mean, xFIP a higher mean and tERA the highest mean.

Irrespective of this scaling differential, however, I have compiled a list of variations among the big three peripheral ERA stats (FIP, xFIP, tERA) for all qualified starting pitchers in 2010 (P_Variance) to give us a look at who the various peripheral stats agree on the most. This, in theory, gives us a list of the “safest” pitchers to forecast; that is, an indexed list of those most likely to perform as expected by any given ERA predictor.

This spreadsheet is intended to help you make better decisions on how to allocate your auction budget on draft day in 2011. For reference, there’s also a variance column (T_Variance) which compares the big three peripheral ERA stats to actual ERA.

You can download the UPDATED pre-sorted spreadsheet by clicking here.

First, a sample of the most desirable guys who seemingly fall within the “you know what you are getting” category (sorted by P_Variance):

     Name         Team     ERA   FIP   xFIP  tERA  P_Var     T_Var
Ted Lilly      - - -       3.62  4.27  4.16  4.21  0.003033  0.090033
Shaun Marcum   Blue Jays   3.64  3.74  3.90  3.77  0.007233  0.011492
Rick Porcello  Tigers      4.81  4.29  4.34  4.43  0.005033  0.055492
J. Masterson   Indians     4.70  3.93  3.98  3.81  0.007633  0.162433
Trevor Cahill  Athletics   2.97  4.19  4.11  4.01  0.008133  0.326533
Roy Oswalt     - - -       2.76  3.27  3.45  3.34  0.008233  0.093500
Jon Lester     Red Sox     3.25  3.13  3.29  3.30  0.009100  0.006092
Ryan Dempster  Cubs        3.96  3.96  3.83  4.04  0.011233  0.007558
C. Carpenter   Cardinals   3.22  3.69  3.84  3.62  0.012633  0.070092
Phil Hughes    Yankees     4.18  4.24  4.32  4.09  0.013633  0.009425
Ricky Romero   Blue Jays   3.77  3.65  3.73  3.88  0.013633  0.009158
Scott Baker    Twins       4.49  4.02  4.10  4.25  0.013633  0.042700
Max Scherzer   Tigers      3.50  3.71  3.84  3.95  0.014433  0.037400
Carl Pavano    Twins       3.81  4.00  3.97  4.20  0.015633  0.025633
Tim Hudson     Braves      2.79  4.03  3.84  3.79  0.016033  0.311358
W. Rodriguez   Astros      3.60  3.50  3.68  3.77  0.018900  0.013225
Cole Hamels    Phillies    3.15  3.67  3.46  3.72  0.019033  0.067133
R.A. Dickey    Mets        2.84  3.65  3.88  3.63  0.019300  0.206467
A. Wainwright  Cardinals   2.42  2.86  3.14  2.93  0.021233  0.091625
Dan Haren      - - -       3.91  3.71  3.67  3.96  0.024700  0.020692
Tim Lincecum   Giants      3.43  3.15  3.21  3.46  0.027033  0.024158

The best buy values of this list seem to be Justin Masterson and Scott Baker. The underlying numbers on these two pitchers seem to agree that both are in for much better and quite useful (4.00 ERA or better) fantasy baseball seasons following terrible ERAs in 2010. I doubt either will cost you much in 2011.

In terms of stable values, Shaun Marcum and Jon Lester seem to take the cake. Their ERAs seem to most match their underlying numbers. They will not come cheap, but they will likely not disappoint either.

Guys to avoid (omitted above, see data file) on this list of “you know what you are getting” include Bronson Arroyo, Matt Garza and Wade Davis. They seem the most likely to regress of the bunch.

And then we have a sample of the wild cards:

     Name         Team     ERA   FIP   xFIP  tERA  P_Var     T_Var
Jason Vargas   Mariners    3.78  3.95  4.82  3.57  0.410633  0.301533
Anibal Sanchez Marlins     3.48  3.38  4.22  3.38  0.235200  0.164900
Mark Buehrle   White Sox   4.28  3.90  4.69  3.88  0.213433  0.146092
Tommy Hanson   Braves      3.33  3.31  4.04  3.21  0.205300  0.145892
Jason Hammel   Rockies     4.76  3.67  3.80  4.45  0.174633  0.271133
Mat Latos      Padres      2.92  3.00  3.36  2.53  0.173233  0.115958
J. Verlander   Tigers      3.38  2.95  3.71  3.08  0.165233  0.114600
L. Hernandez   Nationals   3.66  3.95  4.76  4.37  0.164100  0.231900
Johan Santana  Mets        2.98  3.54  4.32  3.76  0.161733  0.307333
C. Billingsley Dodgers     3.57  3.07  3.81  3.21  0.154533  0.113700
C.Kershaw      Dodgers     2.82  3.07  3.74  3.05  0.154233  0.157267
David Price    Rays        2.72  3.42  3.99  3.27  0.144300  0.272600
Matt Cain      Giants      3.15  3.62  4.18  3.46  0.142933  0.186292
Dallas Braden  Athletics   3.50  3.80  4.41  3.74  0.137433  0.150025
Jered Weaver   Angels      3.01  3.06  3.51  2.79  0.132300  0.091225
Doug Fister    Mariners    4.11  3.65  4.27  3.65  0.128133  0.101467
Cliff Lee      - - -       3.18  2.58  3.23  2.65  0.127300  0.117267
James Shields  Rays        5.18  4.24  3.72  4.40  0.126400  0.365167
Fausto Carmona Indians     3.77  4.11  4.39  3.70  0.120433  0.102292
Gio Gonzalez   Athletics   3.23  3.78  4.18  3.51  0.113633  0.163767
C.J. Wilson    Rangers     3.33  3.53  4.16  3.78  0.100633  0.127933
Mike Pelfrey   Mets        3.66  3.82  4.46  4.20  0.103600  0.131567
Y. Gallardo    Brewers     3.84  3.02  3.42  3.69  0.113633  0.129425
Clay Buchholz  Red Sox     2.60  3.65  4.18  3.94  0.070433  0.484758
Derek Lowe     Braves      4.00  3.89  3.65  4.20  0.076033  0.052567

The best upside gambles of this list seem to be Tommy Hanson (is he a 3.21 tERA guy or a 4.04 xFIP guy?), Mat Latos (his “downside” is seemingly a solid 3.36 xFIP, but he has only one year in the majors) and David Price (is he a low 3’s or low 4’s ERA guy?). Kershaw also seems to have a “pure upside” arm if he’s truly improved his control.

On the other hand, there are some questionable risks: Anibal Sanchez (is he the 3.38 tERA guy or the 4.22 xFIP guy?), Jason Hammel (the 4.45 tERA guy or the 3.80 xFIP guy?) and Johan Santana, who is already coming off injury. Clay Buchholz’s underlying peripherals have changed so much year-to-year that it’s hard to peg his true talent too.

And then there are some guys who are just not worth the risk (at least not in my eyes): Fausto Carmona, Doug Fister, C.J. Wilson and Jason Vargas.

As always, leave the love/hate in the comments.

A Hardball Times Update
Goodbye for now.

Jeffrey Gross is an attorney who periodically moonlights as a (fantasy) baseball analyst. He also responsibly enjoys tasty adult beverages. You can read about those adventures at his blog and/or follow him on Twitter @saBEERmetrics.
17 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Brad Johnson
13 years ago

The variance column is the group of guys you want to spend a little more time scouting. A little visual data (or someone else’s take on the visual data) could go a long way toward answering the “which player is he” questions.

Bryce
13 years ago

Great article. I especially agree with your point on Scott Baker, but think that C.J. Wilson will be okay to go with next year. His league leading walk total should decrease with a full year of starting experience, he does not give up many home runs and thus has a good fly ball to ground ball ratio, and gets strikeouts at a good enough rate. I am a Rangers fan though so that might be wishful thinking, and the thing that caught my eye the most was that next to Cliff Lee his team name was “- – -“ (sniff, sniff… please come back Cliff! Please!).

Brad Johnson
13 years ago

I’d be much keener on Baker in a H2H format than in roto.

John K
13 years ago

if they have different scales why not put them on the same scale?  you can just convert each metric to a z-score, no?

Jeffrey Gross
13 years ago

John,

Z-Scores are not my scientific specialty, so I cannot myself answer that question. Smarter men than me can, perhaps.

I’ve shared the data sheet in this article and if anyone wants to convert the numbers into zScores, I’d be happy to include those above or you could just post them in the comments.

Sorry I can’t be of more help in that regard.

Jeffrey Gross
13 years ago

@Brad,

Quit looking at my chart! smile

How else am I gonna have a chance to win fantasy in 2011?

Matt
13 years ago

Could the “type” of pitcher a guy is help clear up some of the larger variance between the metrics?  For instance, someone with a very low GB rate; or perhaps someone who had an extreme LD rate.  Maybe identifying these things could show a trend?  Or are these things already factored in?

Chicago Mark
13 years ago

What about non-qualified pitchers?  What is the innings limit?  What is the minimum IP to qualify your data?  I sure would be interested in data for Brandon Morrow, Bud Norris, etc.  AND DON”T ASK ME TO DO IT MYSELF!!!  I don’t know the math.

Brad Johnson
13 years ago

You don’t have to, just download the spreadsheet he links to before the first table and ctrl-f for Morrow/Norris.

Dave Studeman
13 years ago

Got to correct one thing. The “traditional” formula of xFIP did not include infield flies, if by traditional you mean original (and the one we carried here at THT).

By the way, I’m not convinced that any of them really do a significantly better job of predicting future ERA than ERA does itself. At least I’ve never found that they do. ERA is far from a “bunk” stat.

Jeffrey Gross
13 years ago

Mark,

Here are the numbers on Norris and Morrow. Norris seems to be a “confirmed” 4.1 kinda guy. He actually has the lowest P_variance in the set.

Name           Team     ERA FIP xFIP   tERA  P_Variance T_Variance
Brandon Morrow         Blue Jays       4.49 3.16 3.63   3.17   0.072100   0.390292
Bud Norris           Astros         4.92 4.17 4.12   4.14   0.000633   0.151225</pre>

PS, take Morrow with a grain of salt. His BB/9 decreased, but he got ahead of batters less often this season. I expect the BB/9 to spike again in 2011.

Jeffrey Gross
13 years ago

Dave,

You are 100% correct. Traditional xFIP uses OFFB and thats how I self calculate. However, Fangraphs uses FB% en toto. Appelman or Cameron found that the difference in the predictive rates are similar, but I misspoke.

Also, ERA, from what I’ve read, takes 2-3 times as long (larger samples) as peripheral stats to reflect actual talent, which is problematic with high turnover. I’ll try and find that study and repost in the comments

Jeffrey Gross
13 years ago

@Matt,

The threshold for this dataset was 150 IP, if memory serves.

Additionally, if you want to look at non-regressed batted ball numbers, xFIP and tERA are the best. xFIP looks at flyball patterns (a function of GB%, also ignores LD%), whereas tERA looks at total batted ball distro. tERA is not regressed to my knowledge, but I could be wrong. By regressed, I mean LD% is adjusted to 19% and the remaining batted balls distributed based on FB/GB rates…kind of like how my xWHIP formula converts ball distros into expected baserunners.

Honestly, however, I should have sorted the data by TBF. In hindsight, that would have been the best method. I am uploading a new file with a TBF threshold, which you can access in the link above.

That will be done later however, so everyone sit tight. I’ll update by Saturday night (busy with Law papers at the moment still…sitting pretty on a long Ted Lilly fantasy article I’ve been writing too…)

Jeffrey Gross
13 years ago

UPDATE.

I am not update the specific listed pitchers in the article above, BUT the new spreadsheet has been uploaded. it features TWO sheets.

First sheet: list of pitchers with 500+ TBF
Second sheet: all pitchers, min. 40 IP (sample size warning!)

Dave Studeman
13 years ago

Found the article you’re probably thinking of, Jeff:

http://www.hardballtimes.com/main/article/how-well-can-we-predict-era/

Bottom line, however, is that if you have at least three year’s data, ERA (better yet, RA) is probably a better predictor of future ERA than any of these component stats.

Jeffrey Gross
13 years ago

Thanks Dave, this was one of the articles I was thinking of.

I’m just referring to short-set data however. Three-years worth, ERA may be fine, but if you are foolish like me and love to fawn over one and two years…well…

But point well noted. ERA is not the junk stat I proclaimed so above.

Jeffrey Gross
13 years ago

I got a separate email that I’d also like to publicly address here:

With respect to the “T_Variance” column, here is how you can use it best. Note RA Dickey, Clay Buchholz, Trevor Cahill, Tim Hudson above. These are players with strong peripherals, but whose ERA and Peripherals are highly out of sync meaning that despite being solid pitchers, they are likely to be overvalued by the masses on draft day. (xprofit < 0)

In comparison, Jon Lester looks solid across the board. He’ll very likely provide the value you pay for him (xprofit = 0)

Justin Masterson, Josh Beckett, James Shields and Jason Hammel seem the most likely to turn a profit on draft day.
Finally, some guys like