Player Search 

Order Now


The Hardball Times Baseball Annual 2010 is now in development and will ship in mid November! This year's book will feature articles by THT's staff as well as Bill James, Rob Neyer, Tom Tango and Craig Wright. If you use this link to purchase the Annual, you will be in the first group to receive it and you'll be supporting THT.


And here's the full roster.
Got a question for our fantasy baseball experts? Email us:

Heater Magazine

Add 10 MPH to your fantasy team — see for yourself
  • In-depth team pages
  • Players organized by position and league
  • Next Week Forecasts
  • Playing-time trackers
Plus Derek Carty's "LABR Diary" & daily emails of The Rundown

HEATER MAGAZINE
Winner, 2008 CBS Sportsline Fantasy League of Experts





Plus our Statistical Definitions

Most Recent Comments

Monthly Archives



Gear up for baseball season with Chicago White Sox tickets and New York Yankees tickets. LA Angels tickets, Houston Astros tickets, and Atlanta Braves tickets are hot sellers! You can get Boston Red Sox tickets, San Diego Padres tickets or Chicago Cubs tickets for your favorite baseball fan. Coast to Coast Tickets has the best MLB tickets like Minnesota Twins tickets, LA Dodgers tickets, Milwaukee Brewers tickets, New York Met tickets and St. Louis Cardinals tickets.
Find premium Chicago Cubs tickets and other Chicago tickets at JustGreatTickets.com.
Chicago Cubs Tickets
Chicago Tickets



Creative Commons License
All content on this site (including text, graphs, and any other original works), unless otherwise noted, is licensed under a Creative Commons License.

Monday, June 15, 2009

Using FIP to evaluate pitchers? I wouldn’t

Posted by Derek Carty at 2:01am

image
Is Derek Lowe the most overvalued pitcher in baseball this year? If you're looking at FIP he is. (Icon/SMI)
I've seen a lot of articles popping up recently that attempt to evaluate pitchers using the FIP statistic. While the original, underlying premise for FIP is sound, and while it's absolutely better to use than simple ERA, and while there are certainly uses for FIP in some circumstances, for 99 percent of fantasy purposes, I ignore FIP completely and absolutely.

What is FIP?


FIP is a statistic that attempts to estimate what a pitcher's ERA would be based on his peripheral statistics—or rather, the peripheral statistics originally suggested by Voros McCracken when he introduced DIPS Theory. This includes strikeouts, walks, hit-by-pitches, and home runs. While we know that strikeouts and walks are extremely important, eight years after Voros's work was originally published, we can definitively say that home runs aren't entirely under a pitcher's control.

Here's how things work: a pitcher can influence the rate of fly balls he gives up. By this logic, the more fly balls allowed, the more total balls will clear the fences for home runs (all else being equal). However, while a starting pitcher can control the rate of fly balls allowed, he cannot do a very good job of controlling the rate at which those fly balls become home runs (with very few exceptions).

To put it more simply, starting pitchers don't have any underlying ability to prevent home runs—the best they can do is prevent fly balls. If those fly balls are clearing the fence at too high a rate (or too low), we say that the pitcher has been unlucky (or lucky).

And therein lies the problem with FIP.

Pitchers undervalued by FIP


Here is a list of pitchers that have been undervalued by FIP so far in 2009. If you were to look at the FIP of these pitchers, you would believe that they have pitched worse than they actually have.

To explain the table below, we're seeing the pitcher's LIPS ERA (which also estimates ERA but normalizes HR/FB, in addition to some other things), his FIP, and the difference between the two. I've also included each pitcher's HR/FB rate so you can see just how much this stat influences FIP (for the worse). League average is around 11 percent, and you'll notice that just about every player on this list is well above that number.
+-------------+------------+----+----+-------+------+----------+------+----------+-------+
| LAST        | FIRST      | G  | GS | IP    | ERA  | LIPS ERA | FIP  | LIPS-FIP | HR/FB |
+-------------+------------+----+----+-------+------+----------+------+----------+-------+
| Geer        | Joshua B   | 11 |  9 |  58.7 | 6.14 |     4.48 | 5.97 |    -1.49 |   18% |
| Guthrie     | Jeremy     | 13 | 13 |  73.3 | 5.52 |     4.39 | 5.78 |    -1.39 |   17% |
| Colon       | Bartolo    | 11 | 11 |  55.3 | 4.23 |     4.21 | 5.60 |    -1.39 |   19% |
| Harden      | Rich       |  9 |  9 |  49.7 | 4.53 |     3.18 | 4.47 |    -1.29 |   23% |
| Bush        | David T    | 13 | 12 |  74.7 | 4.58 |     4.34 | 5.52 |    -1.18 |   17% |
| Baker       | Scott S    | 11 | 11 |  67.7 | 5.59 |     3.76 | 4.93 |    -1.17 |   17% |
| Young       | Chris      | 13 | 13 |  73.7 | 4.76 |     4.16 | 5.13 |    -0.97 |   11% |
| Johnson     | Randy      | 13 | 13 |  70.0 | 4.89 |     3.61 | 4.55 |    -0.94 |   22% |
| Galarraga   | Armando    | 13 | 13 |  69.7 | 5.56 |     5.08 | 6.02 |    -0.94 |   17% |
| Blanton     | Joe M      | 12 | 12 |  71.3 | 5.17 |     3.97 | 4.86 |    -0.89 |   19% |
| Suppan      | Jeff       | 13 | 13 |  70.3 | 4.48 |     4.88 | 5.72 |    -0.84 |   16% |
| Moyer       | Jamie      | 12 | 12 |  66.3 | 6.11 |     4.97 | 5.81 |    -0.84 |   16% |
| Parra       | Manny      | 13 | 13 |  64.7 | 7.52 |     4.39 | 5.12 |    -0.73 |   15% |
| Davies      | Kyle K     | 13 | 13 |  77.0 | 5.14 |     4.67 | 5.34 |    -0.67 |   14% |
| Eaton       | Adam       |  8 |  8 |  41.0 | 8.56 |     5.29 | 5.96 |    -0.67 |   16% |
| Looper      | Braden     | 12 | 12 |  68.0 | 4.50 |     4.58 | 5.23 |    -0.65 |   18% |
| Oswalt      | Roy        | 14 | 14 |  82.3 | 4.37 |     3.96 | 4.56 |    -0.60 |   15% |
| Slowey      | Kevin      | 13 | 13 |  78.7 | 4.23 |     3.57 | 4.09 |    -0.52 |   11% |
| Sonnanstine | Andy       | 13 | 13 |  70.3 | 6.65 |     4.97 | 5.48 |    -0.51 |   16% |
| Carmona     | Fausto C   | 12 | 12 |  60.7 | 7.42 |     5.58 | 6.08 |    -0.50 |   18% |
+-------------+------------+----+----+-------+------+----------+------+----------+-------+

While true for nearly all players to some extent, if you see analysis done on these players using FIP, take note that the conclusions should probably be much more positive.

Pitchers overvalued by FIP


Here is a list of pitchers that have been overvalued by FIP so far in 2009. If you were to look at the FIP of these pitchers, you would believe that they have pitched better than they actually have.
+-------------+------------+----+----+-------+------+----------+------+----------+-------+
| LAST        | FIRST      | G  | GS | IP    | ERA  | LIPS ERA | FIP  | LIPS-FIP | HR/FB |
+-------------+------------+----+----+-------+------+----------+------+----------+-------+
| Lowe        | Derek      | 13 | 13 |  83.7 | 3.44 |     4.75 | 3.20 |     1.55 |    3% |
| Billingsley | Chad R     | 13 | 13 |  85.7 | 2.73 |     4.07 | 2.73 |     1.34 |    3% |
| Stults      | Eric W     |  9 |  9 |  45.0 | 4.80 |     5.47 | 4.14 |     1.33 |    3% |
| Jurrjens    | Jair       | 13 | 13 |  79.0 | 2.85 |     4.83 | 3.53 |     1.30 |    5% |
| Garland     | Jon        | 13 | 13 |  76.0 | 5.45 |     6.42 | 5.13 |     1.29 |    9% |
| Blackburn   | Nick N     | 13 | 13 |  84.3 | 3.31 |     5.40 | 4.22 |     1.18 |    7% |
| Greinke     | Zack Z     | 13 | 13 |  94.3 | 1.72 |     3.01 | 1.92 |     1.09 |    2% |
| Pelfrey     | Mike A     | 11 | 11 |  65.3 | 4.68 |     5.48 | 4.40 |     1.08 |    8% |
| Lincecum    | Tim        | 13 | 13 |  88.0 | 2.66 |     3.24 | 2.22 |     1.02 |    4% |
| Rodriguez   | Wandy      | 13 | 13 |  79.7 | 2.82 |     4.16 | 3.16 |     1.00 |    7% |
| Hammel      | Jason A    | 12 |  9 |  54.0 | 4.33 |     4.94 | 3.99 |     0.95 |   11% |
| Carpenter   | Chris      |  7 |  7 |  44.0 | 1.23 |     3.34 | 2.39 |     0.95 |    3% |
| Jimenez     | Ubaldo     | 13 | 13 |  82.7 | 3.92 |     4.21 | 3.27 |     0.94 |    6% |
| Meche       | Gil        | 13 | 13 |  75.3 | 3.70 |     4.00 | 3.08 |     0.92 |    3% |
| Happ        | J.A.       | 16 |  4 |  45.3 | 2.98 |     5.34 | 4.42 |     0.92 |    9% |
| Pineiro     | Joel       | 12 | 12 |  76.7 | 3.99 |     4.01 | 3.09 |     0.92 |    4% |
| Verlander   | Justin B   | 13 | 13 |  86.3 | 3.02 |     3.26 | 2.36 |     0.90 |    6% |
| Lee         | Cliff      | 13 | 13 |  88.0 | 3.17 |     4.03 | 3.15 |     0.88 |    6% |
| Miller      | Andrew M   | 10 |  8 |  46.0 | 4.30 |     4.45 | 3.62 |     0.83 |    5% |
| Johnson     | Josh       | 13 | 13 |  89.0 | 2.73 |     3.55 | 2.72 |     0.83 |    6% |
| Washburn    | Jarrod     | 12 | 12 |  76.3 | 3.30 |     4.53 | 3.71 |     0.82 |    7% |
| Padilla     | Vicente    | 11 | 11 |  65.3 | 4.82 |     5.70 | 4.89 |     0.81 |    8% |
| Wakefield   | Tim        | 12 | 12 |  76.0 | 4.50 |     5.50 | 4.75 |     0.75 |    6% |
+-------------+------------+----+----+-------+------+----------+------+----------+-------+

While true for nearly all players to some extent, if you see analysis done on these players using FIP, take note that the conclusions should probably be much more negative.

Evaluating on your own


If you're looking for an alternative to FIP for evaluating players on your own, you have a few options. LIPS ERA is my favorite, and it's now available in Heater Magazine. You could also simply go with xFIP, which appears on every THT player page and is sortable on the leaders pages. Other free options include stats like tRA and QERA.

Concluding thoughts


Hopefully this cleared some things up for some of you, as I know I've gotten a few questions about FIP this season. For those of you who were already aware of its shortcomings, well, hopefully you at least enjoyed chewing on the leaderboards presented. As always, if you have any questions, feel free to e-mail me or comment.

Derek Carty is a 22-year old fantasy baseball analyst residing in New Jersey. In addition to writing for THTF, his work has appeared at Rotoworld (NBC), Sports Illustrated, FOX Sports, and Heater Magazine. In his two years competing in expert leagues, he has won 2 titles with 4 four top three finishes, including a LABR NL title in 2009, making him the youngest person to ever win a major expert league title. Derek is a proud graduate of the MLB Scouting Bureau's Scout Development Program and is a firm believer in the importance of combining stats and scouting. He welcomes questions via e-mail.


Comments

Derek Carty said...

Drew,
I think John covered your question pretty well.  As I qualified in the article, some pitchers do indeed vary from this general rule, but not very many and not by very much.  Also, because HR/FB is so unstable, we really need to see several years of a pitcher beating the rule for us to say he has some sort of knack for it.  As to Ubaldo, he’s only really bested a league average HR/FB for one and a quarter years now, so I don’t think we can call him an exception yet.  Check out Brett Myers, though.  There’s very little doubt that he is an exception to this rule (for the worse).

Kampfer,
FIP probably should be closer to ERA in this table because there’s some selection bias at play.  On a full list of pitchers, this wouldn’t be the case.  In this table, pitchers with abnormally high or low HR rates are chosen, and while LIPS essentially ignores HRs, FIP does not.  So while both FIP and ERA are counting HRs, LIPS is not, which is exactly what you picked up on.

As to year-to-year consitency, David answered you pretty well.  If you’re seeing year-to-year consistency, it’s coming from an exception like Brett Myers or from players who are hovering around league average, which is what we’d expect.

Posted 06/15  at  08:04 PM
Derek Carty said...

Eric/OR,
It absolutely would, which is why it’s frustrating to see analysts continue to use FIP!  I do prefer LIPS, though, because it’s methodology is sounder, accounts for more things, and tests I’ve run have shown it to have a higher predictive value.

Colin,
I would have addressed you the same way John and David did.  It depends upon the question you’re asking, and I too have some reservations with using the in-season ZiPS projections.  A few I’ve seen have looked a bit strange.

Also, for fantasy purposes, it’s sometimes useful to know how a pitcher is performing *now*, not necessarily his true talent level (or our estimation of it, rather).  At this point in 2008, Cliff Lee’s true talent ERA might have been in the high 4.00s, but that’s not to say he wasn’t worth owning on a fantasy bench to see if he could continue with whatever LIPS ERA he was posting at the time (probably well below 4.00).

As to a test of different ERA estimators, I did run some this past off-season on my own.  Perhaps not the most rigorous possible, but it was pretty well constructed if I remember correctly.  It found that just about every ERA estimator tested (besides ERC, I believe), did better than FIP, for whatever that’s worth.  Maybe I should put together a really good study and publish it.

Posted 06/15  at  08:06 PM
DT said...

excellent article and comments by everyone. 
I would love a detailed explanation of LIPS vs xFip if possible.

Posted 06/15  at  10:29 PM
Colin Wyers said...

“Also, for fantasy purposes, it’s sometimes useful to know how a pitcher is performing *now*, not necessarily his true talent level (or our estimation of it, rather).”

If that’s true - and I don’t know that it is, necessarily - then we certainly should be FIP, shouldn’t we? If what we’re after is actual value, not true talent level, it’s wrong to regress the HR/FB. The reason to use FIP or DIPS in that case is not to regress the pitcher’s performance to the mean, but to regress the pitcher’s FIELDER’S performance.

In other words, we use FIP to measure quality of pitching performance right now, not because pitchers have little control over BABIP but because the rest of the defense has a large amount of control over BABIP.

As far as a measure of true talent - I think it matters by how much. FIP has many advantages over LIPS, in that it’s simple to calculate and can be done with official statistics. It’s a “man portable” sabermetric tool, and that explains a lot of ubiquity. It’s not enough to simply assert that it’s better; it should be demonstrated that it’s enough better to justify the step up in complexity.

And at a certain level of complexity, to where I can’t easily calculate it myself - why shouldn’t I just use an updated projection instead, be it ZiPS, Marcels or otherwise?

I think the sort of tables presented here would have worked much better as a sidenote to the sort of study you discuss, rather than as a prelude. There are wide error bars around all ERA estimators (the study I did showed something like a 1-run average error on pretty much everything tested), so I’m not sure how much stock to put in any of the differences shown.

Posted 06/16  at  12:59 AM
David Gassko said...

Colin,

What Derek meant (I believe) was that sometimes a pitcher’s performance is not in-line with his track record. What we want in that case is a statistic that can tell us whether or not his performance thus far has been “real,” in so far as his peripherals backing it up. LIPS is a more useful tool here than FIP. Yes, an updated projection is best but (1) There aren’t any top-notch regularly updated projections out there as far as I’m concerned, and (2) An updated projection can’t give you the same kind of scope that LIPS might. It tells you the pitcher’s most likely talent level at the moment, but it won’t tell you that he’s been blowing that away this season.

Posted 06/16  at  01:11 AM
John Burnson said...

To rely on FIP, you have to believe that a pitcher deserves his HR/FB, often wacky in only 2+ months of play. If you want something simple, swap out HR for 0.11*FB (or whatever), which is xFIP.

Posted 06/16  at  01:13 AM
SharksRog said...

The numbers here indeed make Tim Lincecum seem overvalue by a LOT—but is he?

I agree that Tim’s 2.22 FIP is too low—although his ERA after his first two of what may have been weakened starts is a surprisingly close 2.15.

What I don’t agree is that his LIP is closer to how he has really pitched than his FIP.

First of all, Tim’s ERA of 2.66 is actually closer to the 2.22 FIP than the 3.24 LIP.  Second, that 2.66 ERA includes a BABIP against of .338.  Tim’s bases per hit are among the lowest of all pitchers and watching him pitch, he has given up a TON of hits that weren’t well struck.

Tim’s 23.3% line drive rate is high and would actually support a BABIP of slightly HIGHER than .338, but many of those line drives have been nearly as much fliner/flies and fliner/liners.

In Tim’s rookie season, he allowed 1.61 bases per hit. Last season he reduced that to 1.43 bases.  This season he is down to 1.35.

Meanwhile his BABIP against has risen from .295 to .313 to .338.  As I look at how those numbers fit together, it appears to me there has been less and less solid contact against Tim and more and more floinks that have become base hits.

This season Tim has allowed 16 doubles, but only 1 triple and 3 homers in 13 games.  Close to half of those doubles haven’t been hit very hard, nor have an even higher percentage of his singles against.

I think Tim’s FIP is closer to how he has actually pitched and that despite a possible mini-slump before month’s end, his ERA at season’s end will be closer to 2.22 than to 3.24.

By the way, lest you think that as a Giants fan I favor their pitchers, Matt Cain has pitched closer to his 4.04 FIP than his 2.39 ERA.  Matt’s 88.2% strand rate will almost certainly be close to a dozen points lower by season’s end, and his ERA will jump accordingly.

Matt’s hits and walks against are close to his career averages and his homers are at an all-time high rate for him.  Although he has pitched well in his last five starts and may be on his way to actually PITCHING better the rest of the season, his OPS against is just higher than in 2007, when due to a lack of run and bullpen
support, he went only 7-16 with a 3.65 ERA.

Some have posited that Matt’s home run rate is up because he has had more leads in the past and may have been just throwing strikes in blowout games.  In reality, each of his nine homers against have been in close games in which the game was within two runs either before or after the home run.  All but one were within one run.

Matt’s 2.39 ERA has been refracted with mirrors.  I see no reason why Matt shouldn’t start the All-Star game in his first appearance there, but by season’s end, many will wonder how he got that start.  For Matt, unless he truly starts pitching considerably better, the funhouse is ending.

Posted 06/16  at  02:06 AM
Dave Studeman said...

One thing to remember is that xFIP is a park-neutral stat (at least on the home run line), so it won’t predict future ERA well.  It isn’t constructed to do that.

Posted 06/16  at  07:47 AM
Sky Kalkman said...

Also, for fantasy, we don’t really care about a lot of the neutralization, right?  We WANT to know what ERA a pitcher’s skills combined with park and defense will produce.  You want to use something like DIPS, but using actual team BABIP rates and ballpark rates for HR/FB, BABIP, etc.

Posted 06/16  at  10:09 AM
Cyril Morong said...

I think Colin asked about year-to-year correlation of HR/FB rates. Did anyone address? Maybe I missed it or mis-understood something.

Posted 06/16  at  06:01 PM
David Gassko said...

Cyril,

It was answered. The year-to-year correlation of HR/OF is very, very low, so much so that it’s better to ignore home runs completely unless you have many years of data.

Posted 06/16  at  10:41 PM
Cyril Morong said...

Thanks. I was just wondering if there was an actual number like a correlation of .01. I found a correlation of .30 for HR/BFP, taking out IBBs, year-to-year from 2007-2008, for pitchers with 500+ BFP in each year. That seems a little too high for luck. But it is way less than it was for strikeout rate (.78) or walk rate (.67).

Maybe the .30 is due to pitchers tending to be flyball pitchers year in and year out. Do pitchers ever change or do they tend to stay flyball or goundball pitchers throughout their careers?

Posted 06/16  at  10:50 PM
David Gassko said...

Taking pitchers with 350+ BFP, I found a y-t-y “r” of 0.84 for GB/Batted Ball and a 0.17 “r” for HR/OF in the 2007 Annual.

Posted 06/16  at  10:55 PM
Cyril Morong said...

Thanks, I appreciate it.

Posted 06/16  at  11:36 PM
James said...

Will a pitcher not go far from 11% HR/FB since pitchers cannot control how many HR fly out of the park? If so, what is the normal variance?

Posted 06/19  at  12:54 AM
David Gassko said...

Exactly, James. The range of “true” HR/OF is probably around 9-13%.

Posted 06/19  at  09:40 AM
James said...

Thanks! I will be sticking to xFIP since I don’t have access to LIPS. However, I’ll make sure to keep an eye on the stats that xFIP doesn’t take into account, which I normally do anyway!

Posted 06/19  at  10:46 AM
Page 2 of 2  <  1 2

Leave a comment:

Name (required):

Email (required):

URL:

Remember my personal information

Notify me of follow-up comments?



     Next Post:  Roster Doctor - 6/16/09>> <<Previous Post:  Waiver Wire