Thursday, July 16, 2009
For those who still don’t believe FIP is bad for fantasy analysis…Posted by Derek Carty at 4:11pm
Our friend Brian Joura of RotoGraphs posted an article today citing my own article about the problems with FIP from earlier in the year. My assertion from then, which I still stand by completely:
While the original, underlying premise for FIP is sound, and while it’s absolutely better to use than simple ERA, and while there are certainly uses for FIP in some circumstances, for 99 percent of fantasy purposes, I ignore FIP completely and absolutely.
I noticed a few comments to Brian's article that didn't seem to completely buy my explanation, so I thought I'd run some quick numbers to help provide further evidence that a stat like LIPS or xFIP is better than FIP.
By definition, the only substantial difference between FIP and xFIP is that xFIP adjusts each stat line to assume a league average HR/FB, so this crude study will focus entirely on HR/FB.
I looked at all pitchers with at least 12 games started in adjacent seasons from 2004 to 2008. Over this period, we find 63 pitcher seasons where a pitcher's HR/FB strays at least four percent from league average* in Year 1. In Year 2, just 5 of those 63 pitchers (7.9%) failed to regress in the direction of league average. That's a very small number, especially when you consider that Chien-Ming Wang (who may be one of the rare exceptions I mentioned) and Brett Myers (who almost certainly is one of those rare exceptions) accounted for 2 of those 5 seasons. Exclude them, and the percentage becomes 4.8%.
This is a very crude study, but hopefully it reestablishes my point. HR/FB is unstable and because FIP makes no alterations, it will be misleading and less accurate than other indicators. David Gassko did some much more thorough work on HR/FB in the THT Annual 2007 (which can be read for free here), but the short version is that for pitchers with 350+ TBF, the previous season's HR/FB explains just 3% of the variance of the following season's HR/FB.
*I used a rough estimation of league average, using the aggregate league average for all five years. This is the lazy way to do it but won't change my point.
Anecdotal evidence and precision
One comment from Brian's article that I thought would be useful to answer for everyone:
"Well…FIP definitely helped predict Ricky Nolasco’s turnaround. Not sure what his xFIP was…."
We must remember that FIP is not so utterly useless that it will be incorrect in every scenario. In scenarios where the pitcher has a lucky or unlucky BABIP or LOB% (Nolasco's BABIP was over .400 at one point), FIP will be able to predict the general direction the pitcher's ERA should move as long as the HR/FB isn't too far away from league average.
While we'll know that Nolasco isn't a 6.00 ERA pitcher, it is important to make a distinction over whether his ERA should be 4.50 or 4.00 or 3.50. Even the difference between a 4.25 and 4.00 ERA is the difference between 'solid starter' and 'waiver wire material' in many leagues. FIP is ill-equipped to make this distinction.
We can't allow anecdotal evidence to rule our decision making. While FIP may have worked in Nolasco's case given a very rough objective, the numbers tell us that a stat like xFIP or LIPS will be more accurate, for more pitchers.
Derek Carty, 23, has also been published by NBC's Rotoworld, Sports Illustrated, FOX Sports, and USA Today. This season, he'll be contributing to FanDuel and will be linking to all of his work at DerekCarty.com. In his three years competing in expert leagues, he has won 2 titles with 4 top three finishes, including a LABR NL title in 2009, making him the youngest person to ever win a major expert league title. Derek is a proud graduate of the MLB Scouting Bureau's Scout Development Program and is a firm believer in the importance of combining stats and scouting. He welcomes questions via e-mail, Facebook, or Twitter.