Wednesday, September 09, 2009
Is EqA better than wOBA?
Posted by Colin WyersAaron (YYZ): Doesn't the AL ROY almost have to be Elvis Andrus? He's the 8th most valuable SS (by WAR) in all of baseball (and the 9th best SS offensively by wOBA)
Christina Kahrl: Why use wOBA when EqA's testably more accurate?
BP has tested the accuracy of EqA before; however it's notable that wOBA was not included in the comparison. So I thought I would run a quick test to see if EqA really is more accurate. I used this as my formula for calculating Equivelent Runs and this to calculate runs as per wOBA.
FanGraphs figures wOBA as far back as 1974, and if we extend that far back, looking at RMSE, we discover that in fact EqA bests wOBA by a slight amount in our typical tests, looking at team runs per season:EqR |
wOBA_R |
|
Correl. |
0.97 |
0.97 |
RMSE |
27.3 |
27.6 |
MAE |
21.4 |
21.9 |
I would go ahead and call that essentially a dead heat, but it wouldn't be unreasonable to declare victory for EqA.
But of course the question was about Elvis Andrus in 2009, and from 1980 onwards the advantage shifts to wOBA. Narrowing it down to 1993-2008, the modern offensive era:
EqR |
wOBA_R |
|
Correl. |
0.97 |
0.97 |
RMSE |
28.3 |
27.5 |
MAE |
21.9 |
21.7 |
Looking at RMSE, wOBA is better by about a run per season. I'd consider that more significant overall, and far more relevant to Elvis Andrus.
Colin Wyers knows exactly how much of a nerd he is. He is very interested in hearing about any other concerns you may have; you can reach him by e-mail, and he will try his best to respond in a timely fashion. He also blogs at Statistically Speaking.
The problem is, aggregate accuracy is absolutely not the litmus test of a run estimator.
Teams represent a very narrow profile of hitter. The vast majority will have an OBA .300-.350, a SLG .390-.470, a BA .250-.280, 140-210 HR. Thus, if the contant for a HR were ‘off’ by 20%, it could be balanced out by an offsettingly (hehe… nice word) bad singles constant…. since there is uch little diversity in team btting, it would rarely show a meaningful error.
All systems (wOBA, RC, LWTS, BaseRuns, XR, ETC) are going to do 99.5% as well as the next in terms of estimating team runs (literally).
Put another way…. what if you found that you could make EqA .0002% more accurate by switching the stolen base constant to -2.3 and the hit by pitch constant to 3.1? I’m sure you could find a way to do such a thing. Does that make EqA ‘better’? Well, it makes it more accurate… but those are just numbers pulled out of la-la land, with no basis in things that happen on a baseball field. They will succeed in producing a more accurate (on average) prediction of team runs scored. But when you apply that same formula to any player who falls outside the very narrow profile that teams represent as hitters, all bets are completely off.
Thus, a systems logic basis is king. Ergo, LWTS (or maybe BaseRuns) is king.