Monday, March 30, 2009
The Diamond Mind Projection Blowout 2009
Posted by David GasskoSG posted the results of his annual Diamond Mind projection blowout today, running 1,000 simulations each with six different projection systems to try to predict the 2009 standings. You can find his results for THT here.
Last year, of the six systems SG tested, THT had the best average error at the end of the season (meaning we were closer, on average, to predicting the team's final record than any other system) and the third-best RMSE. Obviously, I hope we finish first in both categories this year.
If we trust that each projection system has something useful to say, one way of figuring out how well a job they're doing is by aggregating their projections and seeing how each system correlates with the overall average. Here are the results for this year:
CAIRO, 0.976
THT, 0.971
PECOTA, 0.970
ZiPS, 0.966
CHONE, 0.938
Marcel, 0.908
You can see that CAIRO is closest to the consensus, with THT a close second. Marcel is the big outlier here, probably mostly because it doesn't know anything about minor league performance, park adjustments, or any other complications the other systems include. We'll see at the end of the season how well each system did in projecting actual performance.
David Gassko is a former consultant to a major league team. He welcomes comments via e-mail.