Monday, September 29, 2008
Why I’m using qualitative risk managementPosted by Victor Wang at 2:13am
In my last post I introduced the risk management process I hope to use in the future when evaluating player risk. In that article, you'll notice that most of the risk assessment takes a qualitative approach. This might seem kind of weird considering that this could be considered a sabermetric site. There are a few reasons why I am taking a qualitative approach with risk.
First of all, the quantitative tools for risk management are just not that advanced right now. PECOTA produces a beta score, Marcels has a reliability score, and Baseball HQ uses a reliability score. Besides that, there is not that much else we can use for risk management. Secondly, even if we did have more advanced statistical programs that could measure risk, I'm not sure it would help us that much, since they would likely use past risk to project future risk.
I believe this approach would just lead to incorrect evaluation of risk as we cannot project risk like we can player performance. Most projection systems now take a weighted average of a player's past performance, regress that to the mean, and then make an age adjustment. This generally works fine and we can get a pretty solid estimate of a player's true talent. Risk, however, does not work like this.
The Wall Street debacle is a perfect example of this. You can look at past risk as much as you want, which some Wall Street risk management systems did, but that is not necessarily going to tell you how risky something will be in the future. This sort of crisis, while it may seem shocking, cannot be considered all that rare on Wall Street.
Anyone who has read the Black Swan knows that somewhere Nassim Taleb is shaking his head at how surprised people were of the latest Wall Street events. This is because finance does not follow the predictable bell curves taught by many statistical classes. Rather, it has large jumps that go up and down. And when those large jumps go down, seemingly shocking things can happen.
So how does this all apply to baseball analysis? I don't think risk management for baseball is quite as random as the areas in the financial markets, but I don't think it's quite as predictable as a something that follows a bell curve distribution. Therefore, I don't think developing a statistical model for risk management will alone be able to help us solve our risk management needs.
With that being said, there may be some areas that statistical methods can be applied to. For example, Sig Mejdal, now of the St. Louis Cardinals, has developed a regression model to predict injuries. Also, Will Carroll has developed a statistical injury risk system. Still, I don't think we're quite at the point where we can quantitatively predict all areas of risk. This is why I have begun to use more qualitative analysis.
I hope we'll be able to develop some quantitative tools in the future but for now this qualitative approach is going to be the center of my basic approach for managing risk. I hope to write a series of player risk profiles in the future, similar to the Troy Tulowitzki article, to help provide you all with a framework for measuring the risk of players. Also, I hope to write about how we can actually use risk ratings to effect the evaluation of player value and ranking.
Remember that managing risk doesn't necessarily mean we'll eliminate risk, it just means that we use risk to give us preferrential situations. If you have any comments or ideas, I'd love to hear from you.
Victor Wang's work on OPS has been featured in SABR's By the Numbers magazine, and was the 2007 recipient of SABR's Jack Kavanagh Memorial Youth Baseball Research Award. He can be reached via email here.