Thursday, June 03, 2010
The method’s the thing
Posted by Jonathan Halket at 6:30amAsk yourself to think of a random number—any number. Now think of 1,000 more and write them down (no, don't really do this). We can put it through some statistical tests for randomness and I promise you it will fail, probably massively. Not surprising, really—once the number two is on your brain, it can show up in many ways.
You can ask your computer to do the thinking for you and the numbers will be a lot more random-like. But, barring perhaps a quantum computer, these numbers will be, at best, "pseudo-random." There'll be patterns there somewhere. No problem for you as far as fantasy baseball goes, but it does illustrate one thing: Computers and humans can suffer from the same problems, though perhaps not in equal magnitude.
I've been following some little bit of the discussion on the Cardrunners fantasy baseball site—part Socratic dialog, part Crossfire parlay—on the benefits and pitfalls of constructing a model to use to value players. I call it Quants versus Quaints. Here's why.
The Quants want to formally value a player. There are two steps in this process.
Step One: Come up with forecasts for each player based on past performance, present team, league averages, hair color, whatever you want. These forecasts should include (ideally) not only, say, one number for expected home runs, but rather the entire distribution. What's the chance he hits 10, 20, 30 home runs?
Step Two: Come up with a weighting system. This system inputs the forecasts and outputs a value for the player. What you're doing here is putting relative weights on stats. How much is an RBI worth versus a home run? How costly is risk?
Off the cuff, I think mostly the heat in the discussion is over doing the second part. The Quaints believe in forecasts—though some may also believe in mixing in a healthy dose of gut-feeling here ("so and so is gonna breakout this season"). They don't believe (strongly) in value systems for a variety of reasons. For one, forecasts are forecasts and you can get good ones from other places (like using our Oliver!). But value systems are strongly context-dependent, varying from league to league, and varying within leagues over time depending on standing. They often can vary over the course of a draft or auction, too.
The Quaints want to turn the strength of a value system on its head: its constancy.
All a method is doing is asking you to formalize your reasoning. A Quant prefers to ignore anything he can't formalize, and by formalize I mean computerize (or pencil and paper, if you're old-school). If he's deciding between two players with equal numbers, he wants to give them equal value. If he feels in his gut that one is better than another, he wants to know why and then apply that reasoning to all players similarly.
The reason to have a systematic method of valuing players is that your spreadsheet has no bias—or at least it has only the bias you give it. Anything you can do, it can do. Do you think grittiness is an important stat? Then rate all the players on a 1-10 scale (0-1 scale, whatever you want), use "insert column" on your spreadsheet and put in the numbers. Value it as you like.
The Quaints worry about the constancy. What if, after your do your weights, your system spits out David Eckstein as the 20th most valuable player? Clearly even the most steadfast Quant would go by feel and downgrade Eckstein? "If you stick slavishly to your model," they say, "you might make ridiculous mistakes. If you do not, then you might as well just admit you're a Quaint."
"Ah," reply the Quants, "What's to say that your heuristic value system isn't prone to similarly bad mistakes? Don't owners often overvalue rookies and stolen base threats?" It is common sense versus consistency.
What do I think?
A bad value system is no help. It is no shame to admit that you don't have the time or effort or desire or brainpower to come up with the formula and crunch the numbers to come up with your league and time-specific value system. If the best you can do formally is rank batters by home runs, then you're better off sticking with the eyeball and gut method. But be warned—the human gut is notoriously prone to suggestion. Just like all those number twos, you may not notice that your bias is for players who are also good fielders, but there's probably something like that dwelling in your subconscious.
Even the best value systems are going to be necessarily imperfect. A model is, mathematically speaking, a projection—it cannot capture the infinite of life with its finiteness. But if you really are up to it, you can turn any value system—even a Quaint's ranking—into a more formal Quant system. The value of formalizing a Quaint system is that you can try to improve on it for next time—you can formalize and ameliorate your mistakes.
(Here's a rule of thumb for how you would start formalizing a Quaint system. Getting geekily mathematical for a second—you can rank your players however you like. Then take all the forecasted stats you like for all these players, include grittiness or whatever you like. Project (i.e. regress) this ordered list onto the players' stats. (Easier said then done, but if you're a quant-minded Quaint, it is doable. I'm not going to get into details here, though.). Bammo! You have your weighting system.
It'd be harsh to say that a Quaint is just a lazy or inept Quant. Fantasy is supposed to be fun, and for some the Quant methodology is akin to doing your taxes. Quantism versus Quaintism isn't dogma, and you don't have to adhere to one or the other slavishly.
If you have a question for the Roster Doctor email here. Emails in simple text with players' full names properly spelled are much more likely to get responses. Also be sure to include your league's player pool (mixed, AL-only, NL-only), number of teams, scoring format (roto, head-to-head, points, etc.), categories, whether or not it's a keeper league, and any other pertinent information.
I tried implementing a system this season for one fantasy league where I used the leagues past top three as a standard against which to evaluate the necessary averages against which I rated players.
For each league scoring category I compared players’ projected totals to the average player on a top 3 team from the previous year. Then looking at each stats standard variation amongst draft-able players I weighted each to even out discrepancies caused by overly high or low totals in a single category.
This system is extremely helpful in both creating a draft strategy and evaluating moves in season, but it really needs an ADP component to be thoroughly effective. Otherwise you would windup with a lot of foolish overdrafts if you followed it blindly. CHONE’s projections (yes, I should have used Oliver, will next time) loves some low ranked players (Colby Lewis, Eric Young Jr) much more than my fellow draftees. Even within a quant system, a basic “feel” for the market is required.