The method’s the thing

Ask yourself to think of a random number—any number. Now think of 1,000 more and write them down (no, don’t really do this). We can put it through some statistical tests for randomness and I promise you it will fail, probably massively. Not surprising, really—once the number two is on your brain, it can show up in many ways.

You can ask your computer to do the thinking for you and the numbers will be a lot more random-like. But, barring perhaps a quantum computer, these numbers will be, at best, “pseudo-random.” There’ll be patterns there somewhere. No problem for you as far as fantasy baseball goes, but it does illustrate one thing: Computers and humans can suffer from the same problems, though perhaps not in equal magnitude.

I’ve been following some little bit of the discussion on the Cardrunners fantasy baseball site—part Socratic dialog, part Crossfire parlay—on the benefits and pitfalls of constructing a model to use to value players. I call it Quants versus Quaints. Here’s why.

The Quants want to formally value a player. There are two steps in this process.

Step One: Come up with forecasts for each player based on past performance, present team, league averages, hair color, whatever you want. These forecasts should include (ideally) not only, say, one number for expected home runs, but rather the entire distribution. What’s the chance he hits 10, 20, 30 home runs?

Step Two: Come up with a weighting system. This system inputs the forecasts and outputs a value for the player. What you’re doing here is putting relative weights on stats. How much is an RBI worth versus a home run? How costly is risk?

Off the cuff, I think mostly the heat in the discussion is over doing the second part. The Quaints believe in forecasts—though some may also believe in mixing in a healthy dose of gut-feeling here (“so and so is gonna breakout this season”). They don’t believe (strongly) in value systems for a variety of reasons. For one, forecasts are forecasts and you can get good ones from other places (like using our Oliver!). But value systems are strongly context-dependent, varying from league to league, and varying within leagues over time depending on standing. They often can vary over the course of a draft or auction, too.

The Quaints want to turn the strength of a value system on its head: its constancy.

All a method is doing is asking you to formalize your reasoning. A Quant prefers to ignore anything he can’t formalize, and by formalize I mean computerize (or pencil and paper, if you’re old-school). If he’s deciding between two players with equal numbers, he wants to give them equal value. If he feels in his gut that one is better than another, he wants to know why and then apply that reasoning to all players similarly.

The reason to have a systematic method of valuing players is that your spreadsheet has no bias—or at least it has only the bias you give it. Anything you can do, it can do. Do you think grittiness is an important stat? Then rate all the players on a 1-10 scale (0-1 scale, whatever you want), use “insert column” on your spreadsheet and put in the numbers. Value it as you like.

The Quaints worry about the constancy. What if, after your do your weights, your system spits out David Eckstein as the 20th most valuable player? Clearly even the most steadfast Quant would go by feel and downgrade Eckstein? “If you stick slavishly to your model,” they say, “you might make ridiculous mistakes. If you do not, then you might as well just admit you’re a Quaint.”

“Ah,” reply the Quants, “What’s to say that your heuristic value system isn’t prone to similarly bad mistakes? Don’t owners often overvalue rookies and stolen base threats?” It is common sense versus consistency.

What do I think?

A bad value system is no help. It is no shame to admit that you don’t have the time or effort or desire or brainpower to come up with the formula and crunch the numbers to come up with your league and time-specific value system. If the best you can do formally is rank batters by home runs, then you’re better off sticking with the eyeball and gut method. But be warned—the human gut is notoriously prone to suggestion. Just like all those number twos, you may not notice that your bias is for players who are also good fielders, but there’s probably something like that dwelling in your subconscious.

Even the best value systems are going to be necessarily imperfect. A model is, mathematically speaking, a projection—it cannot capture the infinite of life with its finiteness. But if you really are up to it, you can turn any value system—even a Quaint’s ranking—into a more formal Quant system. The value of formalizing a Quaint system is that you can try to improve on it for next time—you can formalize and ameliorate your mistakes.

(Here’s a rule of thumb for how you would start formalizing a Quaint system. Getting geekily mathematical for a second—you can rank your players however you like. Then take all the forecasted stats you like for all these players, include grittiness or whatever you like. Project (i.e. regress) this ordered list onto the players’ stats. (Easier said then done, but if you’re a quant-minded Quaint, it is doable. I’m not going to get into details here, though.). Bammo! You have your weighting system.

A Hardball Times Update
Goodbye for now.

It’d be harsh to say that a Quaint is just a lazy or inept Quant. Fantasy is supposed to be fun, and for some the Quant methodology is akin to doing your taxes. Quantism versus Quaintism isn’t dogma, and you don’t have to adhere to one or the other slavishly.


8 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
KY
13 years ago

Good summary.  I never understood the debate really.  There is nothing a Quaint can do informally that can’t be Quanted.  A model can have something as simple as a column for “I feel like this guy is worth $3 more”.  Which isn’t Quaintlike because of the important distinction of it being an item in a model.  A better debate I’d like to see would be, once you have a set of “expected things the players in the league will do” how do you turn them into rankings or dollar values with the most accuracy.  It seems like there is plenty of room for a debate by people smarter them me between Standings Gain, Z-Scores, repetitive simulations, or something else.

db
13 years ago

Seems to me the quants would have a big advantage in drafting after the season is over, because they value everything.  Ultimately, that is useless, however.  Because of the uncertainty in projections, it seems the real key is player analysis, not valuing individual components.

bflaff
13 years ago

The trite but true critique of any quant valuation is ‘garbage in, garbage out’. Sure it looks definitive to say one guy’s WAR (not picking on WAR, just using it as an example) is higher than another’s, but if you unpack the components, you may strongly disagree with how things are weighted. The weights (and components) are subjective, which really throws a boatload of unreliability into the mix.

KY
13 years ago

This is where I get lost.  “it seems the real key is player analysis”  That’s exactly what a Quant method is, a refined player analysis.  If you have the ability to observe something about a player, you have the ability to write that thing down in a spreadsheet.  “if you unpack the components, you may strongly disagree with how things are weighted”  of course you unpack the model and inspect it, and if you find it is wrong you put it back together the better way.  Number driven player analysis is not a “dummy, same thing for all players” method, at its best its “anything you can do in your head PLUS stuff you can’t”.  I think Jonathan is pointing out that Quants are going to do exactly what Quaints are going to do except they are going to put it in a model to help them keep track of it and do it more accurately.  And that’s really the only difference.

Matt S
13 years ago

I tried implementing a system this season for one fantasy league where I used the leagues past top three as a standard against which to evaluate the necessary averages against which I rated players.

For each league scoring category I compared players’ projected totals to the average player on a top 3 team from the previous year. Then looking at each stats standard variation amongst draft-able players I weighted each to even out discrepancies caused by overly high or low totals in a single category.

This system is extremely helpful in both creating a draft strategy and evaluating moves in season, but it really needs an ADP component to be thoroughly effective. Otherwise you would windup with a lot of foolish overdrafts if you followed it blindly. CHONE’s projections (yes, I should have used Oliver, will next time) loves some low ranked players (Colby Lewis, Eric Young Jr) much more than my fellow draftees. Even within a quant system, a basic “feel” for the market is required.

eric kesselman
13 years ago

Actually most of the strife in our debate has been on prong 1, the player analysis. The ‘quaint’ counter argument has been that projections are so inaccurate that hyper accurate pricing of them (step 2) is worth very little. They argue that to really make a leap forward in advantage play, you would need more accurate projections, not more accurate pricing of our current projections.

Otherwise, I think you describe the positions on crfantasybaseball.com nicely.

Personally I think the argument should have been that quaint edge comes from Step 1 (player forecasting)and quant edge comes from Step 2 (pricing).

From there as you point out, there’s nothing stopping someone from coming in with both a superior understanding of the player pool AND quant techniques. That’s basically what we’ve been trying to argue on CR.

KY
13 years ago

Thank you, I guess I see the Quaint point then if they believe there are Quants out there who spend less time/ability figuring out what players are likely to do that season then a Quaint would.

eric kesselman
13 years ago

Well, like I said thats what I think the argument SHOULD have been in its strongest form. From there it seems its a question of how good the quaints really are at pricing, and how good the quants really are at player analysis and how much of each aspect is really worth more?