When the wisdom of crowds is wrong

Philosophers have been debating free will vs. predetermination for centuries. Many who subscribe to the latter point of view believe that the actions of an individual are the natural outcome of a series of fixed, historical events. In other words, free will is merely an illusion.

Anyone who participated in a fantasy baseball draft or auction before the season started may find that viewpoint surprising. After all, don’t we exercise choice when determining things like whether to ignore Alex Rodriguez’ injury and take him anyway in drafts or whether to take Nick Markakis or Matt Kemp in the 3rd round?

Maybe so.

But consider everything that seemed almost conventional thinking by the time we all showed up to draft. For example, Chris Davis was drafted anywhere between the 54th pick and the 80th pick in almost every draft, according to average draft data. Exactly who determined that Davis was this kind of value — and how?

Understanding the answer to this question helps us make decisions when subsequent, unexpected things happen.

Last October, the 2008 baseball season ended. At the time, most people who participated in fantasy baseball turned their attention to other things — like remembering their girlfriend’s name. Or football.

Meanwhile, a group of hard-core enthusiasts started to assess the previous season. Many began conducting mock drafts. Pretty soon, people like Sean Smith and Bill James and organizations like Baseball Prospectus and BaseballHQ began considering things like a player’s past performance, a player’s age, and peripheral similarities to others who have played the game in the past century, releasing projections for what we might expect in the 2009 season. (Methods vary.) More mock drafts. Soon, the big fantasy service providers like ESPN, Yahoo, and CBS Sports introduced their own rankings and projections. More mock drafts.

By the time the average person who plays fantasy baseball got into a draft or auction room, they were looking at guidebooks and rankings based on collected intelligence gathering and making “choices.”

Perhaps one individual may have decided to dismiss the hype on a certain sleeper. Perhaps another individual may have fixed himself to a player who was largely dismissed. But by and large, the fantasy baseball community arrived at a consensus valuation of every ballplayer in professional baseball.

As James Surowiecki pointed out in his book, “The Wisdom of Crowds,” group decision-making, based on an aggregation of available data, tends to be surprisingly accurate compared to the decisions of a lone individual.

For this reason, it’s very, very unwise to panic and make rash decisions on slumping or surging ballplayers based on what’s happened to date in the 2009 season. Three weeks of baseball represents a small statistical sampling of less than 10 percent of the 2009 season. In most cases, the collected preseason wisdom of millions, who based their decisions on data collected and produced by a handful of experts, will still be the best oracle to what will happen in the final five months of the season.

However, there are some flaws in this group-sourced valuation process.

The first flaw concerns players without a great deal of experience in major league baseball. With a smaller body of work, young prospects are tough to project. Expert forecasters have tried to solve this by making use of “minor league equivalencies,” but these numbers are still not as reliable as actual major league. In mock drafts, deviation on young, inexperienced players tends to be high. In other words, the consensus is less tight.

The second flaw concerns injuries. Nobody can predict them. Of course, some players are injury-prone (and the fantasy world adjusts valuations based on the tag), but any player dealing with a new, unexpected injury becomes less likely to hit a projection target.

The third and last flaw —at least that we can think of at the moment — concerns opportunity. Unfortunately, the fantasy baseball world holds no sway over the decisions of a real-life major league baseball manager. We may see a player as a good bet to reach 100 runs based on a high ability to get on base, but if a fickle manager disagrees and puts the batter last in his lineup, he’ll also become more likely to disappoint. Conversely, a pitcher or batter who we’ve figured had little opportunity and then figures into a promotion, will surpass our expectations.

Consider these three things when deciding to veer off from the masses and make decisions based on a new “reality.”

A Hardball Times Update
Goodbye for now.

8 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Millsy
14 years ago

While you make some good points about the choices and variability in the latter part of the article, the first part is depressing.  It’s something I’ve noticed happening the more I play fantasy.  The game was built on everyone having somewhat asymmetric information.  That has changed, as you say.  It kind of takes the fun away from bragging to your buddies and it seems that winning a Fantasy Leauge has involved more and more luck as the information becomes more mainstream.  Pouring over all these numbers that everyone else sees.  Metrics for prediction being stretched pretty close to the limit of knowledge at this point.  Everyone has access to it.  Kind of leaves you saying, “Well, what’s the point?”.

John Burnson
14 years ago

If you think that we have it bad, imagine being a big-league hitter in 2015, and knowing that the opposing pitcher knows EXACTLY what pitches you like to chase, and the sequence of pitches that never fails to fool you, and the fielders are perfectly arrayed for your hit tendencies, factoring the wind and Coriolis Effect, and the chip in the catcher’s head contains Tango’s Run Expectation table, and he has min-maxed the outcome of every possible throw, and you’ll be lucky to be selling beer by the 7th inning.

birk
14 years ago

John, I think you’re forgetting that the players are human and not Strat-O-Matic cards (to steal a line from Kevin Goldstein). So there’s a “sequence of pitches that never fails to fool you?” You learn that sequence and make an adjustment. If all of the information you mention above is taken into account fully, the game will change. Do we know exactly what will happen after these changes? No, we don’t, but that’s why the game’s so fun to watch.

Plus, you ignored the fact that hitters will know exactly how pitchers pitch to players similar to themselves. New information is not going to completely favor the pitcher. In the end, both sides of the ball should improve, and we get better baseball overall.

John Burnson
14 years ago

(John rolls d20, looks up appropriate response.)

I forgot no such thing! Granted, my note was a bit facetious. Nonetheless, on the spectrum of regarding human beings as Strat-O-Matic cards, and regarding them as infinitely malleable, I incline toward the former. Even without the Information Age, there are plenty of chances for ballplayers to divest themselves of bad habits, but many don’t. (For that matter, have WE dropped all of OURS?)

Also, I don’t know that a game with more known information is “better.” The most entertaining games have a precious balance between whim and knowledge. To me, it’s not impossible that the explosion of data will shift baseball out of its comfort zone—after all, if one agrees with Millsy’s point, why should MLB be immune? Maybe fantasy is the canary in the coal mine. (That’s actually plausible—generations of strategy elapse in fantasy in the time that it takes an MLB GM to discover that rocks make good weapons.)

The wild card is the frontier—what areas of baseball remain to be explored? People tend to under-imagine, and I’m probably guilty of that.

Millsy
14 years ago

When it comes to real baseball, in terms of analysis and getting an edge, innovation is always being explored.  Of course, there is a limit in terms of statistical analysis (especially in fantasy), but any business that isn’t constantly trying to innovate to get ahead of the competition is going to lose out.  The most obvious example is Billy Beane…but he wasn’t the only one.  Other teams caught up, and the next wave of innovation will give a team an edge.  Perhaps it will have to do with ‘team chemistry’ types of analysis.  Who knows…like you say…imagining things is difficult.  Who in 1920 would have thought we’d have high definition television and the internet?  Very few people could even think that up.  The same things could apply to baseball and its research, of course.  But in terms of fantasy, the main thing we have left is developing new formats and taking advantage of that until everyone catches up (like getting all 2-start pitching in the old CBS Head to Head format and only drafting hitters).  It’s a constant game and I only hope maybe I can think up the next big thing.

chattanooga
14 years ago

I agree with millsy.  I’ve been playing fantasy baseball for 20 years, and it was more fun when you could use the knowledge of the teams you followed to gain an advantage over your opponents.  10 years ago, no one would have thought to draft Matt Wieters or Tommy Hanson-type players.  Now those types of players are mid to late round pics.  And worse yet, that kind of farm-system info is available on practially ANY fantasy enthusaist (or pay) website.  You don’t really have to know anything about baseball or the players to draft a competitive fantasy team anymore.

Eric Hinz
14 years ago

I agree with Millsy.  There is little information out there that isn’t broadly known or unknown for much longer than a season thanks to 24/7 off-season coverage as Eriq points out.

While this may sound like heresy, maybe the key to winning fantasy baseball is ignoring all the expert advice and treating players the same way momentum traders play stocks – buy hot and sell cold regardless of peripheral analysis.

Scott
14 years ago

Another problem with the “wisdom of crowds” as it relates to fantasy sports: the theory is largely based on “independent” guesses by individuals.  By the time we get around to mock drafts, everyone has done their research on the major sites, and those initial “expert” guesses color everyone’s thinking.

If we could run a hundred mock drafts with owners that never see a preseason prediction, expert mock draft, or other analysis I bet we’d get an appropriate ADP for Chris Davis.  But after a dozen sites declare Javier Vazquez a “sleeper” he’s not going to get drafted like a sleeper anymore.