Prediction Markets Reduxby John Beamer
April 16, 2007
Let’s cut to the chase—take a look at this.
That is a live win expectancy chart courtesy of Tradesports—it actually tracks the recent Braves/Mets game when John Smoltz out-dueled Tom Glavine in a 5-3 Braves win. The graph is similar to the myriad win expectancy charts you see on Fangraphs and other blogs, but rather than calculating the data using math, the Tradesports chart is based on what hundreds of people think and bet the likely game outcome will be. Welcome to prediction markets.
We’ll come back to these Tradesports charts later, but let’s first back up and refresh our memory as to what win expectancy and win probability added are and how they work.
Win Expectancy 101
Over the last couple of years, one of the new-found fads of the baseball blogosphere has been win probability added (WPA) and its close cousin win expectancy (WE)—we at THT have chronicled the insights and implications of WE/WPA on many occasions: here, here and here to name three instances.
In case you can’t be bothered to read our other gems on the subject, let me give you the skinny on these two measures. Every play in baseball, in some way, oftentimes small but sometimes large, contributes to whether a team wins or loses the game. This concept is the foundation of win expectancy.
It is best illustrated by considering two extremes. First, take a come-from-behind-walk-off home run. Before the play the outcome of the game is uncertain (perhaps the home team only has a 25% chance of winning); after it, well, it's home time. In just one play our expectation of which team will win alters dramatically. At the other end of the spectrum consider the first at-bat in a game. If the pitcher mows-down the lead-off man (or gives up a home run—it really doesn't matter) in the first inning we're not going to make sweeping conclusions about the outcome—our expectation of which team will win changes only slightly.
We can extend this concept and work out the odds of a team winning at any point in the game given the particular situation (normally limited to score, inning and base/out state for simplicity). This is how we define WE. If you want to know how to work out WE check out Studes’ awesome spreadsheet. As a team sequences through plays WE changes. The change in WE that results from a particular play is called WPA.
Fangraphs tracks the WE and WPA for each game. Here is the log for the Braves/Mets game that was charted earlier.
Why do we care about this?
Good question. A lot of people don’t, calling it saber-garbage. However, I love it. By looking at a WE chart we get a feel for how the nine innings ebbed and flowed. It is the voice that tells the story of a baseball game—no other tool comes close to doing this so eloquently.
The Smoltz/Glavine graphic at the top of this column charts win expectancy using a prediction market. For those of you who don't know, a prediction market is essentially a binary bet (it has only two outcomes), which is typically run through a betting exchange—it is a futures market for baseball. (Note: Other types of non-binary markets exist, but I'm only looking at binary markets here).
Analysing what prediction markets are saying has been a hobby of mine for sometime. In fact, if you cast your mind back to last week’s column you'll recall we briefly used prediction markets to check the accuracy of the THT division projections. In future, every few weeks, I intend use my column to look at what prediction markets are telling us about our national pastime.
So How Do Prediction Markets Work?
Simply put, a prediction market is a contract that establishes two boundaries to reflect opposing outcomes of a binary bet. The contract then trades between the boundaries. The point at which it trades reflects the market's expectation of the outcome. An expiry date is also built into the contract: In baseball this could be either a specific date or the end of a game or a season.
For example, a particular contract may award 100 points if a certain prediction comes true (team A wins) and 0 points if it doesn't (team A loses). If the market thinks that there is a 75% chance that this prediction will come true the contract will trade at 75 points. In a betting market these points will be linked to cash (ie, 100 points might equal $10, or $100).
Complicated? To see what we mean let’s walk though an example. Take the recent titanic match-up between Daisuke Matsuzaka and Felix Hernandez. You don’t need an eidetic memory to remember that King Felix chucked a one hitter—looks like he is finally living up to his sobriquet!
King Felix vs Dice-K
At the start of the game who do we think would have had the best shot at winning: Dice-K or the King? Tough call I know.
STARTING WIN EXPECTANCY Fangraphs HFA Tradesports Boston 50% 54% 62% Seattle 50% 46% 38%
Fangraphs, because of how it is implemented, states that the win expectancy is 50% for each team. However, we have more information. We know that Boston is at home and we also know that home teams win, on average 54% of the time (this is the HFA column). This assumes that both teams have equal talent. This isn't true. Taking pitching, hitting and fielding together we’re fairly certain that Boston are a better team. Our division projections say as much. Even if you think Dice-K versus the King is an even pitching match-up you’ve got to favor Boston's lumber. Fangraphs doesn’t; the HFA assumption doesn't either. The market does and attributes a price of $62 to the Red Sox contract.
This implies that the Red Sox have a 62% chance of winning before a ball is pitched. From a trading perspective if you buy a contract for the Red Sox to win at $62 you scoop the full $100 if they do, earning a return of $38. On the other hand, if you believe that the Mariners will come through you can sell the contract at $62 (which is the same as buying the opposite contract for $38). Hence if the Ms win you rake in $62.
Now look at what happens as the game progresses.
The red line on the chart represents the win expectancy for the Red Sox (technically the red line is the price of buying a contract for the Red Sox to win), while the green bars indicate volume of contacts traded (a proxy for liquidity). Without watching the game in real time it is difficult to know the precise events that caused the market's view of win expectancy to change. Let me try to help out.
Yuniesky Betancourt drove in the first run at about 7:35. If you look at the trading activity just before this you can see a slight dip as Jose Guillen got on base and then a much bigger dip as Kenji Johjima doubled. At that point the market adjusted its expectation of a Boston win down to 50%. Because Betancourt's RBI was a sac fly, WE didn't change much after the run scored. Huh, Seattle take the lead and the market still doesn't think they're favorite to win!
After that point the contract price varied only slightly reacting to in-game events (and also possibly adjusting after realizing that Felix looked lights out). In the fifth (between 8:10 and 8:30) you can see Seattle’s other two runs score to make it 3-0. At this point the market started to believe the West Coasters would triumph. As it became clear that the King was in charge and as Boston ran out of innings, the market kept adjusting its expectation of Red Sox win down to zero.
So how does this particular market compare to the theoretical WE chart of the same game from Fangraphs?
Hey, fancy that; the theoretical WE chart shape matches the market pretty closely—which isn't great news for arbitrageurs. However, Fangraphs' WE is worked out on the basis that both teams are as equally likely to win at the start of the game, so only considers the score, inning and base/out state in its calculation. A futures market assimilates all of the available information (skill level of the two teams, line up, home advantage, weather, park factors, inning, number of outs, number of bases, hot dogs eaten etc.) and, provided the market is liquid enough, reflects this in the contract price, which should precisely model real win expectancy.
To be honest this game wasn’t the most spectacular in terms of swings in win expectancy. It is better to track a game where the lead changes hands a few times. For instance, look at the Yankees/Athletics game on April 13 when Nick Swisher hit the tying home run in the seventh and Travis Buck scored the winning run in the bottom of the 11th.
The theoretical win expectancy for this game can be found at Fangraphs (you have to invert it to compare to the Tradesports chart).
So, Are Markets Accurate?
Many studies have shown that markets are the most efficient means of determining the probability of an event happening. It is well known that futures markets predicted the outcome of the last two US Presidential elections far more accurately than the usual slew of opinion polls. Provided the market has enough liquidity (those on Tradesports mostly do, despite Congress' best efforts) then the same is true of baseball (and other) contracts.
Last year, for instance, Diamond Mind's team win-loss predictions had a standard error of 7.9 wins. This means that a team projected to win 77 games (ie, a talent level of 77) had a 68% chance of winning somewhere between 69 and 85 games. Although PECOTA may have had a tighter standard error, I am willing to bet that this year futures will beat most comprehensive, publicly available projections. At the end of the season we'll check back and see.
Prediction Markets Elsewhere
Prediction markets in baseball aren't necessarily limited to in-game play. There are myriad other markets that one can bet on. Want to know who is favorite for the NL West? Or whether Barry Bonds will beat Hank Aaron's HR record? Or whether David Wells will chow down more than three burgers in the seventh inning stretch? Prediction markets can tell you.
One instance of where such a market may be useful is in assessing impact that a trade has on the probability of a particular team winning their division. One of the most aggressive trades last year was the Rangers' acquisition of Carlos Lee from the Brewers. How many wins did the market expect Lee to add to the Rangers, and were they more likely to make the playoffs? Let's take a look at the contract for Texas to win the AL West before and after that trade happened:
The Lee trade was finalised on the morning of July 28. Early during the day of the 28th you can see that the Rangers gained about 2-3% in additional win expectancy, which reflects the value that the market put on the Lee deal—though bear in mind that the market also hears a lot of other information that affects the price. Unfortunately the Rangers got tonked by Kansas that evening and this gain in win expectancy was erased. After that defeat the market believed that Lee wouldn't make a jot of difference to the Rangers' playoff hopes. The market was right.
This is pretty cool stuff, and the good news is that there is plenty of scope to extend the use of futures markets in baseball. If you can imagine any possible situation with an uncertain outcome we can use a market to understand what the probability of each outcome is.
The Creation of Markets
Where this technique could also come in handy is in play-money markets. Studies have shown that play-money markets are as effective as real money. It is possible to set up a variety of play-money markets which could be used to allow GMs and fantasy managers to assess the impact of particular trades in isolation. Something like this may or may not catch on but, provided there is enough liquidity, would be an invaluable decision making tool. Protrade is a similar initiative to what I am talking about but operates more like a stock market than a futures market—this makes it difficult to work out probability of outcomes as we can only look at relative changes in actual or expected performance and not changes in win expectancy.
Certain fantasy games are great examples of prediction markets. Any fantasy league that involves play-money to bid for the services of certain players is an indicator of the expected performance of that player. Leagues that span several seasons apply a time dimension that allow rookies and prospects to be properly valued. However, we must be slightly cautious as fantasy games are often constrained by specific rules which can easily influence the value of players. For example, if a disproportionate number of points are rewarded for an RBI it may favour drafting players in the middle order rather than lead-off men. Fan bias is also an issue—a Mets fan is more likely to draft Tom Glavine than Curt Schilling.
Next up at THT?
This column is really just an introduction to the potential of prediction markets. I encourage you to zip over to the Tradesports site and follow a game in real-time—it is more fun than sitting in front of the box and recording each play with a clunky spreadsheet or even than watching the live Fangraphs charts. While you're at it hunt around and look at the other markets that exist.
Every so often I’m going to use my Monday slot to chew over the swings in WE for whatever contracts Tradesports happens to be trading. Keep reading.
References and Resources
A big thanks to Tradesports and Fangraphs for making this column possible.
John is an unashamed glory supporter having followed the Atlanta Braves since 1991. He blogs the Braves at Chop-n-Change. He welcomes comments, criticisms and suggestions via e-mail