Dorkapalooza 2009: The sports analytics conference at MITby Sal Baxamusa
March 09, 2009
Every year, MIT's Sloan School of Business holds a sports analytics conference. This year, there were over 450 attendees and 50 media outlets, including The Hardball Times. The conference brings analytics and fact-driven decision-making to the world of sports. Or, in the words of ESPN writer Bill Simmons, it's a complete dorkapalooza. But the melding of the dork minds from across sports made for some interesting discussions at the conference, which was held last Saturday, Mar. 7, at MIT.
|Statheads never gave Tim Purpura much credit, but maybe they should have (Icon/SMI)|
How do you measure success?
"It's not good enough to beat the market index, you have to be better than everyone else."
— Daryl Morey, general manager of the Houston Rockets
That's just it, isn't it? When building a team, whether in baseball or otherwise, the goal is to win a championship. I'm sympathetic to the view that the playoffs are a crapshoot, but hearing a general manager state his goal unequivocally makes you realize that building a sports team is more like managing a hedge fund than a mutual fund. The bottom line isn't just winning; it's being better than every other team. It underlies the whole risk calculus of team building.
"You better deliver something other than a promise of winning. You can win with a bunch of no-name players, but you won't maximize your potential revenue."
—Brian Burke, general manager of the Toronto Maple Leafs
On the flip side, sports is an entertainment venture. It's all about the Benjamins. Speaking on a panel discussing the fan experience, Burke's comments centered around the theme that his job is to entertain consumers of the Maple Leafs. He understands that he's competing with vacations, new cars, and the Canadian Football League. To that end, he believes that marquee and high-skill players have intrinsic value, and that contact and fighting on the ice will encourage fans to come back for more.
There's a school of thought that winning drives attendance. It's a position that I, and I suspect many of you, take. But I'm a hardcore baseball fan and I can't understand why anybody wouldn't come out to support the local nine if they're winning ballgames. My stake as a fan is to share in a team's glory as they win.
But not every fan is like me. There are lots of ways to consume sports. Burke said that his job was to not only get a hardcore fan into the arena, but that fan's grandmother and boss as well. Burke suggested that over the long-term, every team was going to be about .500 anyway. Whether you think that's defeatist, self-fulfilling, or just the nature of the beast, building an entertaining brand to get a team through the lean times was part of his message.
When I was a kid, the Golden State Warriors were never more than a mediocre team. But they were an exciting bunch, and my dad kept taking me to games. Sure they'd lose all the time, but they'd score 120 points while doing so. Burke's a smart, successful guy—maybe he's onto something.
...and I'd like to pay $8 for that beer
"I have a 50-inch plasma TV at home"
—Bill Simmons, ESPN writer
Simmons served as the "voice of the fan" on the fan experience panel. Predictably, he complained about ticket prices. Who would pay money to go to the stadium when he could get better angles, better replays, cheaper beer, and access to email in their living room? It's a fair point, but teams are still selling tickets. So what keeps fans coming back?
Mark Donovan, senior vice president of business operations for the Philadelphia Eagles, defined what he calls "points of contact." From the time you leave your house to the time you return, his job is to make sure that you connect with the team and want to come back. It can as simple as the usher high-fiving you on the way out, or as complex as installing the most advanced HDTV displays in the stadium even at the expense of premium seating.
Burke, building on his earlier theme, gave several examples of ways to connect with fans that might seem bizarre to males 18-35 demographic. In his opinion, "Ob-la-di Ob-la-da" was the worst song ever written and recorded (it isn't; that distinction belongs to "We Built This City"). But it is frequently heard at venues because studies show that fans connect with the song. I guess the baseball equivalent would be Boston's tradition of playing "Sweet Caroline" during every home game despite the song's pedophilic undertones. In-venue voting via fan text messaging has nothing to do with the game, but fans enjoy it. The ridiculous polar bear mascot that the Leafs trot out has value to kids and families.
Donovan summed it up nicely when he said that "Our fans tell our story." The fan experience revolves around making sure that fans tell the right story. I exchange my stories on blogs with other hardcore fans. But word-of-mouth is still a powerful marketing force, especially among casual fans. I find "Sweet Caroline" annoying, but its continued use and integration into the "Fenway Experience" tells me that somebody is doing their job right.
Baseball analysis is dead
"Most of the actions suggested by rigorous analysis that haven’t already been adopted are difficult to sell to a skeptical audience, and inertia usually wins the day."
—Gary Huckabay, in a 2007 commentary published at Baseball Prospectus, famously proclaiming "baseball analysis is dead."
I have no doubt that even the most traditional organization will one day have a small army of quants analyzing their every move. The sabermetric sales pitch to front offices will have to be successful, because front-office personnel are coming from walks of life where a data-driven approach is the norm. Shiraz Rehman, director of baseball operations for the Arizona Diamondbacks, was a commodities trader and financial consultant in his previous life. As a panelist on the baseball analytics discussion, he said that the Diamondbacks have a database with information and video clips of every pitch thrown. They query this database not only for evaluative purposes, but also for advance scouting and game planning. Almost every team uses a similar system.
But can sabermetrics be used to improve the performance of players? That's a much tougher pitch. Tim Purpura, former GM of the Houston Astros, was never known as a sabermetric darling, but he made most of the most important comments of the day. He said that the Astros developed metrics to demonstrate to pitchers the importance of getting ahead in the count. They used it as a way to educate players about their own performance. It would take a special seamhead—or ballplayer!—to convince an athlete to change the approach that brought him to the top of his profession. Credit Purpura for being willing to use data not only to inform his own decisions, but to help his players reach their full potential. Yes, it was something as simple as pitching ahead in the count. But if you're going to bring the data directly to the players, you've got to start somewhere.
The Fielding Bible's John Dewan, the moderator of the panel, asked the panelists what they thought of the defensive shift used against pull-happy lefties. Rehman said that the Diamondbacks were very aggressive with using the shift, and that manager Bob Melvin was very receptive to the data suggesting its use. He said that the team occasionally took a hit with the local media for employing it so frequently, but that it was worth it over the long-run.
Baseball Musings blogger David Pinto said that the effectiveness of the shift at neutralizing lefthanders suggested that more hitters should attempt to dunk or bunt the ball the other way. Rehman was not optimistic, saying that for players it was a buy-in and ego issue that may not be worth broaching. He made a good point: how do you convince a slugger to dunk the ball the other way rather than rip a 400-foot bomb to right?
A few other tidbits from Rehman:
- He said that a greater understanding of replacement level and the talent available in the minor leagues helped drive the decrease in the value for contracts to veterans.
- The Diamondbacks use analytics as a way to value players, but that they had no large scale effort to evaluate the effects of leadership or team chemistry.
- My scientist's heart melted when he said that the objective process is just as important as the analysis itself.
Christina Kahrl of Baseball Prospectus mentioned that they had had finally updated their definition of replacement level and added a play-by-play defensive metric. It was nice to hear that Baseball Prospectus had responded to the constructive criticism offered by the sabermetric community over the last few years.
Basketball analysis is alive
"I code a lot in R"
|Should this man be holding a baseball instead? (Icon/SMI)|
—Mike Zarren, assistant executive director of basketball operations, Boston Celtics
That might scare a lot of would-be employees of basketball teams, but it got a very warm reception at MIT. The growth of basketball analysis has been completely unlike baseball analysis. In baseball, it took decades for front offices to realize the value of analytics. In basketball, teams are snapping up analysts before any of the good stuff gets out to the public. That's great for teams, but bad for statheads.
Zarren, Morey, and Denver Nuggets director of quantitative analysis Dean Oliver were all extremely circumspect in discussing their teams' use of analytics. ESPN columnist John Hollinger was surprised that teams didn't pool their resources, but none of the team employees seemed very receptive to that idea. Morey wouldn't even reveal how many analysts he had on his staff.
The most talkative of the panelists was Dallas Mavericks owner Mark Cuban. That may not surprise ... well, anybody, but Cuban didn't open his mouth just to talk. Even when debating Hollinger on the merits of the now-infamous Devin Harris for Jason Kidd trade, Cuban was reasoned, analytical, and quantitative; he used terms like "assigning weighted probabilities" and "portfolio management." He might be North America's first stathead franchise owner.
I really hope that the fraternity of MLB owners lets this guy buy a team one day. It's clear that he loves to win and that he'll look at novel ways to help his team do that. Among his (many) comments:
- The least valuable data in basketball is the boxscore data. The Mavericks have to track and capture their own data. I'll note that this is in contrast to baseball, where boxscore data from a century ago can be translated to a variety of advanced metrics.
- The Wages of Wins has the dumbest data he's ever heard.
- The NBA needs video experts who can automatically read, interpret, and capture elements.
- The Mavericks use a +/- system that is extremely similar to Tom Tango's WOWY analysis.
- Basketball teams are most profitable when they're rebuilding, and the marginal value of a win in basketball is about half a million dollars.
- There are thirteen guys on the court at any time, and three of them have 80 percent of the influence. The audience, familiar with his frequent criticism of officiating, roared.
The highlight of the discussion was when Cuban mentioned that they don't share any of their information with other clubs. Without missing a beat, Morey said that Cuban was always willing to share his referee ratings. The crowd loved it.
Until next year
My first year covering the conference, David Pinto attended on a press pass. This year, he was a panelist. It speaks volumes about the growing influence of analytics and the power of the online sabermetric community. Teams across all sports have been getting smarter about their decision-making. Events like the Sloan Sports Conference make sure that the trend will continue.
Sal Baxamusa is a graduate student in chemical engineering. He can be reached here.