Educating Busterby Dave Studeman
November 09, 2006
Buster Olney talked about fielding awards and statistics the other day in his blog (subscription required). I enjoy Olney's blog a lot, and his description of how coaches typically choose Gold Glove awards reinforced what I sort of assumed. Olney says that many coaches, particularly the third base coaches, take the awards seriously and truly have great insight. On the other hand, here's how Olney described how other coaches make their decisions:
Just as often, however, this is kind of how the Gold Glove voting played out, in a roomful of coaches (I'm keeping names out of it, to protect the innocent and guilty):
Coach 1: "First base … hmmm … who are the best guys at first base?"
Coach 2: "That guy for the team we just played is pretty good."
Coach 1: "Yeah, you're right. Can't throw worth a damn, but he can pick it."
Coach 2: "What about [Candidate X]?"
Multiple coaches, all at once: "Yeah, he's a great choice."
And then, many times, the coaches on the same team would all vote for the same guy, in a block vote, without considering other players.
Interesting and enlightening. I really appreciate Buster Olney sharing that insight with us. But Olney then said something I really disagree with: But pure statistical analysis is almost as flawed.
Now, I certainly don't think fielding stats are perfect, but I do think they're a lot better than the current Gold Glove award process. Unfortunately, Olney's criticism of advanced fielding stats is truly misinformed. I was a bit disheartened by this, because Olney is a pretty smart guy, a fine writer, and he obviously follows baseball closely. If he doesn't understand fielding stats, then I assume most baseball writers and fans don't understand them. That's too bad; there is a lot we all can learn from today's fielding stats and they shouldn't be dismissed. Consider this article my feeble attempt to improve the situation.
First of all, Olney attacks fielding stats by spending three paragraphs debunking Range Factor. That's like saying personal computers are no good because they run on DOS. Range Factor was eclipsed as a cutting-edge fielding stat by Zone Rating almost twenty years ago. In fact, we recently had an internal debate at THT about whether we should include Range Factor in our THT Annual stats. We decided to go ahead because Range Factor is a useful rate stat; but no one argued that it should be used to truly evaluate fielders.
Olney then says:
Other statistical measures depend on an evaluator sitting in the press box effectively gauging whether a fielder might have/should have reached a ball, and there are many problems with this system, as well.
I don't know what "other statistical measures" Olney is referring to, but he isn't talking about the most widely used advanced fielding stats, like Mitchel Lichtman's Ultimate Zone Rating, John Dewan's Plus/Minus system or David Pinto's PMR. These measures don't rely on someone in a press box subjectively determining whether a ball was fieldable or not. They are much more objective and precise than that.
Olney received some e-mail complaining about his characterizations and here's what he said in response:
How do you determine how hard a ball was hit? Is gauging a hard-hit ball or really hard-hit ball or medium hard not a subjective measure? I'd bet that if you were to ask 10 different stringers to evaluate the same game, there would be lots of differences in how they gauge such information as how hard a ball was hit, and even where a ball was hit or whether a ball was a line drive or fly ball, etc.
What an amazing answer. Instead of admitting he had described the process incorrectly, Olney defends himself by changing the terms of the discussion. That might qualify Olney to run for president, but I think he would flunk debate class. And he still doesn't get it right; stringers don't classify batted balls as "medium hard" or "really hard-hit."
So let me attempt my own, vastly superficial but essentially correct, description of how today's most advanced fielding stats are collected:
- Stringers log baseball games. In some cases, stringers are at ballparks, but in many cases games are watched on video, allowing the stringer to replay games and make sure they log each play correctly.
- When a ball is hit, the stringer (by the way, stringer is an old newspaper term for a freelance journalist; I don't know why it's used for baseball fans who like to track ballgames) records it as a certain type: ground ball, fly ball, line drive or bunt. In 2006, Baseball Info Solutions added another category called "fliner" (batted balls that are in between fly balls and line drives and hard to categorize). This is certainly the most subjective part of the process.
- Stringers also track where the ball landed. Different companies have different ways of tracking this info. I believe STATS Inc. uses zones, while BIS uses coordinates based on charts they have of every ballpark. This isn't really subjective information, though you can certainly debate whether one approach is better than another.
- Finally, they classify how hard every batted ball was hit as soft, medium or hard. That's it; not "medium-hard" or "really hard-hit." Is there some subjectivity here as well? Of course. But it would be nice if Olney described the process correctly.
Zone Rating, UZR, PMR and Dewan's system all work essentially the same way: they compare how well each fielder fielded balls hit in specific areas compared to the average fielder at his position. This means that they are essentially corrected for the tendencies of the pitching staff. And while parts of the process are subjective, they aren't subjective at the point at which a ball is labeled "fieldable" or not. This is a very important distinction.
Yes, there are problems with this data. For instance, if a team puts on a shift against Carlos Delgado, with the shortstop positioned at second base, and Delgado hits the ball exactly where the shortstop would normally be playing, the result won't look good for that shortstop (through no fault of his own). More subtle positioning differences can also have an impact. John Dewan acknowledged that fielding metrics aren't perfect in the Fielding Bible and organized his own Fielding Bible awards (published in this year's Bill James Handbook, along with the plus/minus leaders at every position), based on ballots submitted by ten experts.
But if you ask me to choose either the Gold Glove awards or advanced fielding stats to select the best fielders in the majors, it's an easy choice. The stats.
The stats allow you to do some amazing things. For instance, look at Derek Jeter's performance on balls hit to specific zones (or vectors) in 2004:
David Pinto created these graphs from BIS's data over a year ago, and they are still the best fielding tools I have ever seen. Do you see how Jeter is worse than the average shortstop on balls in the third base hole? The blue lines on the graph represent the "standard" range of outcomes—and Jeter is worse than the lower part of the range in the third base hole.
Is the data subjective? Only a little, because this graph is based only on ground balls. So does Jeter deserve a Gold Glove? Some say he has improved since 2004, but the BIS stats don't show it. I say not only does he not deserve the Gold Glove, he's one of the worst fielding shortstops in the majors.
And the real shame is that Adam Everett keeps getting snubbed by the Gold Gloves. Let me say this unequivocally: Adam Everett is the Ozzie Smith of our time. According to the Handbook, Everett made 43 more plays than the average shortstop last year; the Gold Glove winner, Omar Vizquel, made less than seven more. That's a difference of almost 40 plays.
Here's what Everett's graph looked like in 2004:
Due to the way the graphs are drawn, it may be hard to pick out just how good Everett is. The dotted red line helps—he was particularly good moving to his right last year, though he was outstanding covering the hole this year. Everett just gets to everything and he has a rocket arm. Let's give the guy the credit he deserves, and let's not let a simple misunderstanding of fielding stats get in the way.
References and Resources
I'm no fielding statistics expert. If you'd like more detailed information about fielding stats, I'd suggest you read Chris Dial's blog, not only for Chris's posts but for the many fascinating comments that follow each post. I first read the comparison between Everett and Ozzie from Chris, and the description seems dead on to me.
Chris posted his own fielding awards for the National League and the American League.
You can find this year's Gold Glove winners on the MLB Awards page.
Dave was called a "national treasure" by Rob Neyer. Seriously. Comments about this article can be sent to him through the miracle of e-mail.