Measuring Rangeby David Gassko
September 28, 2005
Let me tell you something: I'm a fan of UZR. I love it, I love it, I love it. So when Mitchel Lichtman (also known as MGL—he's the man who publishes UZR) decided to stop making it public last season, I was crushed. UZR was really by far the least flawed, best thought-out fielding system out there. It was better than Zone Rating because it incorporated all balls in play. It was better than David Pinto's PMR because it had better park factor applications and did not include putouts for infielders. It was better than Clay Davenport's DFTs because it used much more detailed data.
Now to be honest, I haven't listed all of its advantages over these systems. I have, however, listed those that I think can be matched by a fielding system that uses publicly available statistics. A system that can be reproduced free of charge by anyone crazy enough to waste his or her time on it. A system that would be validated by other play-by-play (PBP) systems, while still beating them in terms of accuracy. Such a system will be called BBARF, Batted Ball Adjusted Range Factor.
Why BBARF? Because I'm tired of new acronyms permeating baseball lingo. Actually I just can't come up with a name for my system. (I seriously spent about 10 hours the past week trying to come up with one.) Any suggestions are not only welcome but begged for. I'll be using this system to measure 2005 fielders next week so I need a better name for it. So write in, please.
Anyways, let me try to explain this system in as few paragraphs as possible. First, I split up a team's Balls In Play (BIP) based on the number of ground balls, fly balls and line drives they allowed. This allows me to estimate how many outs should have been made by infielders and how many outs should have been made by outfielders. For infielders, only grounders count, while for outfielders, I use outfield line drives and fly balls less home runs.
Next, I find a team's BIP against left-handed batters and right-handed batters. Using "Position Rates" published by Charlie Saeger, I estimate how many balls in play were hit to each fielder.
These are the two biggest advantages of my system over any other non-PBP system. Because I use batted ball data and because I know how balls were put into play by left-handed batters and how many were put into play by right-handed batters, I can come up with a razor sharp estimate of each fielder's "chances." Therefore, the greatest disadvantage a non-PBP system has in comparison to a PBP system is minimized.
The final steps are easy. Let me explain them through an example:
In 2004, Yankees pitchers allowed 2,027 ground balls. Based on the amount of balls put into play by left-handed and right-handed batters against the Yankees, I expected Yankee shortstops to make 450 assists. Derek Jeter played 93% of defensive innings played by the Yankees, so I expected him to have 419 assists. Jeter actually had 392 assists, which puts him at -27 assists above average. Converted to runs, he's -16.
This exercise is repeated for every player at positions four through nine (meaning I don't look at first basemen, pitchers or catchers). For outfielders, of course, I use putouts instead of assists.
Okay, so I've described my method, but the question is: what is the advantage of my system over other fielding measures? Let's get back to that first paragraph. My system has every one of those advantages that I listed for UZR. It has its disadvantages as well; actually, there's only one and that is that I do not have PBP data. Really accurate defensive systems must have PBP data at their disposal; however the only PBP system that is currently publicly available is PMR.
I don't want to trash PMR, as with a few improvements that have been suggested in the past, it would actually be as good or better than UZR. But as long as the system applies park factors the way it does, includes putouts for infielders and (this is in my opinion, a lesser problem) does not include distance travelled by a ball, PMR can be and needs to be augmented by another metric. And of all metrics not named PMR or UZR, mine is the best one out there.
A clumsy segue. Because the season isn't over yet, I'm not posting 2005 ratings yet. I do have 2004 numbers, and at the end of this article, I have attached a spreadsheet with ratings for all players with at least 450 defensive innings played in 2004. But let's look at 2004 leaders and trailers at each position.
Last First RAA PMR Hudson Orlando 27.1 23.1 Cora Alex 21.1 -0.3 Rivas Luis 20.1 11.2 GraffaninTony 13.2 9.3 Polanco Placido 12.4 13.1 Bellhorn Mark -11 -24 Ginter Keith -11 3.1 Cairo Miguel -12 -22 Hairston Scott A -15 -9.8 Boone Bret -18 -1.1
PMR and my system (which is labeled RAA, runs above average) agree on seven of the 10 second basemen, with Cora, Ginter and Boone being the big exceptions. 2004 UZRs are available for both Cora and Boone, and they seem to agree with Pinto's system. With Ginter, DFT agree with me while ZR puts him somewhere in between.
Last First RAA PMR Tejada Miguel 25.3 -9.1 Valentin Jose 25.3 1 Wilson Jack 23.8 -1.1 Reese Pokey 17.6 10.2 Lugo Julio 17.5 17.9 Clayton Royce -19 -8.9 Rollins Jimmy -22 14.3 Renteria Edgar -23 -3.2 Eckstein David -24 -19 Young Michael -28 -10
These two systems really disagree at shortstop; the ratings for Tejada, Valentin, Wilson, Rollins, Renteria and Young really differ. UZR has Tejada and Valentin as +20s in 2004, Young as -32, and Renteria at +12. So three wins for my system, and half-a-win for PMR with Renteria. Ratings for Wilson and Rollins were not made available by MGL, but ZR loves both of them. DFT agrees with me, but the problem is that we're using too-similar data. So I think this can be marked as a split between PMR and my system. Overall, my SS ratings seem to be closer to the "truth."
Last First RAA PMR Beltre Adrian 31.5 24.4 Bell David 27.2 1 Rolen Scott 26.1 23.4 Lamb Mike 9.59 2.7 Spiezio Scott 9.53 -2.6 Crede Joe -11 -20 Ensberg Morgan -11 -3.7 Ramirez Aramis -14 -9.3 Blalock Hank -16 -5 Hinske Eric -20 -27
These two systems are almost in perfect agreement at third base (which, to be honest, was a surprise as I expected the greatest disagreements to be here), with the exceptions being Bell and Spezio. UZR put Bell at +15, so score one for my system right there. (I know that +15 is actually right between our two ratings, but I am counting this as a win for my system because MGL says Bell is one of the top third basemen in the game, as I do, while Pinto is saying that Bell is basically average.) Zone rating sees Spezio as somewhat above average while DFT's love him. Basically, I'd call this one a split. Again, my system does better than PMR!
Last First RAA PMR Crawford Carl 26 24.9 Conine Jeff 20.5 -2.3 Bigbie Larry 20.4 -0.6 Bay Jay 12.4 2.4 Thomas Charles 11.8 12.8 Jenkins Geoff -15 -6.1 Berkman Lance -16 -6 Stewart Shannon -21 -12 Ramirez Manny -23 -2.2 Dunn Adam -32 -9.4
Here, the big disagreements are in regards to Conine, Bigbie, Ramirez and Dunn. UZR definitely agrees with my system on Bigbie (+12) and Ramirez (-26), is between my rating and PMR on Dunn (-20) and is really strange in regards to Conine. On one hand, he is the fourth-best left fielder in the National League according to UZR; on the other hand he is a -2. In the interest of fairness, this will be measured as a win for PMR since it gives the exact same result, but I couldn't help but bring to this to your attention.
Last First RAA PMR Payton Jay 39.3 13.5 Cameron Mike 15.3 5.6 Jones Andruw 13.5 17.9 Kotsay Mark 10.6 10.6 Redman Tike 10.3 6.6 Crisp Coco -20 0.62 Pierre Juan -20 -2 Byrd Marlon -20 -2.3 Burnitz Jeromy -20 -6.1 Williams Bernie -22 -25
Alright, for those of you who are fans of bif disagreements, there are quite a few here: Payton (though both systems love him), Crisp, Pierre, Byrd and Burnitz. Burnitz (-22) and Payton (+34) definitely are closer to their UZR ratings under my system. Unfortunately, ratings for Crisp, Pierre and Byrd were not published. Zone Rating agrees with my system on Pierre and is somewhere between me and PMR on Byrd, but Crisp's PMR rating is definitely "better" than mine. Still, my system does a little better at yet another position.
Last First RAA PMR Clark Brady 24 5.8 Jones Jacque 22.5 8.6 Giles Brian 13.4 -5.2 Bautista Danny 12.6 -0.4 Tucker Michael 11.3 -2.3 Rivera Juan -11 -7.1 Burnitz Jeromy -11 -8.1 Cabrera Miguel -16 -9.1 Cruz Jose -16 4.2 Abreu Bobby -23 -16
Okay, so there are even more disagreements here in right field. This is surprising as I expected Pinto's inclusion of putouts in infield numbers to have a greater impact on differences between the two systems and thus for infielder ratings to clash more than outfielder ratings. Clearly, and this is the first time in history that this has ever happened, I was wrong.
The two systems disagree on all the fielders I like except for Jones (who is one of the top fielders in the game with both systems, even if the runs saved they assign him differ much) and on Cruz as well. Cruz is one of the worst right fielders in the game according to UZR so score one for me there. But what about the rest? Unfortunately, MGL never provided UZRs for NL right fielders, so there is little I can do but rely on zone ratings to make my conclusions here. Zone rating sees Giles as the best right fielder in the game, so score one for me there. Zone rating things Bautista and Clark are average, so PMR is probably closer to the truth with those two. And zone rating thinks that Michael Tucker is actually the worst right fielder in the game, so there's another score for PMR.
The reason I provided this long comparison is three-fold: 1) Your enjoyment (or waste of time, whichever); 2) To show just how well my system matches up with PBP systems; and 3) to show how much my system can help improve an imperfect PBP measure like PMR. (And I swear this is no diss of David Pinto, whose work I love and enjoy greatly.) Did you notice how many times a player's UZR or ZR came halfway between what I said and what PMR said? Or how many times my system was closer to the "truth" than PMR? Well, if we combine the two, we can end up with smooth and deadly accurate ratings.
But even just using my system, you can have very accurate defensive ratings right at your fingertips. Literally. I'm attaching a spreadsheet with ratings for all players with at least 450 defensive innings played in 2004, so you can go crazy with them. I'll do the same next week with 2005 ratings. I just ask that you use them responsibly. (Don't try to use them to justify the Red Sox signing Alex Cora, for example. Nothing can justify that).
References and Resources
2004 UZR Ratings
David Gassko is a former consultant to a major league team. He welcomes comments via e-mail.