Ordinal comparisons of wOBA and OPSby Shane Sanders and Adam Winn
October 18, 2012
As we observed in a previous article, the OPS measure tends to greatly bias fundamental offensive events relative to wOBA. Compared to wOBA, it was found that OPS inflates the marginal value of singles and extra base hits relative to walks. These biases likely provides a fundamental explanation as to why OPS fares worse than wOBA in explaining variation in runs scored.
Given these biases, one might consider to what degree choice of measure influences ordinal player comparisons. Could a major league player have a top ten OPS value but a wOBA value that is outside the top fifty? Indeed, it is possible that individual player ratings are affected more than runs scored regressions by fundamental biases in the OPS measure. Below, we consider the level of consistency between wOBA player rankings and OPS player rankings. Each ranking set reflects the relative performances of 305 major league players during the 2011 season (i.e., all players appearing in at least 81 games).
We first ranked each player in terms of wOBA. The sampled player with the highest wOBA, Mike Napoli, was ranked #1, and the sampled player with the lowest wOBA, Drew Butera, was ranked #305. We then produced separate rankings for OPS. The sampled player with the highest OPS, Jose Buatista, was ranked #1, and the sampled player with the lowest OPS, Reid Brignac, was ranked #305.
Using set computational programming, we compared all player pairs within the set of 305. There are ((305*304)/2) or 46,360 total player pairs. In our comparison, two possible outcomes exist. If the two measures are consistent, we would expect (for each pair) that one player outranked the other player in both measures. If the two measures are inconsistent, we would expect that one player in a pair had a higher wOBA, and the other player in the pair had a higher OPS.
However, in 1,968 pairs (4.2 percent of all pairs), the two measures diverge in choosing a superior offensive player (i.e., one player had a higher wOBA while the other player had a higher OPS). If a team compares 16 randomly selected players in a pairwise manner, we expect 5 of the 120 pairwise comparisons to differ depending on the measure used.
Based on the two sets of player rankings, we developed a wOBA percentile ranking and an OPS percentile ranking for each player. These percentile rankings are similar to one’s percentile ranking on the SAT exam. A 90th percentile ranking, for example, indicates that one performed better than 90 percent of peers taking the exam.
For each player, then, his wOBA percentile ranking was subtracted from his OPS percentile ranking to establish a measure of ranking divergence. For example, assume that a player has a wOBA percentile ranking of 90 and an OPS percentile ranking of 95. This player’s OPS percentile - wOBA percentile would be 5, indicating that he is ranked five percentile points higher in the OPS ranking. The following table reports summary statistics for this divergence measure.
Sample Size 305 Mean 0 Std. Dev. 4.49 Min -23.28 Max 12.46The standard deviation of the variable (OPS percentile - wOBA percentile) is 4.49. In other words, a player’s ranking changes by 4.49 percentile points, on average, when going from OPS to wOBA. For a significant set of players, the measures tell very different stories. For example, Jason Bourgeois’ wOBA ranking is a full 23.28 percentile points higher than his OPS ranking. Similarly, Tony Campana’s wOBA ranking is 20.33 percentile points higher than his OPS ranking. Tables 2a and 2b report the players upon whom the two measures least agree.
Players most favored by OPS
(Most positive in OPS-wOBA)
Rod Barajas 0.125 J.P. Arencibia 0.121 Adam Lind 0.098 Ryan Raburn 0.095 Jorge Posada 0.082 Martin Prado 0.082 Wilson Ramos 0.079 A.J. Pierzynsk 0.079 Mark Trumbo 0.079 Raul Ibanez 0.079 Yorvit Torrealba 0.079
Players least favored by OPS
(Least positive in OPS-wOBA)
Jason Bourgeois -0.233 Tony Campana -0.203 Brett Gardner -0.148 Coco Crisp -0.131 Drew Stubbs -0.121 Elvis Andrus -0.121 Jamey Carroll -0.108 Jordan Schafer -0.102 Michael Bourn -0.102 Emilio Bonifacio -0.098The first table represents players who have the highest values of OPS percentile - wOBA percentile. In other words, they are the most significant beneficiaries of biases within the OPS measure. We can examine the 2011 batting statistics of Rob Barajas, J.P. Arencibia, Adam Lind, Ryan Raburn, Jorge Posada, and Martin Prado to find if players in Table 2a represent a “type.”
First, note that the three of these six players are catchers, and the other three are outfielders a significant portion of the time. We also note that none of the six players got on base at an average rate in 2011. In fact, the six players ranged between “awful” and “below average” in terms of on-base percentage according to fangraphs.com. Of the 305 players sampled in 2011, Arencibia is 32nd in terms of on base percentage, Barajas is 37th, Lind is 53rd, Raburn is 57th, Prado is 69th and Posada is 114th. Thus, five of the six players were below the 23rd percentile in terms of on-base percentage, and Posada was below the 38th percentile.
Moreover, each player was somewhere between “below average” and “awful” in terms of BB%. Five of the six players were somewhere in this same range for K%, with Posada again being the exception. We can also look at the ratio TB/H to get a sense of the distribution of hits generated by the players. In the overall sample, this ratio has a mean of 1.56 and a median of 1.54. In other words, the average player earns about 1.54 bases for a typical hit.
Five of the six players in the table were well above the mean and median for this ratio, with Prado being the exception. This is an indication that these players earned a higher proportion of extra base hits than the typical player. In support of the previous article, this finding provides symptomatic evidence that OPS overvalues extra base hits relative to walks and singles. Thus, we expect the OPS measure to be most upward biased for a power hitter who has a low on-base percentage, doesn’t walk a lot, doesn’t generate many singles, and strikes out at an alarming rate.
Now let's consider trends that may emerge from an evaluation of players in the second table, which comprises players for whom the OPS measure is most downward biased. We consider the first seven players in the table. Jason Bourgeois, Tony Campana, Brett Gardner, Coco Crisp, Drew Stubbs, and Elvis Andrus are in the table primarily because they were all highly successful base stealers in 2011. In addition, Jamey Carroll was a successful base stealer but did not have the volume of steals that the others did.
If a player can successfully steal more than two bases in every three attempts, then the wOBA measure estimates that there is value added in the activity. The OPS measure, on the other hand, does not credit a player for a stolen base nor debit a player when he is caught attempting a steal.
Moreover, all seven players had a below average TB/H ratio (i.e., none are power hitters).
For a few of these players, additional factors appear to have contributed to the downward bias of their OPS measure. Gardner, Andrus, and Carroll all possessed high on-base percentages and were “above average” to “great” in terms of K%. Moreover, two of these three players were well above average in terms of BB%. Andrus was nearly average in terms of BB% but offset this with an extremely low K% (11.1%).
We can conclude that players in the second table are typically prolific and efficient in base stealing and do not hit for power. Also, there is a sub-group of players in the table who are able to get on base at a high rate. In general, this further confirms the result that the OPS measure tends to overvalue extra base hits relative to walks and singles.
It has been observed many times that the OPS measure provides a rough estimate of a player’s offensive productivity. It appears, however, that this estimate can become unusably rough around the edges.
References and Resources