The Hardball Times -- JC Bradbury

PrOPS: 2005 and Beyond

JC Bradbury — 2005-12-14T04:06:15+00:00

Last spring, I developed a new offensive metric I called PrOPS, which is short for predicted OPS. Obviously, it's a method for predicting OPS, which is a good shortcut method for measuring a hitter's productivity. But I didn't develop the statistic to be just another option to choose from when judging a player's offensive value. After all, I believe Bill James was largely correct in saying that “the world needs another offensive rating system like Custer needs more Indians (or, for that matter, like the Indians need another Custer).”

Instead, I intended to find some information that is lost in the noise of common baseball statistics. My goal was to use some of the new batted-ball data published by The Hardball Times to find information that outcome-based metrics (i.e., batting average, on-base percentage and slugging percentage) miss. In particular, I was curious about the problem of luck. When a cylindrical bat hits a round ball lots of funny things can happen to influence the outcome.

Just because a hitter does some things that normally lead to good/bad outcomes doesn't mean those outcomes will always happen. Good process can result in bad outcomes, and bad process can result in good outcomes. If these outcomes don't cancel out, but pile up in one direction, outcome-based metrics can distort a player's true performance. PrOPS is a tool for extracting the noise from the statistics that we normally find to be quite reliable predictors of run production.

The method for generating PrOPS involves using batted-ball data, along with a few other important factors, to predict the offensive output of players using linear regression. The projections are thus based on the way players hit the ball and not actual outcomes. PrOPS credits players who hit the ball similarly with outcomes typical of all players of the comparable hitting profiles. This method strips out some of the luck that pollutes outcome-based hitting statistics, because no player receives direct credit for the actual outcomes at the plate.

One of the uses of PrOPS is that it can spot fluke seasons. For instance, it's often unclear if players with abnormally good and bad years are experiencing improvement, decline or just runs in random bounces. When compared to outcome metrics, deviations from predictions indicate that a player has been lucky or unlucky. In The Hardball Times Baseball Annual 2006 I test PrOPS's usefulness in identifying fluke performances using four seasons of data (2002-2005).

It turns out that PrOPS does a decent job of spotting lucky and unlucky performances. One advantage of working with the data from many seasons—the first version was based on 2004 data only—was that I was able to refine my approach to generate more accurate estimates of the impact of batted-ball types on hit outcomes, which greatly improved the PrOPS metrics. It turns out that some of the corrections that I needed to make in the previous version—speed, for example—were no longer necessary.

Over the summer, I posted weekly updates of PrOPS until I just got out of the habit of doing it. But when the free agent and trade markets began to heat up, I found it useful to look at the newer 2005 PrOPS numbers for guidance. While hanging out over on Mac Thomason's Braves Journal—the unofficial online home for Braves fans—I kept posting PrOPS lines of guys in whom the Braves might have some interest, because many of these players were coming off odd years.

For example, PrOPS said Julio Lugo didn't play as well as his 2005 numbers, and Edgar Renteria's down year in Boston was right on target with his 2004 in St. Louis. (Not that it makes losing Andy Marte any easier to take.) A few of my online friends found the numbers to be helpful as well, so I decided it was time to post the final 2005 PrOPS numbers for all players on THT for others to see. The statistics are listed separately for the AL and NL, and I have sorted them by team. If you want to see my analysis of these numbers, pick up a copy of the Annual.

But I didn't stop at just updating the old numbers. In addition, I decided to take the metric a little further. After all, if PrOPS has the power to distinguish between fluke and true performances, then I should be able to use it to predict future performances. Thus, I developed a PrOPS-based projection system. It's not fancy, and it doesn't attempt to predict anything more than the OPS for a player in the following season. I used two years of data to project OPS for all players with a minimum of 130 plate appearances in the previous season based on PrOPS, player ages and the home parks’ effects on hitters.

I also calculated 95% confidence intervals for the projections. For rookies or other players with fewer than 130 plate appearances in 2004, I generated projections using a single year of data. I have less confidence in these numbers, but they are better than no numbers at all. I did not generate projections for players with fewer than 130 plate appearances in 2005. (Sorry, no Bonds projection.) The projections assume the player plays for the same team as he did in 2005. The 2006 PrOPS Projections are listed alphabetically.

How accurate were the projections for past seasons? Using a method of comparison reported in the 2004 Baseball Prospectus for comparing OPS projections of competing projection systems, I tested the PrOPS projections on a similar sample of players—I could not replicate it exactly. The PrOPS projections performed about as well as the top systems. I wouldn't put much stock in this test in terms of stating any one system is superior to any other; however, I think it shows that the PrOPS projections are worth using along with other systems.

Please enjoy the 2005 PrOPS and 2006 PrOPS projections. If you are interested in further discussion of PrOPS, check out my article “Giving Players Their PrOPS: A Platonic Measure of Hitting" in The Hardball Times Baseball Annual 2006. I welcome comments and suggestions.

Click here to learn about THT's download subscriptions.

A PrOPS Tweak

JC Bradbury — 2005-08-03T12:09:15+00:00

Click here to learn about THT's download subscriptions.

Moneyball and Efficient Efficiency

JC Bradbury — 2005-07-26T04:06:15+00:00

Yes, this is another Moneyball article. But it's not a rehash of who's right, who's wrong, and let's just hold hands in the mushy middle. I'm as sick of those articles as everyone else. Sure, maybe we need more of this analysis, but I think we deserve a break right now. What I want to do is add a little tangent to the debate that has escalated since the publication of Moneyball.

One of the central lessons of Moneyball is this: to get the most output from your inputs in order to maximize the return on the dollars that your organization spends on running team. A GM must be efficient in running his organization. In economic terms, he's attempting to put all of his resources to their most highly valued uses. If the market overvalues a particular baseball talent–for example, saves–then a team should liquidate its assets in this area. If the market undervalues a talent–for example, OBP–you acquire it while it's cheap. It's all very simple in theory, but difficult in practice.

This is clearly something all GMs try to do, and we're told that Oakland A's GM Billy Beane excels at it. Whether it's hitting, pitching, fielding, aging, speed, or intelligence; the A's can value it. And once the values are in place, Beane is able to do the wheeling and dealing to make sure the club prospers from this knowledge. As a result, the A's win. Moneyball, the philosophy, involves properly allocating scarce resources, explaining the book's popularity among economists–and the A's play it better than anyone.

But, there is one aspect of in the book that often gets overlooked. It deals with efficiency, but a type of efficiency different from the type commonly attributed to the book. Efficiency is also a term statisticians use to judge what are known as estimators. Estimators are theoretical tools for predicting a numerical estimate from a sample of data. Rather than being a number, an estimator as a method of predicting an estimate; it's a function of the data.

There are lots of different methods we could chose predict estimates from a group of data. For example, let's say I was asked to predict the SAT score of a randomly selected student at a university. I would have many tools at my disposal. I could ask the next student who walked down the street what his score was, and go with that score. Or I could go with the mean or the median of the student population. These choices represent estimators that will generate a prediction.

Statisticians try to determine properties of competing estimators that will minimize the mistakes of estimates. To me, this sounds a lot like the "stat-heads versus scouts" debate that has raged since the book's publication. It's an argument over estimators, except the estimators are not necessarily purely known mathematical functions of the data. Yet, each camp makes predictions based about the same population of players based on these different estimators.

In Moneyball, Lewis pits traditional scouting methods of personal observation against performance scouting via statistics. In reality, no matter which way it leans on the traditional/performance scouting spectrum, no organization completely ignores the other method. Sabermetric clubs have plenty of guys with left-arm only tans from driving, just as the traditional clubs have pasty-white computer nerds stashed in their basements. So, the actual debate is over the slant or preference each organization has on the scouting spectrum. Lewis portrays the push toward sabermetric methodology to identify talent as reason for the A's success in winning on a tight budget.

It's tempting to say the method the A's employ—of which sabermetrics play a large role—is superior to all other methods, and that's why A's have been winning as of late. This upsets a lot of people. After all, haven't teams been successful without employing sabermetric methods? There is no arguing that Beane and DePodesta used a sabermetric mind-set to stay ahead of their competitors.

But, in looking at the success of baseball teams during Oakland's recent run of success, the A's clearly aren't the only team that's been winning, even in terms of their limited budget. Below is a list of teams ranked on total budgets as a percent above/below the league average payroll and the number of playoff appearances by team for the past five seasons.

Rank    Team    Payroll     Playoff     Rank    Team    Payroll     Playoff
1       FLO     -44.91%     1           16      BAL     -1.21%      0
2       KC      -41.92%     0           17      COL     -0.59%      0
3       MON     -40.74%     0           18      LAA     1.08%       2
4       MIN     -40.27%     3           19      HOU     2.78%       2
5       MIL     -40.27%     0           20      CHC     11.11%      1
6       PIT     -34.91%     0           21      SF      11.36%      3
7       TB      -34.53%     0           22      STL     16.04%      5
8       OAK     -32.61%     4           23      SEA     21.19%      2
9       SD      -28.81%     0           24      TEX     25.57%      0
10      CIN     -24.57%     0           25      ARI     26.08%      2
11      DET     -17.10%     0           26      ATL     38.22%      5
12      CH      -15.36%     1           27      BOS     49.39%      2
13      PHI     -5.05%      0           28      NYM     50.39%      1
14      TOR     -4.93%      0           29      LAD     57.84%      1
15      CLE     -1.66%      1           30      NYY     94.13%      5

While the big-budget teams dominate the playoff appearances, the A's are not the only members of the winning paupers club. The most noted comparable to Oakland is the Twins, with three post-season appearances over this span. Also, the Marlins and the White Sox have posted some success on smaller-than-average budgets. None of these teams are considered Moneyball™ clubs. We think of these franchises as successes of traditional scouting with good overall management that can also win on a tight budget. If playing Moneyball is just winning on a tight budget, Lewis would have added some chapters on these other clubs. To say that Moneyball is just finding bargains is really a cop-out.

But the message of Moneyball isn't that statistical analysis is superior to all other methods in projecting major league talent, either. It's a method that's successful because the teams that have employed it have been using it with great success on a particular subset of talent that includes: major leaguers, minor leaguers, and college players. In particular, the Moneyball™ GMs have been criticized and praised heavily for their draft strategy of focusing on college players. I believe the focus on college players has to do with the usefulness of the estimators these clubs have chosen to use to evaluate talent. Playing Moneyball is not just about finding bargains, but it's also a method (a set of estimators) for finding bargains. It just so happens that the statistics-driven method of evaluating talent is more efficient, in a statistical sense, than old methods in evaluating college players.

Estimators are judged on two properties: bias and consistency. First, we want our estimator to be unbiased–that is not consistently above or below the true value we're estimating. Second, we want to minimize the size of any predicting mistakes; a quality known as consistency. When we choose from several unbiased estimators to predict a true outcome from population, we want the one that minimizes the variance of the errors of the prediction. Fewer and smaller mistakes are preferred to more and larger mistakes, right? In the language of statistics, the estimator with the greatest consistency, or smallest variance, is said to be the most efficient of all estimators. In this sense, being efficient means making fewer and smaller mistakes compared to other available estimation methods.

The figure below shows a the variance of estimates of a pitching prospect's true talent using two hypothetical methods, A and B. Both methods estimate the player's talent to be the same. The difference is in the consistency. If we can predict a pitcher's true talent level with more accuracy (method A), then this will result in fewer mistakes in evaluating talent than with method B. These methods are estimators, and A is more efficient than B. This, in turn, increases the economic efficiency of the organization, because it now needs fewer resources to devote to scouting talent.

The A's focus on college players not because of a bias of stat-heads in thinking college players are superior to high school players; but because they are more predictable based on the statistical tools the A's favor. A technological innovation in performance scouting, such as DIPS, can increase the efficiency in evaluating talent. DIPS ERA can be used to better predict a college player, but maybe not a high-school player.

As a result, the A's are going to have a higher confidence when drafting from this talent pool. And if a technology can be employed in one area but not another, it's no surprise that the A's would concentrate on a talent pool where this new technology is useful. Just as the cotton gin caused southern farmers to switch to cotton farming, where the technological innovation could be used, so too did the A's turn to the college talent pool where it's inventions were useful. This reason for this choice is well-documented in Lewis's book.

From Paul's point of view, that was the great thing about college players: they had meaningful stats. They played a lot more games, against stiffer competition, than high school players. The sample size of their relevant statistics was larger, and therefore a more accurate reflection of some underlying reality.

And, as it just so happens, the stat-head methodology happens to be a cheap operation to run if you do it right. Rather than scattering the country with scouts to report back on thousands of personal observations (from which personal impressions could be quantified), why not target those who exhibit qualities of successful ballplayers? You can afford to limit the talent pool you draw from the more efficient you are at evaluating it, because you will have more confidence in the predictions you make.

Other organizations that have been successful in evaluating talent through traditional methods have their own innovations for evaluating talent that have caused them to be successful on a small budget, too. Moneyball speaks to the success of the implementation of one methodology; a method that happened to come to baseball from the outside, which makes it interesting. And just as traditional scouting organizations would be wise to adopt innovations from sabermetric organizations, so too can stat-savvy clubs learn from the innovations in traditional scouting–innovations are innovations. And clubs that wish to win will shift their resources to take advantage of these new methods.

I guess my point is, I don't understand why Moneyball upsets so many people. And as much as I hate to admit it, maybe sociology or psychology can explain the reason for the backlash. The book is about one method, comprised of many complicated parts, that one team has used to win baseball games. It's possible that the A's recent successes are a product of dumb luck, and it's really too early to say it's not with any certainty. But, I don't believe it's luck. I find sabermetric methods to be powerful methods elicit many powerful truths about the game that the conventional wisdom seems to have missed. For this reason, I expect that organizations that employ these methods for evaluating talent will benefit from their use.

Click here to learn about THT's download subscriptions.

The PrOPS-Star Team

JC Bradbury — 2005-07-18T04:05:15+00:00

I'm writing my All-Star break reflection piece a little later than most thanks to the timing of my summer travels, but that's not really important. My purpose for looking back at the first-half of 2005 is to check on PrOPS (see Introducing PrOPS). I designed PrOPS back in May on a whim—I think there was some beer involved. My goal was to design a metric to estimate how well a hitter should be hitting based on the way he has hit the ball during the season. The outcome stats we commonly use to judge hitters (i.e., AVG, OBP, SLG, and OPS) are polluted by luck, as some hard-hit balls become outs and some dinkers become hits.

The PrOPS formula rewards batters for hitting the ball hard, and punishes them for weakly hit balls. I wanted a performance stat as opposed to an outcome stat, and I thought that the hit-type data (groundball/flyball ratio, line-drive percent, etc.) provided on The Hardball Times Stats Page might provide the information needed to generate such a metric; that's how PrOPS was born.

PrOPS is in no way perfect--it does not remove all luck from batting outcomes. PrOPS (derived from Predicted OPS) is to OPS what component ERA is to ERA. After many suggestions, I refined the metric a bit to control for speed and divided PrOPS into components (PrAVG, PrOBP, and PrSLG). Below I present some general information about PrOPS, and generate a list of the best and worst PrOPS performers for the first-half of 2005, along with my first half PrOPS All-Stars.

General Trends

Here are the general trends in PrOPS for players with more than 150 plate appearances by league.

League  PrOPS   OPS     PrOPS%  PrOPS+  PrAVG   PrOBP   PrSLG
AL      0.787   0.771   -2.56%  -0.017  0.283   0.344   0.443
NL      0.798   0.784   -2.50%  -0.014  0.282   0.351   0.447

Because The Hardball Times' hit-type data is new, I only had the 2004 data available to generate my projections. Therefore, the PrOPS metrics simply indicate what players with similar batting performance in 2004 did in 2004. For 2005, PrOPS consistently overshoots actual OPS, which isn't too surprising, as offense is down some this year. Nonetheless, the difference in predictions is small enough to make me happy.

On average, players seem to be doing a little worse in 2005 than they did in 2004, with the average player producing 2.5% less OPS than they did in 2004, based on hit-types. A player with a PrOPS+ (OPS - PrOPS) that is -0.015 OPS-points this year is performing about the same on batting outcomes as the rest of the league in terms of hit-types.

The Best and Worst PrOPS

There are no real surprises among the best and worst performances based on PrOPS (minimum 150 PAs).

Top-25 AL PrOPS

Rank    First               Last                Team    PrAVG   PrOBP   PrSLG   PrOPS   OPS
1       Manny               Ramirez             BOS     0.317   0.397   0.611   1.008   0.910
2       Brian               Roberts             BAL     0.334   0.402   0.567   0.969   1.007
3       Alex                Rodriguez           NYA     0.288   0.391   0.560   0.951   0.998
4       Mark                Teixeira            TEX     0.297   0.365   0.581   0.946   0.930
5       Miguel              Tejada              BAL     0.327   0.369   0.562   0.931   0.977
6       Vladimir            Guerrero            LAA     0.319   0.372   0.558   0.929   0.971
7       Tino                Martinez            NYA     0.284   0.373   0.550   0.923   0.780
8       Rafael              Palmeiro            BAL     0.312   0.379   0.534   0.913   0.800
9       Jason               Giambi              NYA     0.275   0.419   0.489   0.908   0.890
10      Jason               Varitek             BOS     0.307   0.370   0.535   0.906   0.892
11      Gary                Sheffield           NYA     0.284   0.390   0.513   0.903   0.920
12      Jermaine            Dye                 CHA     0.291   0.348   0.552   0.899   0.860
13      Paul                Konerko             CHA     0.278   0.372   0.527   0.898   0.828
14      Alfonso             Soriano             TEX     0.293   0.323   0.564   0.888   0.840
15      Kevin               Mench               TEX     0.285   0.348   0.524   0.873   0.906
16      Trot                Nixon               BOS     0.302   0.384   0.489   0.873   0.861
17      Derek               Jeter               NYA     0.317   0.403   0.466   0.869   0.844
18      Michael             Young               TEX     0.315   0.355   0.510   0.865   0.906
19      Dmitri              Young               DET     0.295   0.350   0.511   0.861   0.781
20      Melvin              Mora                BAL     0.291   0.345   0.509   0.854   0.861
21      Eric                Byrnes              OAK     0.298   0.363   0.490   0.853   0.810
22      Richie              Sexson              SEA     0.253   0.355   0.489   0.844   0.865
23      Hank                Blalock             TEX     0.284   0.342   0.491   0.833   0.825
24      Bobby               Crosby              OAK     0.307   0.362   0.471   0.833   0.925
25      Jhonny              Peralta             CLE     0.283   0.336   0.497   0.833   0.874

Top-25 NL PrOPS

Rank    First               Last                Team    PrAVG   PrOBP   PrSLG   PrOPS   OPS
1       Derrek              Lee                 CHN     0.315   0.399   0.653   1.052   1.186
2       Andruw              Jones               ATL     0.298   0.376   0.610   0.986   0.930
3       Albert              Pujols              STL     0.317   0.402   0.583   0.985   1.017
4       Adam                Dunn                CIN     0.260   0.400   0.570   0.970   0.935
5       Bobby               Abreu               PHI     0.305   0.427   0.541   0.968   0.955
6       Tony                Clark               ARI     0.311   0.339   0.607   0.947   0.983
7       Morgan              Ensberg             HOU     0.265   0.367   0.567   0.934   0.983
8       Cliff               Floyd               NYN     0.276   0.349   0.564   0.913   0.903
9       Hee Seop            Choi                LAN     0.297   0.371   0.539   0.910   0.777
10      Aramis              Ramirez             CHN     0.302   0.354   0.551   0.905   0.905
11      Reggie              Sanders             STL     0.275   0.337   0.568   0.905   0.897
12      Xavier              Nady                SD      0.291   0.350   0.550   0.901   0.839
13      Moises              Alou                SF      0.298   0.391   0.509   0.900   0.909
14      Preston             Wilson              COL     0.299   0.357   0.540   0.897   0.813
15      J.D.                Drew                LAN     0.270   0.400   0.497   0.897   0.931
16      Carlos              Lee                 MIL     0.280   0.347   0.546   0.893   0.864
17      Brian               Giles               SD      0.296   0.428   0.463   0.891   0.942
18      Milton              Bradley             LAN     0.310   0.355   0.535   0.891   0.857
19      Carlos              Delgado             FLA     0.279   0.376   0.512   0.888   0.922
20      Jose                Cruz                ARI     0.259   0.376   0.512   0.888   0.796
21      Clint               Barmes              COL     0.326   0.361   0.525   0.886   0.886
22      Jim                 Edmonds             STL     0.248   0.370   0.511   0.881   0.942
23      Ryan                Klesko              SD      0.280   0.381   0.498   0.879   0.846
24      Wily Mo             Pena                CIN     0.279   0.317   0.561   0.878   0.891
25      Julio               Franco              ATL     0.305   0.366   0.511   0.877   0.824

Bottom-25 AL PrOPS

Rank    First               Last                Team    PrAVE   PrOBP   PrSLG   PrOPS   OPS
1       Tony                Womack              NYA     0.266   0.302   0.329   0.631   0.542
2       Juan                Castro              MIN     0.267   0.290   0.373   0.663   0.620
3       Jeremy              Reed                SEA     0.267   0.333   0.343   0.676   0.681
4       Travis              Lee                 TB      0.255   0.331   0.345   0.676   0.656
5       John                Buck                KC      0.246   0.290   0.386   0.676   0.629
6       Omar                Infante             DET     0.250   0.298   0.392   0.690   0.667
7       Nook                Logan               DET     0.283   0.327   0.365   0.692   0.665
8       Ruben               Gotay               KC      0.250   0.315   0.378   0.693   0.681
9       Mark                Teahen              KC      0.281   0.329   0.364   0.694   0.658
10      Scott               Podsednik           CHA     0.282   0.360   0.338   0.697   0.713
11      Julio               Lugo                TB      0.275   0.336   0.361   0.698   0.711
12      Mark                Ellis               OAK     0.283   0.339   0.362   0.701   0.694
13      Terrence            Long                KC      0.284   0.323   0.379   0.702   0.670
14      Michael             Cuddyer             MIN     0.258   0.336   0.370   0.706   0.719
15      Juan                Uribe               CHA     0.268   0.304   0.404   0.708   0.655
16      Toby                Hall                TB      0.274   0.313   0.397   0.710   0.680
17      Orlando             Cabrera             LAA     0.280   0.329   0.388   0.716   0.648
18      Alexis              Rios                TOR     0.271   0.314   0.402   0.716   0.739
19      Eric                Hinske              TOR     0.243   0.324   0.398   0.721   0.712
20      Angel               Berroa              KC      0.285   0.320   0.401   0.721   0.669
21      Darin               Erstad              LAA     0.281   0.342   0.378   0.721   0.749
22      Nick                Punto               MIN     0.271   0.336   0.386   0.723   0.749
23      Alex S              Gonzalez            TB      0.269   0.328   0.399   0.727   0.704
24      Tadahito            Iguchi              CHA     0.277   0.338   0.391   0.728   0.747
25      Aubrey              Huff                TB      0.261   0.325   0.403   0.728   0.702

Bottom-25 NL PrOPS

Rank    Last                Team                Team    PrAVE   PrOBP   PrSLG   PrOPS   OPS
1       Chris               Burke               HOU     0.229   0.288   0.309   0.597   0.585
2       Royce               Clayton             ARI     0.255   0.307   0.333   0.639   0.615
3       Alex                Gonzalez            FLA     0.246   0.299   0.349   0.648   0.708
4       Cristian            Guzman              WAS     0.261   0.299   0.355   0.653   0.530
5       Jose                Reyes               NYN     0.261   0.288   0.365   0.653   0.650
6       J.J.                Hardy               MIL     0.248   0.342   0.318   0.660   0.560
7       Vinny               Castilla            WAS     0.236   0.315   0.354   0.669   0.727
8       Brian               Jordan              ATL     0.259   0.310   0.362   0.672   0.627
9       Jamey               Carroll             WAS     0.281   0.334   0.338   0.673   0.585
10      Mike                Lamb                HOU     0.263   0.308   0.371   0.679   0.618
11      Mike                Lowell              FLA     0.262   0.316   0.367   0.683   0.632
12      Willy               Taveras             HOU     0.274   0.313   0.370   0.683   0.705
13      Chris               Snyder              ARI     0.256   0.334   0.355   0.689   0.633
14      Junior              Spivey              MIL     0.234   0.306   0.386   0.692   0.682
15      Kazuo               Matsui              NYN     0.267   0.318   0.379   0.696   0.604
16      Brad                Ausmus              HOU     0.276   0.356   0.341   0.697   0.621
17      Raul                Mondesi             ATL     0.247   0.307   0.391   0.698   0.630
18      Brad                Wilkerson           WAS     0.234   0.341   0.359   0.701   0.809
19      David               Bell                PHI     0.273   0.322   0.379   0.702   0.662
20      Cesar               Izturis             LAN     0.298   0.343   0.362   0.705   0.660
21      Quinton             McCracken           ARI     0.274   0.342   0.366   0.708   0.545
22      Jack                Wilson              PIT     0.281   0.315   0.396   0.711   0.614
23      Carlos              Beltran             NYN     0.248   0.307   0.405   0.712   0.754
24      Sean                Casey               CIN     0.291   0.344   0.368   0.712   0.761
25      Damian              Miller              MIL     0.267   0.336   0.379   0.715   0.728

The Over/Under-Performers

This is where PrOPS becomes fun. These are the guys who may be performing better or worse than their outcome stats indicate. In terms of PrOPS+ (absolute difference) and PrOPS% (percent difference) under-performance is good, and over-performance is bad. If PrOPS is right and these players continue to hit the ball as they did in the first half, the under-performers should improve and the over-performers should decline.

Top-25 AL Under-Performers

Rank    First               Last                Team    PrOPS   OPS     PrOPS+  PrOPS%
1       Aaron               Boone               CLE     0.781   0.628   -0.153  -24%
2       Tino                Martinez            NYA     0.923   0.780   -0.143  -18%
3       Rafael              Palmeiro            BAL     0.913   0.800   -0.113  -14%
4       A.J.                Pierzynski          CHA     0.833   0.722   -0.111  -15%
5       Richard             Hidalgo             TEX     0.801   0.691   -0.110  -16%
6       Victor              Martinez            CLE     0.802   0.692   -0.110  -16%
7       Manny               Ramirez             BOS     1.008   0.910   -0.098  -11%
8       Tony                Womack              NYA     0.631   0.542   -0.088  -16%
9       Casey               Blake               CLE     0.782   0.697   -0.086  -12%
10      Steve               Finley              LAA     0.778   0.695   -0.083  -12%
11      Edgar               Renteria            BOS     0.793   0.714   -0.079  -11%
12      Dmitri              Young               DET     0.861   0.781   -0.079  -10%
13      Sammy               Sosa                BAL     0.766   0.689   -0.077  -11%
14      Mark                Bellhorn            BOS     0.767   0.693   -0.075  -11%
15      Mark                Kotsay              OAK     0.822   0.748   -0.074  -10%
16      Paul                Konerko             CHA     0.898   0.828   -0.070  -8%
17      Orlando             Cabrera             LAA     0.716   0.648   -0.068  -11%
18      Joe                 Crede               CHA     0.792   0.727   -0.066  -9%
19      Laynce              Nix                 TEX     0.735   0.670   -0.065  -10%
20      Russ                Adams               TOR     0.793   0.734   -0.059  -8%
21      Orlando             Hudson              TOR     0.778   0.722   -0.056  -8%
22      Jason               Kendall             OAK     0.745   0.690   -0.055  -8%
23      Juan                Uribe               CHA     0.708   0.655   -0.053  -8%
24      Angel               Berroa              KC      0.721   0.669   -0.052  -8%
25      Kevin               Millar              BOS     0.763   0.711   -0.052  -7%

Top-25 NL Under-Performers

Rank    First               Last                Team    PrOPS   OPS     PrOPS+  PrOPS%
1       Quinton             McCracken           ARI     0.708   0.545   -0.163  -30%
2       Juan                Pierre              FLA     0.803   0.666   -0.137  -21%
3       Hee Seop            Choi                LAN     0.910   0.777   -0.133  -17%
4       Cristian            Guzman              WAS     0.653   0.530   -0.123  -23%
5       J.D.                Closser             COL     0.766   0.643   -0.122  -19%
6       Mike                Lieberthal          PHI     0.787   0.683   -0.104  -15%
7       J.J.                Hardy               MIL     0.660   0.560   -0.100  -18%
8       Damion              Easley              FLA     0.864   0.764   -0.100  -13%
9       Sean                Burroughs           SD      0.734   0.635   -0.099  -16%
10      Jack                Wilson              PIT     0.711   0.614   -0.097  -16%
11      Austin              Kearns              CIN     0.795   0.700   -0.095  -14%
12      Yadier              Molina              STL     0.750   0.656   -0.094  -14%
13      Ryan                Langerhans          ATL     0.833   0.739   -0.094  -13%
14      Rafael              Furcal              ATL     0.797   0.703   -0.093  -13%
15      Kazuo               Matsui              NYN     0.696   0.604   -0.092  -15%
16      Jose                Cruz                ARI     0.888   0.796   -0.092  -12%
17      Jamey               Carroll             WAS     0.673   0.585   -0.088  -15%
18      Preston             Wilson              COL     0.897   0.813   -0.084  -10%
19      Neifi               Perez               CHN     0.741   0.659   -0.083  -13%
20      Jason               Phillips            LAN     0.752   0.673   -0.079  -12%
21      Brad                Ausmus              HOU     0.697   0.621   -0.076  -12%
22      Aaron               Miles               COL     0.752   0.676   -0.075  -11%
23      Desi                Relaford            COL     0.715   0.642   -0.073  -11%
24      Jim                 Thome               PHI     0.784   0.712   -0.072  -10%
25      Corey               Patterson           CHN     0.721   0.649   -0.072  -11%

Top-25 AL Over-Performers

Rank    First               Last                Team    PrOPS   OPS     PrOPS+  PrOPS%
1       Carlos              Guillen             DET     0.752   0.863   0.111   13%
2       Mike                Sweeney             KC      0.798   0.902   0.103   11%
3       Hideki              Matsui              NYA     0.823   0.915   0.092   10%
4       Bobby               Crosby              OAK     0.833   0.925   0.091   10%
5       Adam                Kennedy             LAA     0.730   0.803   0.073   9%
6       Johnny              Damon               BOS     0.799   0.858   0.060   7%
7       Brandon             Inge                DET     0.755   0.803   0.048   6%
8       Alex                Rodriguez           NYA     0.951   0.998   0.047   5%
9       Miguel              Tejada              BAL     0.931   0.977   0.046   5%
10      Garret              Anderson            LAA     0.752   0.795   0.043   5%
11      Vladimir            Guerrero            LAA     0.929   0.971   0.042   4%
12      Michael             Young               TEX     0.865   0.906   0.041   5%
13      Jhonny              Peralta             CLE     0.833   0.874   0.041   5%
14      Brian               Roberts             BAL     0.969   1.007   0.038   4%
15      Emil                Brown               KC      0.765   0.800   0.035   4%
16      Kevin               Mench               TEX     0.873   0.906   0.034   4%
17      Darin               Erstad              LAA     0.721   0.749   0.028   4%
18      Frank               Catalanotto         TOR     0.735   0.761   0.026   3%
19      Nick                Punto               MIN     0.723   0.749   0.026   3%
20      Alexis              Rios                TOR     0.716   0.739   0.023   3%
21      Richie              Sexson              SEA     0.844   0.865   0.022   3%
22      Bobby               Kielty              OAK     0.778   0.799   0.021   3%
23      Tadahito            Iguchi              CHA     0.728   0.747   0.019   3%
24      Gary                Sheffield           NYA     0.903   0.920   0.018   2%
25      Torii               Hunter              MIN     0.806   0.823   0.017   2%

Top-25 NL Over-Performers

Rank    First               Last                Team    PrOPS   OPS     PrOPS+  PrOPS%
1       Derrek              Lee                 CHN     1.052   1.186   0.134   11%
2       Nick                Johnson             WAS     0.819   0.952   0.133   14%
3       Ryan                Church              WAS     0.811   0.924   0.114   12%
4       Brad                Wilkerson           WAS     0.701   0.809   0.108   13%
5       Miguel              Cabrera             FLA     0.854   0.958   0.104   11%
6       Mike                Cameron             NYN     0.781   0.879   0.099   11%
7       Marcus              Giles               ATL     0.772   0.853   0.081   10%
8       Todd                Walker              CHN     0.746   0.827   0.080   10%
9       Jason               Bay                 PIT     0.866   0.930   0.063   7%
10      Jeff                Cirillo             MIL     0.746   0.809   0.063   8%
11      Wilson              Betemit             ATL     0.807   0.870   0.063   7%
12      Jim                 Edmonds             STL     0.881   0.942   0.061   6%
13      Alex                Gonzalez            FLA     0.648   0.708   0.061   9%
14      Vinny               Castilla            WAS     0.669   0.727   0.058   8%
15      Kenny               Lofton              PHI     0.773   0.830   0.057   7%
16      Brian               Giles               SD      0.891   0.942   0.051   5%
17      Sean                Casey               CIN     0.712   0.761   0.050   7%
18      Morgan              Ensberg             HOU     0.934   0.983   0.049   5%
19      Chipper             Jones               ATL     0.876   0.923   0.048   5%
20      Jose                Guillen             WAS     0.853   0.899   0.046   5%
21      Antonio             Perez               LAN     0.768   0.813   0.044   5%
22      Carlos              Beltran             NYN     0.712   0.754   0.042   6%
23      Olmedo              Saenz               LAN     0.810   0.850   0.040   5%
24      Larry               Walker              STL     0.824   0.862   0.038   4%
25      Tony                Clark               ARI     0.947   0.983   0.037   4%

The PrOPS-Star Teams

Finally, I generated All-Star teams for both leagues based solely on PrOPS. I'll call them the PrOPS-Star Teams for fun. Note that these are based solely on offense for the first-half of 2005. I picked the top-three PrOPS players by position regardless of injury status. A player was eligible for the team if he had more that 150 plate appearances and logged more than 200 innings at that position. Outfielders were picked at each position not as a whole.

2005 PrOPS-Star Teams

        AL                                      NL
Pos.    Player              Team    PrOPS       Player              Team    PrOPS
C       Jason Varitek       BOS     0.906       Michael Barrett     CHN     0.831
C       A.J. Pierzynski     CHA     0.833       Mike Matheny        SF      0.803
C       Ben Molina          LAA     0.819       Jason LaRue         CIN     0.789

1B      Mark Teixeira       TEX     0.946       Derrek Lee          CHN     1.052
1B      Tino Martinez       NYA     0.923       Albert Pujols       STL     0.985
1B      Rafael Palmeiro     BAL     0.913       Tony Clark          ARI     0.947

2B      Brian Roberts       BAL     0.969       Jeff Kent           LAN     0.875
2B      Alfonso Soriano     TEX     0.888       Damion Easley       FLA     0.864
2B      Jorge Cantu         TB      0.824       Chase Utley         PHI     0.858

3B      Alex Rodriguez      NYA     0.951       Morgan Ensberg      HOU     0.934
3B      Melvin Mora         BAL     0.854       Aramis Ramirez      CHN     0.905
3B      Hank Blalock        TEX     0.833       Chipper Jones       ATL     0.876

SS      Miguel Tejada       BAL     0.931       Clint Barmes        COL     0.886
SS      Derek Jeter         NYA     0.869       Felipe Lopez        CIN     0.873
SS      Michael Young       TEX     0.865       Bill Hall           MIL     0.824

LF      Manny Ramirez       BOS     1.008       Adam Dunn           CIN     0.970
LF      Kevin Mench         TEX     0.873       Cliff Floyd         NYN     0.913
LF      Eric Byrnes         OAK     0.853       Reggie Sanders      STL     0.905

CF      Hideki Matsui       NYA     0.823       Andruw Jones        ATL     0.986
CF      Mark Kotsay         OAK     0.822       Xavier Nady         SD      0.901
CF      Vernon Wells        TOR     0.813       Preston Wilson      COL     0.897

RF      Vlad Guerrero       LAA     0.929       Bobby Abreu         PHI     0.968
RF      Gary Sheffield      NYA     0.903       Moises Alou         SF      0.900
RF      Jermaine Dye        CHA     0.899       J.D. Drew           LAN     0.897

There are a few surprises here, but nothing mind-blowing. While this was all fun for me, I'm still not certain how good the information PrOPS provides is. Hopefully, at the conclusion of the season I'll have a better grasp on it. As always, I welcome suggestions for improving the metric.

Click here to learn about THT's download subscriptions.

Scout’s Honor: A Review

JC Bradbury — 2005-06-16T04:10:15+00:00

Moneyball isn't just a book by Michael Lewis anymore. The book title has coined a term on a level that no book has done since Catch-22. Moneyball is now a philosophy. There's no doubt that it has infected the bloodstream of baseball. Depending on whom you talk to, Moneyball is either a wonder drug or a plague, and there seems to be little neutral ground on this; you are either for it or against it.

Put Bill Shanks, author of Scout's Honor: The Bravest Way To Build A Winning Ballteam, in the most extreme corner of the anti-Moneyball camp. Not only does he not agree with the Moneyball philosophy, but he doesn't care for the book itself. And it's a shame, because the bitter taste of Lewis' book sours an excellent in-depth look at the recent history of the Braves. Personal stories become merely a prop for the flawed anti-Moneyball agenda of Shanks when jabs at sabermetrics seem to come out of nowhere. Although he doesn't directly address Moneyball until the last chapter, it's clear what the first 23 chapters are building up to.

If you are a Braves fan, you need to buy this book. Though I disagree with Shanks about a lot—and we have aired our personal differences with each other over the past year—after reading this book I feel a strange kinship with Shanks that I think stems from our age. The time between our discovery of baseball and when we become adults is brief, but it's amazing how a few years seem to shape our perceptions. For much of my childhood the Braves stank. What kid in the South could feel sorry for the Cubs when you were a Braves fan in the 1980s?

Sure we had Dale Murphy and Bob Horner, but it was no fun to check the paper to see if the Braves were 20 or 21 games out today. This is why 1991 was such a special year for Braves fans. I remember the teaser headline on the front page of The Charlotte Observer in April of that year: "Braves in First Place: Opening Day Again." And every Braves fan who read that gave a pity chuckle to that sad remark. We had little hope, we stank and would always stink it seemed.

Little did anyone know what the worst-to-first Braves were about to start. I understand why rest of the country has to hate the Braves’ continued success, yet in my mind the 13-year playoff run of the Braves still doesn't seem to right the injustice of the horrible Braves of my adolescence. I think Shanks feels the same way, which is why he does an excellent job of chronicling the Braves' rise from a losing organization into a model major league ball club.

The book's story really starts in "the beginning" as I remember it. Any past glory in the Braves organization is barely mentioned, as it's irrelevant to the modern story. The story begins when Ted Turner buys the club in 1976. Despite some brief flirtations with success in the early 1980s, the Braves were in a sorry state. But everything changes in 1986 when Stan Kasten takes over for Turner as president of the Braves.

Kasten's success with the Atlanta Hawks gave Turner the confidence to allow Kasten the freedom he needed to fix the Braves. Rather than buying big name free agents like Bruce Sutter, Kasten wanted to grow the next generation of Braves on the farm. And to have a good minor league system he was going to have to have good minor league instruction and a large corps of scouts. With Kasten as president, Bobby Cox rejoining the Braves as general manager, and Paul Snyder as the chief scout, the Braves began to turn the organization into a winner with an eight-point plan:

1. Draft as many high school pitchers as possible.
2. Have more tryout camps to find more players.
3. Be patient with the pitchers.
4. Hire more scouts, instructors, and coaches.
5. Get pitchers in every trade.
6. Stay away from free agents.
7. If a season has to be sacrificed, so be it. Don't forget the long-term goals.
8. Don't change the plan.

And once John Schuerholz came on board and Cox moved to the dugout, there was no stopping "The Braves Way" from succeeding.

Shanks tells the story through the individuals who implemented or fit into the plan. Scouts found Tom Glavine in the relatively talent-barren Northeast. Chipper Jones asked the team to wine and dine his family at the Olive Garden (Chipper's restaurant choice) while Todd Van Poppel blew off Cox. I have to say my favorite character analysis is of Schuerholz. Though Shanks makes it clear Schuerholz was not necessarily the architect of the plan, he was the captain. Shanks follows Schuerholz from a jock at Towson State to the Orioles, Royals and finally the Braves.

He comes off as a likeable guy, always loyal, hardworking, and trusting of his underlings. Schuerholz joined the Braves because he liked the strategy the Braves were pursuing: a high school-focused, pitching-intensive and scout-centered farm system. And let's not forget, Schuerholz is really into player "makeup" in evaluating talent. We also get to meet several old and young players who are or were in the organization. Shanks covers the histories of Murphy, Marcus Giles, Adam LaRoche, and Adam Wainwright just to name a few.

However, there is one glaring omission: Leo Mazzone. How on earth do you miss this man's role in the streak? And Mazzone’s role is quite important, because while the Braves have shepherded many good players through the farm system, they like to get their top pitchers (particularly starters) from the outside. Face it, the Braves have been more successful with hired guns than the "young guns." Greg Maddux, Denny Neagle, John Burkett, Russ Ortiz, Jaret Wright, John Thomson and Mike Hampton were not Braves farm system products. Most of these guys were much more successful with the Braves than without, and their pitching on the club has been the key to its success.

Mazzone has squeezed more out of these guys than any other pitching coach could have. Yeah, the Braves did grow a Hall of Famer in Glavine, John Smoltz has been great (though he was not drafted by the Braves), and Jason Schmidt has ended up having a good career; however, most of the Braves’ homegrown starters, at their best, became average major league pitchers. Without Mazzone, Cox, and Schuerholz's ability to grab veteran arms, the Braves' streak would not have happened. I kept waiting for Mazzone’s chapter, but it never came.

Shanks likes makeup, and he thinks it's the real key to the Braves' scouting success. There is no doubt that the word that makes most statheads cringe is thrown around quite a bit by Braves executives. The players the Braves want have something that can't be quantified. We learn the scouts liked Glavine because of his hockey player mentality, and Chipper landed with the Braves because he broke his hand while defending his pitcher. Some even claim makeup is most important of all the qualities the Braves use to evaluate talent.

I have a hard time buying the fact that this differentiates the Braves way from the Moneyball way, or any other way. Defining makeup is difficult. Shanks quotes many people on the definition of this mysterious quality, but never really acknowledges what a vague concept this is. I think Braves scouting director Roy Clark does the best job:

Well, it's the guy you want at home plate with the game on the line and he wants to be there. Or it's the guy who's on the mound in a tough situation and you know that he is thriving on the moment.

Is Billy Beane looking for anything different? How could anyone disagree that this represents an important quality in a player? Makeup is to baseball front offices as education is to politicians; everyone is for it. Of course, whatever it is Clark defines above is important to the Braves, but it's certainly not something that separates the Braves from any other baseball organization. Furthermore, accusing Beane of opposing makeup as it’s defined here is ludicrous, especially considering the "put a Milo on him" scouting discussion in Chapter Two of Moneyball. And didn't Beane ship out OBP poster-child Jeremy Giambi for partying too much?

One of the central examples of the importance of makeup in Moneyball is that the five-tool scouts' dream that was the young Beane was simply spoiled by internal demons that scouts missed. And the scouts, in awe of his tools, forgot to notice that Billy's play on the field reflected things that they were missing in their puppy dog eyes. But Lewis doesn't dwell on makeup because it's simply not something that differentiates baseball clubs from one another.

And though the Braves may stress makeup, they certainly don't have much room to point fingers at the A's as Shanks does. The Braves have overlooked many indiscretions of their own players: Chipper's extramarital affair, Andruw Jones' participation in the Gold Club scandal, Rafael Furcal's two DUIs, and don't forget John Rocker, who fits the mold of everything the Braves crave in a prospect—a smart Georgia high school product with competitive drive. Add to this the past acquisitions of known clubhouse cancers Gary Sheffield and Raul Mondesi.

This is not something the Braves should be ashamed of. Like the A's, the Braves made tradeoffs no different than the one Paul DePodesta made when he brought in the troubled Milton Bradley. These players are human and suffer from weaknesses. But to ignore these players' extraordinary talents in the name of makeup would cost any GM his job. Don't get me wrong—there are certain qualities players must possess to be successful in Major League Baseball that are not reflected in the box score. But if Chipper Jones' willingness to lay out a high school player is more important than his career .400 OBP, then I've got some teenagers to punch.

Makeup is just a buzzword, like "proactive." It's seductive because it seems to imply something important when it's just "active" with a bike horn on the front. Many can't-miss prospects flame out, just as undrafted nobodies become stars. Our minds tell us there has to be some reason for this, and it's an easy out to say "that guy had/lacked makeup.” There's just something so deeply unsatisfying with such an explanation. You can't argue with it, because its definition is so malleable.

Let's just cut out the fluff: some organizations are better at identifying talent than other organizations, and the Braves are one of those organizations. Some old-school scouts are simply better at it than others, just as some statheads differ in their abilities. Let's just say this is so rather than pretend old scouts possess some quality that sabermetricians just can't understand. Yet I have no doubt that deep down in their hearts the Braves front office members feel they're evaluating something called makeup in these kids. Let's try to figure out what that actually is rather than simply identify it as some intangible quality inside a black box. Scout's Honor provides a peek into this box (tryout camps, an abundance of scouts and coaches, and a stable system), but overall the reader is just supposed to be satisfied with "makeup is makeup."

What's selling this book isn't the story of the Braves. The anti-Moneyball marketing strategy makes this book sexy to the masses. The problem is that Shanks just didn't get, or even read, Moneyball, and not because the message was a difficult one to grasp. Shanks’s relationships with scouts, which allowed him to provide such a good picture of what goes on within the Braves, unfortunately caused him to take what Lewis had to say personally. And in his blind rage to strike back he reveals that he is not all that familiar with the book that boils his blood. Take for example Shanks’s interpretation of Moneyball:

There are two differences that set the A's and other "Moneyballers" apart from the rest of baseball. First, their use of statistics is extreme, believing that on-base percentage is the primary indication of big league success, and that stats override makeup in determining who will make it to "the show." Also, speed and defense are trivial. It's all about OBP.

Secondly, due to their financial restrictions, the A's claim that if they're going to spend money on draft picks, they must not miss. They feel the best way to get a value pick is to emphasize college players and to almost ignore talent from the high school level.

The first difference is just plain wrong. As I argued above, the A's aren't ignoring that black box of makeup at all. Maybe they don't talk about it as much as the Braves do, but the A's aren't going to be anymore thrilled with a smooth-fielding, .600-OBP college shortstop who's in prison than the Braves. But the greater sin is to say that playing Moneyball involves ignoring defense and speed.

I seem to recall an extensive discussion in Moneyball on the A's amazing system designed to measure defense. The A's were willing to sacrifice some defense in right field to get Jeremy Giambi's bat in the lineup; however, they were equally distraught over the loss of Johnny Damon's stellar defense in center, despite his relatively modest OBP. To miss these examples, which I pulled straight from Chapter Six of Moneyball, is simply inexcusable from someone who is taking a book to task in such a condescending tone.

Additionally, Shanks misunderstands the "stress" the A's placed on OBP. Using statistical techniques, Beane's sidekick DePodesta (whom Shanks feels doesn’t deserve the Dodgers' GM job) finds that OBP is much more important to producing runs than previously thought, not to mention an excellent predictor of future success among prospects. Had Shanks just taken a brief look at the stats, he would know that the A's OBP isn't really all that spectacular. As economists Skip Sauer and Jahn Hakes have demonstrated, the A's were jumping on an under-pricing of OBP at the time Moneyball was written, but since the labor market for baseball talent has begun to properly value OBP the A's have moved on to find undervalued talent in other areas. That's what Moneyball is—exploiting market inefficiencies. And it's no different than Schuerholz's signing a veteran castoff like Julio Franco.

On the focus on college pitchers, I can't understand why Shanks is so distressed. Didn't Randy Johnson and Roger Clemens go to college? Why not chastise the Braves for their idiotic philosophy that caused them to miss out on the two most dominant pitchers of the current era? But the Braves don't merit this criticism any more than the A's do for focusing on college pitchers. Why did the A's focus on college pitchers? Because they could develop statistical methods to identify pitching prospects at this level. One consistent criticism in Scout's Honor is that statistics don't tell you much about high school players, but that's the whole point of focusing on college players! To quote from Chapter Two of Moneyball:

From Paul's point of view, that was the great thing about college players: they had meaningful stats. They played a lot more games, against stiffer competition, than high school players. The sample size of their relevant statistics was larger, and therefore a more accurate reflection of some underlying reality.

This is the heart of Moneyball: finding new and unique ways to win cheaply. Identifying a low cost method to identify talent was a big part of this. Yes, the A's are bound to miss out on some excellent high school pitchers, but they do find good talent for less. Limiting the focus on high school pitchers allowed Oakland to devote its scouting resources to other areas, like signing Jermaine Dye (whom the Braves traded for Michael Tucker), Eric Chavez and Jason Kendall.

The Braves set up a huge scouting network with extensive cross-checking, identifying honey holes of talent (particularly in Georgia and Latin America), and holding repeated tryout camps to monitor progress of prospects. Like the A's saved resources by limiting scouting on one level, so did the Braves, so they could afford to sign and re-sign players who helped them win. Neither philosophy is wrong; they are just different. The major difference with the A's is that they didn't just do what everyone else was doing a little better—they developed something new and cheaper.

If you are going to try and prove that a book is flawed, it's a prerequisite to read and understand the content at the center of your attack. Shanks seems to be responding to some caricature reported by Beane's critics. To so grossly mischaracterize Moneyball is irresponsible. Maybe Moneyball does need a good swift kick in the pants, but Shanks misses so badly that he kicks himself in the face.

Another oversight is that Shanks fails to see the similarity of the Braves and the A's. The Braves teams in the 13-season run of division titles possessed many of the qualities hyped in Moneyball, and they can be seen in the stats. The table below lists the average league ranks in OBP, SLG, OPS, ERA, plus average wins during Beane's and Schuerholz's respective tenures.

Team    OBP     SLG     OPS     ERA     Wins
ATL     6.6     4.9     7       2.1     96
OAK     5.2     7.6     6.4     3       92

If the Braves are finding something the A's were not, Shanks would have a case. The Braves and A's both won on the field with the stats "Moneyballers" claim to be important on offense. And if you look at the pitching side of the equation the A's "Big Three" of Tim Hudson, Mark Mulder and Barry Zito look much like the Braves' Glavine, Maddux and Smoltz. I'll grant that the Braves' run is much more impressive, and certainly Beane would too, but that's not my point. My point is that both of these organizations achieved very difficult feats of averaging over 90 wins a season for several years by doing the same things on the field: excellent pitching and solid offense.

The Braves may have developed their on-field success in a way that was different from the A's, but this does not prove that the Moneyball philosophy is flawed. In fact, quite the opposite is true. That a careful understanding and use of empirical methods based on sound statistical principles employed by a few intelligent men can achieve success similar to a very large organization of traditional scouts is proof of success, not failure. The success of the Braves is something Beane wanted to emulate, but it wasn't feasible given the constraints imposed by his bosses. Beane had to find a way to win with less, and he did.

The A's still use scouts; they just use fewer of them and may use them in different ways. This is something that is also clearly stated in Moneyball. The sabermetric method that the A's employ is simply a new technology no different from the radar gun carried by the scouts Shanks loves so dearly. And just as the Luddites wished to destroy a new technology that threatened their livelihood, scouts have reason to feel threatened by the new knowledge brought forth by sabermetrics. Moneyball is not the fad that Shanks claims but a new technology. It's superior to the old methods in some areas, but not all. It's not going away. And while traditional scouting methods are an old part of the game, the process of technological innovation (sometimes known as creative destruction) is much older.

To end on a positive note, with Scout's Honor Shanks proves he's a good journalist with a passion for the Atlanta Braves. The writing is good and the angles he takes in portraying his human subjects make the book easy to read and enjoyable. Only in a few cases did I find myself wishing he had examined a person further. I now feel like I know much more about the architects of the Braves' 13 division titles and what the philosophy of the organization is.

This is no small feat, and Shanks should be commended for his excellent work in this area. I am very happy that I purchased and read this book, as Shanks has more than earned his royalties. I only wish he'd stuck to what he does well than veer off into a rant against Moneyball (the philosophy and the book). He fails miserably in this area, and I'm afraid it's the part that people are going to focus on most.

Click here to learn about THT's download subscriptions.

Another Look at DIPS

JC Bradbury — 2005-05-24T04:05:15+00:00

The discovery of Defense Independent Pitching Statistics (DIPS) theory by Voros McCracken dramatically changed the way we evaluate pitchers. DIPS theory rests on the premise that the stats a pitcher generates without the help of his fielders contain nearly all of the information needed to predict pitcher success; therefore, knowing a pitcher's tendency to allow hits on balls in play tells us very little. The strong-form of the argument (preventing hits on balls in play is not a skill) is not held by all, but the weak form (typically, pitchers have very little skill in preventing balls in play) has strong support within the sabermetric community. I decided to take a closer look at DIPS focusing on recent baseball history, and in this article I present what I found. Though many of my results mirror old findings, I did find a few interesting things along the way.

I'm going to separate my analysis into two parts. First, I analyze the predictive power of DIPS. Sabermetricians use DIPS to predict performances of players in the future and to judge players in the present. Forecasting systems rely heavily on DIPS for obvious reasons, and an in-season ERA predicted via DIPS metrics (such as FIP) lets us judge how well a pitcher is currently performing without the noise of ERA. Second, I dig a little deeper to examine how much control, if any, pitchers have over hits on balls in play.

Part I: Repeatable Performance as a Measure of Skill

Simple Correlations

What can we learn about a pitcher from his stats from year-to-year? My first step in answering this question was to determine the correlation of individual pitching metrics over time. The stronger the correlation, the more likely it is that the statistic is useful in predicting future performance. A repeated performance from season-to-season may also indicate some individual skill possessed by the pitcher in this area. So, I estimated the season-to-season correlations of ERA (park-corrected using 3-year pitcher park factors), strikeout-rate (K9), walk-rate (BB9), home run-rate (HR9), and batting average on balls in play (BABIP) --- all rates are per 9 innings. I looked at seasons from 1980-2004 in which pitchers threw more than 100 innings for two consecutive seasons as my sample. This gave me over 500 pitchers and more than 2000 pitcher seasons to examine.

The scatter plots below each include a regression line indicating the best fit of the impact of the previous year's stat on the current year's stat. The R² tells us the percent of the change in the current year's stat that can be explained by that same statistic from the previous year. The first stat is the most common one used to evaluate pitchers: ERA.

http://bradbury.sewanee.edu/tht/erapfr.png

It's obvious to see why sabermetricians have been down on ERA for so long. Without the regression line it would be hard to tell if there was much of a relationship at all. We know that good pitchers tend to have good ERAs from year-to-year, and the opposite with bad pitchers, but for nailing down precisely how good or bad a pitcher will be from year-to-year, ERA is not very helpful. The ERA from the previous season explained only about 13% of variance of the following season's ERA. Next, I moved on to the seeming source of the problem: BABIP.

http://bradbury.sewanee.edu/tht/babipr.png

If pitchers don't have much ability to prevent hits on balls in play, the correlation across seasons should be weak, and the R² of 0.06 seems to support this central tenet of DIPS. From the season-to-season correlation, pitchers did not seem to have much special ability to prevent hits on balls-in-play. It's interesting to note that the reported R² is generated from a bare bones regression that does not control from some important omitted factors. One particular factor, the defense behind a player that doesn't change teams, is probably heavily biasing that paltry 6% upwards. Next, I examined the holy trinity of DIPS: strikeouts, walks, and home runs.

http://bradbury.sewanee.edu/tht/k9r.png

Everyone evaluates pitchers by strikeouts, and they should. No variable produced by pitchers had more constancy over time than the strikeout-rate. The strikeout-rate from the previous year explained over 60% of the variance of the following year's strikeout-rate. What about walks?

http://bradbury.sewanee.edu/tht/bb9r.png

Though walks from season-to-season were not as predicable as strikeouts, the correlation was quite strong, with an R² of 0.42. And homers?

http://bradbury.sewanee.edu/tht/hr9r.png

Well, the correlation for home runs was about half of what it was for walks, but the recent past revealed nearly twice as much about the future for home runs than it did for plain old ERA.

So far, DIPS theory seems to hold up quite well. In fact, compared to McCracken's initial estimates, the correlations are quite similar. But, so what if some metrics are more correlated than others over time? Strikeout-rates may be strongly correlated from season-to-season, but what impact do they have on pitcher run prevention? Next, I wanted to see the impact of each metric on predicting ERA. Again, I found that DIPS theory contains some powerful truths.

Using DIPS to Predict ERA in the Present

In order to figure out the ability of DIPS to predict future ERA I needed to establish a baseline impact for these stats during a current season on the current ERA of the pitcher. I employed a linear regression technique to estimate the impact of different pitching statistics on ERA. This technique is designed to handle many problems of multiple observations of individuals over time -- the regression results were estimated using random effects and corrected for first-order serial correlation.

To begin, I estimated the impact of the big-3 (K9, BB9, HR9), plus the hit-batter-rate (HBP9) DIPS components and BABIP, on ERA, while controlling for the defense of the team, the age of the pitcher , the league of the pitcher, and the season. I used the pitcher's team seasonal BABIP to proxy defense, assumed the impact of age on ERA to be U-shaped (quadratic), and used indicator variables equal to 1 or 0 to identify the league and year in which the pitcher's stats were posted. Here are the unit and percentage impacts of the variables on a pitchers ERA (again, park-corrected).

Table 1. The Impact of Current Season's Peripheral Stats on ERA

Stat    Unit        Percentage
K9¹     -0.17       -0.24%
BB9¹     0.30        0.23%
HR9¹     1.42        0.32%
HBP9¹    0.34        0.02%
BABIP¹   18.56       1.31%

R² = 0.77
Superscripts for levels of statistical significance:
⁵ = 5% significance
¹ = 1% significance

From the unit impact we can see the effect of a one-unit change in the pitching statistic on ERA. In this sample, an increase of one strikeout per nine innings lowered a pitcher's ERA by about 0.17. The percentage impact (or elasticity) tells us the percentage change of ERA in response to a 1% change in the statistic. So, a 1% increase in K9 lowered ERA by 0.24% (the percent changes are calculated at the average values of the statistic and the ERA). The percentage impact helps us judge the impact of the different metrics relative to their normal values. For example, the unit impact of every walk is nearly twice that of a strikeout; however, in terms of the average number of walks and strikeouts their percentage impacts on ERA are nearly identical.

Differences in the variables regression, as a whole, explained 77% of the variance of pitcher ERAs, which is quite good. All of the estimated impacts were "statistically significant," meaning there is less than a 5% chance that these variables have no effect on ERA.

The thing I find most interesting about these estimates is how well they correlated with linear weight values for these events. Walks, HBPs, and home runs were in line with linear weights. Strikeouts were a little low, but close; however, this estimate was for earned run prevention. Strikeouts ought to have some additional run-suppressing impact on unearned runs that may account for the difference. The large impact of BABIP is quite interesting. A standard deviation increase in BABIP (.023) raised a pitcher's ERA by about 0.43 (approximately 10% of the average ERA for the sample). If pitchers have little effect over balls in play, then a random fluctuation of BABIP can influence a pitcher's ERA quite a bit. I'm quite happy with these estimates, and they provide a fine baseline to evaluate how well the DIPS statistics predict future run prevention.

Using DIPS to Predict ERA in the Future

Predicting an ERA from a pitcher's statistics from the prior season was not much different from the exercise above. All of the estimates reported below were estimated using the previous season's K9, BB9, HR9, and BABIP, while the defense, age, and league control variables were from the current year.

Table 2. The Impact of Previous Season's Peripheral Stats on ERA

Stat    Unit        Percentage
K9¹     -0.18       -0.25%
BB9¹     0.13        0.10%
HR9¹     0.18        0.04%
BABIP ¹ -1.96       -0.14%

R² = 0.30

All of the reported estimates were statistically significant (HBP9 did not seem to be important, so I dropped it from the regression).

The result for strikeouts is quite interesting. The previous year's strikeout-rate impacted the current season's ERA about as much as the current season's strikeout-rate (see Table 1 for comparison). This is not totally surprising, because strikeouts were strongly correlated from year-to-year, but I did not expect such a strong relationship. While walks and homers were important, they were not as consistent predictors of ERA as strikeouts.

BABIP had a statistically significant impact on ERA, but the effect was small and in the opposite direction that one would expect. Rather than thinking there is some inverse relationship between BABIP from year-to-year, this is more likely derived from a few extremely high and low BABIP seasons that typically regress to the mean the following year. When I estimated the model using a median regression technique that minimizes the impact of extreme values, BABIP was no longer statistically significant.

But how does the DIPS prediction stand up to a prediction based on the previous season's ERA? Quite well.

Table 3. The Impact of Previous Season's ERA on Current Season's ERA


Stat        Unit       Percentage
Lag ERA     0.005      0.0045%
R² = 0.18

Using all of the same control variables in the DIPS-only model above, I found a very weak relationship between the previous year's ERA and in ERA the following year. In reality the effect was nothing since the estimate was not statistically different from zero. The R² was not quite half of the DIPS-only model. But, I wanted to look a little deeper. Maybe, after controlling for the impact of the DIPS components, ERA contains some extra information about a pitcher's future performance. Possibly, ERA from the previous year could capture some sort of clutch ability to prevent runs.

Table 4. The Impact of Previous Season's Peripheral Stats and ERA

on the Current Season's ERA

Stat        Unit        Percentage
ERA¹        -0.11       -0.11%
K9¹         -0.19       -0.19%
BB9¹         0.15        0.11%
HR9¹         0.26        0.06%
BABIP       -0.68       -0.05%

R² = 0.29

When I included the lag of ERA in the full model with the DIPS variables and BABIP, ERA did become statistically significant ... in the wrong direction. However, the impact was tiny; and, as it was for BABIP, it was probably the result of a few outlying extreme values regressing back to the mean.

So, when it came to predicting pitching success, DIPS contained the most consistent information. This is no shock. But, I'm not sure if these findings necessarily mean pitchers do not have control over balls in play. I'll explain why in Part II.

Part II: Pitcher Control Over Hits on Balls in Play

Now that I have confirmed the predictive value of DIPS, there is another question to answer. Is it possible that pitchers do have the ability to affect hits on balls in play, but that this influence is so strongly correlated with the DIPS that it is masked? Multiple regression analysis identifies a correlation between the predicting and predicted variables included in the model, but it does not tell us why. If a pitcher strikes out a lot of batters it does not necessarily mean that the corresponding effect on ERA comes solely through the direct impact of strikeouts. The correlation between strikeouts and ERA could reflect a pitcher's ability to affect hits on balls in plays in addition to the direct effect on limiting balls in play. If strikeout pitchers cause weak groundouts and walk-prone pitchers serve up more line drives, these factors will be captured in the weights assigned to strikeouts and walks in a multiple regression estimation.

For all practical purposes, this possibility is irrelevant -- if DIPS tells us all we need to know about run prevention, it doesn't matter why -- but I wanted to see if it was true. First, I estimated the impact of the DIPS components on a pitcher's BABIP in the same season.

Table 5. The Impact of Current Season's Peripheral Stats on BABIP

Stat        Unit        Percentage
K9          -0.0004     -0.0087%
BB9⁵         0.0011      0.0114%
HR9         -0.0014     -0.0044%
HBP9        -0.0005     -0.0004%
R² = 0.28

It turns out that within the same season only walks were correlated with BABIP at a statistically significant level. However the effect was tiny in both unit and percentage terms; every walk per game increased the BABIP of a pitcher by about 0.001. Theoretically, I'm not too surprised by this. A pitcher who walks more batters is going to be more likely to place balls over the plate that can be hit hard, plus he may be behind in the count often and have to throw a meatball. But, the effect was minuscule and largely irrelevant: the effect of each walk on ERA through BABIP was about 0.009 earned runs per game.

What about the possibility of all of the DIPS from the previous year influencing the present year's BABIP? Maybe, the previous year's BABIP doesn't give us much information about the following year's BABIP, but the DIPS do because they are correlated with a pitcher's ability to affect hits on balls in play. If this is the case, then it's possible for a pitcher to control BABIP through his DIPS. And in a multivariate regression on ERA, like I ran in Part I, the effect would be captured by the DIPS variables. Therefore, I estimated the impact of the previous year's pitching statistics on the following season's BABIP.

Table 6. The Impact of Previous Season's Peripheral Stats on BABIP

Stat        Unit        Percentage
K9¹         -0.00172    -0.035%
BB9          0.00013     0.001%
HR9⁵        -0.00347    -0.011%
BABIP⁵      -0.04148    -0.041%

R² = 0.30

The results confirmed something startling in the magnitude and statistical significance of the predicting variables: differences in pitcher control over hits on balls in play were somewhat predictable from past performance. But, that information is not in the statistic we would think to look at first, BABIP. This ability has been hidden due to its correlation with DIPS metrics. It turns out that, in fact, the strikeout and home run rates were inversely related to BABIP in the following season. Though it's not widely discussed, Voros McCracken also found correlations between both strikeouts and home runs with a pitcher's future BABIP. And in his DIPS 2.0 article, he adjusted for pitcher influence in this area. On strikeouts he writes,

Looking at the numbers over and over and over again, it becomes clear that a pitchers strikeout rate during a single season is a bit better predictor of his hits in play the following year than his own hits per balls in play. This is there and it's real.

And on home runs he states,

While a shaky relationship, it appears that the more Home Runs a pitcher gives up, the fewer hits per balls in play he gives up.

The effect for strikeouts seems a bit obvious. The fear of strikeouts possibly induces hitters to take weaker protective swings to stay alive, and thus yields softer hits that are more likely to result in outs. The effect of home runs seems a bit counterintuitive at first, but it's capturing the effect of the ground-ball-to-fly-ball ratio. Pitchers who give up more fly balls are likely to give up home runs, but also produce more outs, as fly balls are more likely to yield outs than ground balls.

But just because something is statistically significant does not mean it is practically significant. Using the estimate of the impact of the predicting variables reported in Table 6 and my earlier estimate of the impact of BABIP on ERA (18.56) in Table 1, I was able to assign an earned run value to strikeout and home run prevention through balls in play. The effect of home runs on BABIP is very small (just as McCracken found) so I won't discuss it further, but the impact of strikeouts was large enough to continue investigating. For every one-strikeout increase per game, the BABIP decreases by 0.00172. Multiply that by 18.56 and every strikeout is worth 0.03 earned runs per game. That's small, right? Well, yes and no. Let's use Randy Johnson's 2004 as an example.

In 2004, Randy Johnson struck out 10.6 batters per game. According to the estimates above, this rate would lower Johnson's expected 2005 BABIP by 0.018 (10.6 x 0.00172 = 0.018) and his ERA by 0.34 (18.56 x 0.018 = 0.34); an impact on earned run prevention equal to about two additional strikeouts (2 x 0.17 = 0.34) according to the estimate in Table 1. This translates to 9.3 earned runs saved on decreasing hits on balls in play over the number of innings he pitched in 2004 (0.34 x [245.67/9] = 9.3), which is 13% of his 2004 total of 71 earned runs.

A pitcher with the average strikeout-rate for the sample would gain the ERA benefit of about one extra strikeout per game through his effect on BABIP. This would lower his predicted ERA by 0.18 and result in 3.25 fewer earned runs a season (about 4%). Johnson saves approximately 6 earned runs per season more than the average strikeout pitcher through his ability to prevent hits on balls in play. That is a very real effect. So, why doesn't BABIP correlate very well from year-to-year when strikeouts do? Well, there's just a lot more noise generated by random bounces from year-to-year in BABIP than there is in strikeouts.

The effect that pitchers have over hits on balls in play is small compared to the effects of the other DIPS metrics; however, it is large enough to tell us that pitchers do have the ability to prevent hits on balls in play. So where does this leave us? Well, it turns out that though pitchers do seem to have the ability to prevent hits on balls in play, it does not alter the predictive element DIPS theory one bit. Why not? Because that ability is captured in DIPS statistics. Those who are comfortable evaluating pitchers using DIPS can continue to feel comfortable doing so.

These findings also fit with some recent research by Tom Tippett. In Can Pitchers Prevent Hits on Balls in Play?, Tippett looks at a rather large sample of pitchers' BABIP over their careers and finds that several pitchers seemed to have had a consistent impact over hits on balls in play. It would be an interesting project to see how much of this difference is predicted by pitcher strikeout rates.

Conclusion
In summary, DIPS is right. Knowing DIPS can tell you more about a pitcher's future performance than his previous ERA. While pitchers may have some ability to prevent hits on balls in play, the effect is small. And any effect a pitcher does have is reflected within DIPS metrics. Other studies have shown that pitchers do seem to be able to influence certain hit-types, such as ground balls, fly balls, and line drives, but the effect of these tendencies on run-prevention is ambiguous. And it may be that these tendencies may be correlated with DIPS. As new hit-type data comes in this year to The Hardball Times, hopefully, the impact of hit-type on run prevention will become clearer.

Click here to learn about THT's download subscriptions.

PrOPS Again

JC Bradbury — 2005-05-16T13:04:15+00:00

Click here to learn about THT's download subscriptions.

Introducing PrOPS

JC Bradbury — 2005-05-11T04:05:15+00:00

Have you ever been following a team and noticed players who seem to be continually lucky or unlucky? Maybe there's a player who continually dinks the ball between the infield and the outfield. And what about the guy whose spraying liners all over the field but can't crack the Mendoza line, because there always seems to be a defender in the right spot? There's a good chance that most of the noisy outs and swinging bunts come out in the laundry by season's end -- they may not entirely cancel out for all players -- but the numbers we look at now to evaluate hitters, such as OPS, are tainted. And while there is no such thing as a perfect metric to evaluate how well a player is playing absent luck, I think there may be a way to get a better grasp on how well players are performing than just relying on the standard raw statistics.

Early in the season, things haven't had a chance to even out yet. We see historically poor players putting up good numbers (Brian Roberts), and historically good players putting up bad numbers (Bernie Williams). How much of a player's April OPS is a product of chance, and how much reflects the quality of play? Unfortunately, most of the data we use to evaluate players is based on scorebook outcomes (single, double, walk, etc.), and therefore the numbers themselves reflect both random chance and the quality of play. A double gained on an outfielder's untied shoes counts the same as a liner to the gap in the stats. And we would expect players who are putting up the latter type of doubles are better than those reaping the benefits of funny bounces and bad personnel decisions. The recent influx of new data provided by Baseball Info Solutions through The Hardball Times provides a possible way to separate out good/bad play from lucky/unlucky outcomes.

I set out to estimate the impact of certain areas of player performances on their season's OPS using the 2004 season. In particular, I was curious in the types of batted balls (line drives, flyballs, etc.) players were hitting. Is there a correlation between these variables and hitting success? If so, maybe we can learn something about how "good" players are actually playing by looking at this data. To begin, I looked at several different combinations of variables to find the model that best predicted player OPS in 2004 using linear regression estimation. This model uses estimated weights of hitting performances that are not necessarily officially scored outcomes to generate a predicted OPS, or PrOPS. With this model I can evaluate players by the process with which they reached these outcomes; thereby, hopefully separating useful information from the noise of raw statistics.

The model that best predicted a player's OPS in 2004 included the following variables:

Line drives per batted ball
Groundball-to-flyball ratio
Walk rate
Hit-by-pitch rate
Strikeout rate
Home run rate
Home park of the player

While many of these variables are official scorebook outcomes, we know that players do happen to have skills in these areas, and that these skills translate directly and indirectly into a player's OPS. I am most concerned with the random bounces of batted balls in play, which is why I included line drives and the groundball-to-flyball ratio in the model. it turns out that these variables are important in predicting a hitter's OPS. The R² of the overall regression model was .81, which indicates that about 80% of the differences in OPS from player to player were explained by the changes in the included variables. And while we think of luck canceling out over the course of the season, here are lists of the top-25 under/over-performers of 2004 measured as a percent of the player's actual OPS (minimum 400 plate appearances).

OPS: Actual OPS for 2004
PrOPS: Predicted OPS
PrOPS+: Absolute difference between OPS and PrOPS -- a positive PrOPS+ indicates observed performance better than predicted while a negative PrOPS+ indicates observed performance worse than predicted.
PrOPS%: The difference between OPS and PrOPS expressed as a percent of OPS.

2004 Top-25 Under-Performers

Rank    First   Last            OPS     PrOPS   PrOPS+  PrOPS%

1       Desi    Relaford        0.601   0.708   -0.106  -17.72%
2       Scott   Spiezio         0.634   0.740   -0.105  -16.63%
3       Rafael  Palmeiro        0.796   0.898   -0.102  -12.86%
4       Jason   Phillips        0.624   0.702   -0.079  -12.66%
5       Brad    Ausmus          0.631   0.704   -0.073  -11.52%
6       David   Eckstein        0.671   0.737   -0.066   -9.80%
7       Chipper Jones           0.847   0.930   -0.083   -9.79%
8       Tony    Batista         0.726   0.793   -0.066   -9.11%
9       Joe     Crede           0.717   0.781   -0.064   -8.91%
10      Barry   Bonds           1.422   1.537   -0.115   -8.11%
11      Rob     Mackowiak       0.739   0.799   -0.060   -8.05%
12      Craig   Counsell        0.648   0.700   -0.052   -8.04%
13      Jose    Castillo        0.665   0.718   -0.053   -7.95%
14      Aaron   Miles           0.697   0.751   -0.054   -7.81%
15      Placido Polanco         0.786   0.847   -0.061   -7.72%
16      Steve   Finley          0.828   0.891   -0.063   -7.63%
17      Ramon   Hernandez       0.818   0.879   -0.062   -7.56%
18      A.J.    Pierzynski      0.729   0.783   -0.055   -7.50%
19      Toby    Hall            0.666   0.716   -0.050   -7.45%
20      Alex    Gonzalez        0.689   0.739   -0.050   -7.24%
21      Dmitri  Young           0.816   0.875   -0.059   -7.21%
22      Sammy   Sosa            0.849   0.909   -0.060   -7.03%
23      Bill    Mueller         0.811   0.868   -0.056   -6.93%
24      Orlando Cabrera         0.631   0.672   -0.042   -6.60%
25      Matt    Lawton          0.787   0.839   -0.052   -6.59%

2004 Top-25 Over-Performers

Rank    First   Last            OPS     PrOPS   PrOPS+  PrOPS%

1       J.T.    Snow            0.958   0.846   0.112   11.72%
2       Ichiro  Suzuki          0.869   0.774   0.095   10.93%
3       Melvin  Mora            0.981   0.895   0.086    8.74%
4       Jack    Wilson          0.794   0.728   0.066    8.34%
5       Erubiel Durazo          0.919   0.842   0.076    8.30%
6       Aaron   Rowand          0.905   0.830   0.075    8.26%
7       Lyle    Overbay         0.862   0.792   0.070    8.17%
8       Todd    Helton          1.088   1.003   0.085    7.84%
9       David   Newhan          0.814   0.753   0.062    7.57%
10      Carlos  Guillen         0.921   0.853   0.069    7.46%
11      Travis  Hafner          0.993   0.919   0.074    7.42%
12      Mark    Loretta         0.886   0.822   0.064    7.21%
13      Lance   Berkman         1.016   0.944   0.072    7.07%
14      Chone   Figgins         0.770   0.717   0.052    6.79%
15      Juan    Rivera          0.828   0.772   0.056    6.76%
16      Alexis  Rios            0.720   0.674   0.047    6.49%
17      Carl    Crawford        0.781   0.732   0.050    6.34%
18      Ivan    Rodriguez       0.893   0.837   0.056    6.23%
19      Jimmy   Rollins         0.803   0.753   0.050    6.19%
20      Ray     Durham          0.848   0.798   0.050   5.89%
21      Jason   Bay             0.907   0.855   0.052    5.78%
22      Joe     Randa           0.751   0.708   0.043    5.76%
23      Juan    Uribe           0.833   0.786   0.046    5.58%
24      Bobby   Abreu           0.971   0.918   0.053    5.50%
25      Albert  Pujols          1.072   1.013   0.059    5.49%

Desi Relaford wins the award for worst luck of 2004, while J.T. Snow had the best luck. Now, when I say "luck" I want to be clear as to what I mean. Given the batting statistics included in the regression, PrOPS tells us what all other players in MLB did, on average, based on the variables included in the regression model. You can think of PrOPS as similar to DIPS for pitchers. It is entirely possible that some of these players got lucky with hitting line drives, striking out, etc.; however, given their actual numbers for these events we would have expected them to perform much differently.

Now that I have established a baseline impact for the batting statistics on player OPS, I can apply the model to 2005. With only about a fifth of games finished for the season, it's much less likely that good and bad bounces have had time to even out. The model can help us pull out how well players are actually playing by removing some luck. I'm not saying this is perfect, and players may be getting lucky with their hit types, but it's all we've got to work with at the moment.

Now, let's use the model to tell us what OPS a player ought to have in the current season based on their hitting peripherals. I have calculated PrOPS stats for every MLB player with at least one plate appearance. You can view the stats by team for the AL and NL. Here are the lists of the top under/over performers for 2005.

2005 Top-25 Under-Performers

Rank    First   Last            OPS     PrOPS   PrOPS+  PrOPS%
1       Tike    Redman          0.454   0.806   -0.353  -77.75%
2       Aaron   Boone           0.457   0.751   -0.294  -64.44%
3       Jose    Molina          0.470   0.770   -0.300  -63.82%
4       Luis    Rivas           0.477   0.769   -0.293  -61.35%
5       Miguel  Olivo           0.362   0.568   -0.206  -56.77%
6       Nomar   Garciaparra     0.405   0.619   -0.215  -53.13%
7       Keith   Ginter          0.554   0.842   -0.288  -51.96%
8       Wilson  Valdez          0.466   0.685   -0.219  -46.94%
9       Jay     Payton          0.579   0.851   -0.272  -46.91%
10      Jack    Wilson          0.466   0.675   -0.209  -44.80%
11      James   Hardy           0.457   0.639   -0.182  -39.74%
12      J.D.    Closser         0.470   0.649   -0.179  -38.15%
13      Ty      Wigginton       0.523   0.703   -0.180  -34.48%
14      John    Buck            0.474   0.637   -0.163  -34.39%
15      Quinton McCracken       0.515   0.682   -0.166  -32.31%
16      Jason   Kendall         0.573   0.754   -0.180  -31.48%
17      Jose    Hernandez       0.567   0.741   -0.174  -30.66%
18      Richard Hidalgo         0.562   0.730   -0.169  -30.00%
19      Yadier  Molina          0.475   0.616   -0.140  -29.53%
20      Placido Polanco         0.592   0.766   -0.174  -29.43%
21      Jason   LaRue           0.536   0.685   -0.149  -27.79%
22      Casey   Blake           0.669   0.851   -0.183  -27.30%
23      Marcus  Thames          0.654   0.833   -0.178  -27.27%
24      Eric    Byrnes          0.633   0.803   -0.170  -26.82%
25      Brad    Ausmus          0.567   0.710   -0.143  -25.26%

I would look for these guys to rebound. It's not that we really needed a regression model to tell us this, but we can see that players who put up similar peripherals in 2004 performed much better than these guys have shown in their stats so far this season. So hang in there Tike, because better days are coming if you keep hitting like you have been.

What about players who may be looking for a fall.

2005 Top-25 Over-Performers

Rank    First   Last            OPS     PrOPS   PrOPS+  PrOPS%

1       Jason   Ellison         1.005   0.754   0.251   24.99%
2       Carlos  Guillen         1.015   0.772   0.243   23.97%
3       Bill    Hall            0.796   0.622   0.174   21.84%
4       Brad    Wilkerson       0.851   0.678   0.173   20.32%
5       Alex    Sanchez         0.829   0.669   0.160   19.32%
6       Ryan    Freel           0.917   0.745   0.172   18.78%
7       Ricky   Ledee           0.907   0.740   0.167   18.46%
8       Craig   Biggio          0.873   0.716   0.156   17.89%
9       Vinny   Castilla        0.901   0.745   0.156   17.27%
10      Shea    Hillenbrand     0.894   0.747   0.147   16.48%
11      Derrek  Lee             1.224   1.022   0.202   16.47%
12      Justin  Morneau         1.289   1.077   0.212   16.44%
13      Freddy  Sanchez         0.787   0.659   0.127   16.17%
14      Carlos  Beltran         0.881   0.752   0.129   14.69%
15      Rob     Mackowiak       0.770   0.659   0.110   14.31%
16      Nook    Logan           0.816   0.700   0.115   14.12%
17      Kenny   Lofton          0.938   0.806   0.132   14.12%
18      Cliff   Floyd           1.047   0.900   0.146   13.99%
19      Mike    Sweeney         1.016   0.878   0.138   13.56%
20      Brandon Inge            0.877   0.760   0.118   13.39%
21      Nick    Johnson         0.924   0.801   0.124   13.38%
22      Frank   Catalanotto     0.768   0.666   0.101   13.19%
23      Clint   Barmes          1.082   0.940   0.142   13.16%
24      Jacque  Jones           0.990   0.860   0.130   13.09%
25      Chipper Jones           1.111   0.967   0.145   13.01%

Jason Ellison, with his .500 BABIP, certainly won't continue at his same pace. And when Alex Sanchez returns to being Alex Sanchez by the All-Star break, let's not say it was the steroids wearing off. These are the guys you may want to try to unload in your fantasy league.

What about two guys who are off to hot and cold starts, Brian Roberts and Bernie Williams? Are these just statistical anomalies? Well, they might be, but they would have to be anomalies in the batting peripherals that go into the model. Roberts is putting up numbers a little better than predicted (OPS =1.111 , PrOPS = 1.058), but he's still playing well. Bernie is playing poorly (PrOPS = 0.703), but not as bad as his stats show (OPS = 0.607).

Feel free to take a look around on the stats pages and tell me what you see. This is all new stuff, and I will make changes to the model based on any discoveries people make. Personally, I'm just happy to see Johnny Estrada is due for a little rebound (PrOPS = 0.746 versus OPS = 0.632).

Click here to learn about THT's download subscriptions.

Five Questions: Atlanta Braves

JC Bradbury — 2005-04-01T04:04:15+00:00

Braves fans are in a familiar position this year, with the departure of several key players threatening to put the team's division title streak in jeopardy. 2004 looked to be the end for real with the losses of Gary Sheffield, Javy Lopez, and Greg Maddux. The trio was replaced by the glass-jawed J.D. Drew, the previously anemic bat of Johnny Estrada, and John "yawn" Thomson. With the media once again predicting the end of the reign, Braves fans had reason to listen. Clearly, John Schuerholz, Bobby Cox, and Leo Mazzone must know something we don't. Estrada made the All-Star team (deservedly), J.D. just missed it (undeservedly), and Thomson matched Mad Dog is almost every category. In addition, the Braves received unexpected production from Jaret Wright, Antonio Alfonseca, Charles Thomas, and Eli Marrero.

This year the Braves start the season without last season's best players on offense and defense: Drew and Wright. Solid contributors Thomas, Marrero, Alfonseca, Paul Byrd, Russ Ortiz, and Juan Cruz are gone, as well. The big positive changes are the additions of Tim Hudson and John Smoltz to the rotation with Danny Kolb stepping in to take over Smoltz's closing role. But, the acquisitions of Brian Jordan and Raul Mondesi to cover the outfield corners don't seem to constitute a good-faith effort to replace what was lost. In this sense, the Braves look to be in a situation that is slightly worse than last year on offense, with some improvement on defense.

But this year's situation is much different when you look at the whole organization. Last year's opening day roster included Jesse Garcia, Mike Hessman, Dewayne Wise, Will Cunnane, and C.J. Nitkowski ... yuck. All of these players spent much of the season in Triple-A or being cut. This year, the depth of the club is greatly improved, with some real competition over who will get the last roster spots. The Braves have already sent Kelly Johnson, Billy McCarthy, Kyle Davies, and super-stud prospect Andy Marte to Triple-A. Wilson Betemit and Pete Orr are still fighting it out for the final infield slot on the big club. All of these players are far better than the reserves who made the team last year. So, how does this whole club stand up? Let's go to the questions.

1. Can this outfield work?

Geezers on the Corners

So, Schuerholz, you lose an MVP-quality outfielder in right field and a productive platoon in left, what do you do? Sign a border-line headcase, and roll out a formerly adequate outfielder who was a fan favorite. Schuerholz spent a measly $1.6 million on free agent replacements for the outfield; now, that's being thrifty. And you thought Billy Beane is playing Moneyball? According to press reports Mondesi is in great shape. Whether this new athletic commitment translates to good performance on the field is another question. Even if Mondesi doesn't pull a Bobby Fischer, having him in right field may not yield that much production. He's 34, and two years ago he posted an OPS+ of just 114, not good for an aging outfield corner. But, if Mondesi has that potential he showed so long ago, maybe he will have something a little extra to show. Jordan, on the other hand, has to be a platoon player to be productive. He's old (turning 38 just before opening day) and, like Mondesi, he was never all that great anyway, merely adequate. He's been bothered by injuries ever since he left Atlanta in a huff, but outside of last year, he's really been pretty steady for a guy in his mid-30s. He'll start the year in left, sharing time with rookie Ryan Langerhans, who should be the more productive member of the platoon.

However, theses moves show the genius of Schuerholz at work. The free agent market this year was insane. Whether it was a market correction or a bubble waiting to pop, it is too early to tell. The Braves farm is bursting with talent, so why blow a wad of cash on a possibly overpriced free agent like Jeromy Burnitz when you can take the shotgun approach. By the All-Star break, Mondesi may be on a trip with Ricky Williams, and Jordan could be in traction, but so what? These guys are like disposable plates that blow away in a picnic wind. They are cheap, and there's a whole stack of these guys in the Richmond. By not giving up Magglio Ordonez dollars to replace Drew, the Braves also have some cash available to take on a veteran contract in mid-season if all else fails.

The Next Generation

Schuerholz may not even need to go shopping if the old folks don't work out, because the next group of Atlanta regulars is just about ready to make the jump. And with Jeff Francoeur and Andy Marte needing places to play soon, you don't need high-priced veterans taking up slots for the next few years. The outfield in 2006 could include not only Francoeur or Marte, but Johnson, Langerhans, McCarthy, or even Adam LaRoche. All of these guys, except Francoeur, may get a chance to play some major league outfield in 2005.

The Rock

And let's not forget Andruw Jones in center. He'll provide his normally stellar defense and offers up the promise of becoming an MVP with the bat. Atlanta fans are still waiting for the break-out -- he still is just 28 -- but even if he stays the same, it's one position the Braves don't need to worry about. Expect a 120 OPS+ from AJ this year.

2. Can the infield make up for the outfield?

Geezers on the Corners...again

While the corner outfielders may look a little light for a contending team, the infield makes up for it. Chipper Jones should bounce back from a disappointing 2004, in which he hit a "paltry" 117 OPS+ while suffering through a painful hamstring injury. Expect a full-season from Chipper; thanks to a move to the infield, he should stay in the line-up. A healthy and happy Chipper will allow the Braves to get the most out of a still very productive player. Sure, there's no denying that Chipper has declined some from 1999, but I expect Chipper (who is only 33) to post a 130-140 OPS+ year. That's not shabby at all.

Over at first base, Julio Franco continues to be part of a productive platoon and solid pinch-hitter off the bench. He might finally play like a 46-year-old this year, but that would be like expecting the Braves to lose the division. LaRoche is the youth that brings the average age of this platoon down to 35. He showed us a lot in the second half of 2004 (.302/.368/.576), but it's not clear if this was for real. The problem with LaRoche is that he's trying to play a position that the Braves really need to use for Chipper. If you're a lefty and can't mash like a first baseman, it's probably a good idea to shag some fly balls in the spring. The situation at first is magnified by the presence of Marte, who has his own question below.

Up the Middle

This will be the last year to see the Rafael Furcal- Marcus Giles combo up the middle in Atlanta. If Cristian Guzman can get $4 million a year as free agent, think of what a switch-hitting shortstop who can actually hit is going to get. And there is no way Atlanta is going to pay the big bucks for a guy who hasn't exactly been an upstanding citizen. Furcal will be solid with the bat, and will make some mistakes with the glove. Overall, the trade-off will be a good one. Giles has had some bad luck over his career, he's due for some normalcy. Giles gets on base, hits for power, runs the bases well, and is not a bad glove man. He doesn't project all that well for 2005 because his isolated-power dropped 100 points from 2003, but this was likely a result of his injury. In September of 2004, his power was back up to his old level. Barring another freak accident, he is not a bad pick for the NL MVP.

3. Is the pitching improved enough to make-up for any lost offense?

The Starters

This is one place where the Braves really upgraded over 2004. Hudson and Smoltz should be outstanding. There is the question of whether Smoltz's arm will hold up as a starter. There doesn't seem to be any evidence that he's any more likely to get hurt starting than closing. Yes, no one has really done this before at Smoltz's age, but the one-way movement of starters to relievers is largely driven by the fact that starters lose something as they age. Smoltz appears to have plenty left in the tank. From 1995-1999, his best years as a starter, his strikeout-to-walk ratio was 3.87. From 2002-2004 it was 6.4. This is likely indicative that he has gives a little more per appearance out of the pen, but that's what you'd expect from a guy facing high leverage situations. Closing in 2001 and 2002 was probably a good idea, but I think he left a lot of innings in the clubhouse the last two seasons. His repertoire includes two fastballs, a slider, a curve, a change-up, a knuckler, and a side-arm delivery option. Come to think of it, what on earth has John been doing in the bullpen? John has been an amazingly good pitcher the past three years, and I think he will be just fine in his new role.

John Thomson should be quietly solid to very good. Mike Hampton could be good or bad, who knows? Horacio Ramirez posted a good ERA before his injury last year despite dreadfully weak peripherals. He has been very lucky the last two years, and I would not be surprised if he loses his fifth starter slot to Kyle Davies or Chris Reitsma.

The Relievers

The bullpen suffered some very significant losses from last year (Smoltz, Alfonseca, and Cruz), and the bullpen was quite good, ranking 4th in the NL in terms of Win Probability Added (WPA). Smoltz's anointed replacement, Kolb, is a very different style of pitcher. Like Smoltz, he has pitched in a lot of high-leverage situations, but he has not been as successful in terms of WPA. Kolb is a low-walk, low-K, extreme ground-ball pitcher; he's good because keeps the ball in the park, but gambling on balls in play is not exactly what you want in the ninth. According to The Hardball Times Bullpen Book, Kolb has entered 6 tie games in the ninth in his career and blown 3 of them. A second key difference between Kolb and Smoltz is the amount of innings pitched. In 2004, Kolb threw 57 innings compared to Smoltz's 82, with a quarter of Smoltz's appearances coming in the eighth. Kolb just can't be used like this, so look for Kolb to be used as the main middle-reliever (like Alfonseca in 2004) by the end of the season.

The other potential closers are Romon Colon and Reitsma, who have higher strikeout rates than Kolb. Though Reitsma collapsed down the stretch and was quite unclutch in 2004, he pitched well for 80 innings, with a FIP ERA of 3.8 and a strikeout-to-walk ratio of 3:1. Kevin Gryboski is a poor-man's Kolb. His infuriating ability to walk batters without striking them out seems to be sufficiently counteracted by his ability to keep the ball in the yard. He has allowed only 5 home runs over 95 innings the past two seasons. Tom Martin is trying to play his way off the roster in Florida right now and hopefully will succeed. The Braves may go with Gabe White as their situational lefty. Some other possible contributors include Adam Bernero and Buddy Hernandez. The Braves are always creative with the bullpen, and someone will catch the Mazzone magic, we just don't know who it is yet.

4. When will we see Andy Marte and where will he play?

Player       Age  Class   AVG   OBP   SLG   OPS
Andy Marte    20     AA  .269  .364  .525   889
David Wright  20      A  .270  .369  .459   828

Now do you understand why Braves fans are excited? Marte was recently named the best hitting prospect by John Sickels and the top overall prospect by Baseball Prospectus. After tearing up Greenville despite a bum ankle, Marte proceeded to make a push for the big team by hitting .259/.369/.413 in the Dominican Winter League and putting up a 1.083 OPS in 24 ABs in the Grapefruit League. Sure it could be a tease for 2005, but there is no doubt that this guy will play in the big leagues soon. David Wright's ascension to the majors at age 21 has all Braves fans expecting Marte to do the same this year.

Marte will start the year in Richmond playing third base, which is a huge problem. Chipper plays third base, and no matter what any one else says, he is not going back to the outfield to risk injury. Marte is, by all written accounts, a solid defender. Ideally he will take over the third base job with Chipper shifting to first. This would be fine except this is where lefty infielder LaRoche needs play. The Braves considered shifting Marte to the outfield before spring training began, but decided to abandon the plan until they knew Marte was ready to make the jump. But his recent play led Cox to state that Marte may indeed get some time in the outfield in Richmond.

What is going to happen if Marte actually lives up to these grand expectations? Schuerholz has a problem on his hands, but it's a nice problem to have. There is a good chance that either Mondesi or Jordan will be a dud, which will create room in the outfield. I wouldn't be surprised if LaRoche, and not Marte, goes to the outfield. It's not a transition one wants to make in mid-season, but it's better setting things up in the spring for Marte to succeed only to watch him become the next Betemit. Having Chipper at first and Marte at third makes the most sense for the team in the long run. LaRoche could also be traded, but that could be complicated, especially if Adam gets off to a slow start. Look to see Marte in the Atlanta by June.

5. Will Johnny Estrada hit like he did in 2005?

There is a contingent of people who think that because Estrada couldn't hit several years ago, he shouldn't be able to hit now. Well, I'm not sure why it happened, but Estrada's turnaround appears to be for real. In 2003 he posted a .328/.393/.494 line in Richmond, and followed it up with .314/.378/.450 in Atlanta last year. He may have been slow to develop, but Estrada is a major league catcher now, so deal with it. However, all is not perfect for Mr. Estrada. My own projection system predicts that he will lose about 25 points on his batting average this year. Expect .290/.345/.445 from Johnny in 2005, which is below his 2004 but just fine for a catcher.

Click here to learn about THT's download subscriptions.

Quantifying the Market Size Advantage in MLB

JC Bradbury — 2004-05-28T03:00:15+00:00

Let’s face it; baseball is a business. Men and women who want to make money run it. To me this is not a problem, because many things that make me happy in life businesses provide. But the fact that baseball is a business has a powerful impact on explaining what type of baseball fans receive.

When consumers want steak, the grocer that fails to provide steak is going to bring in less income than if he had steak. Similarly, if fans want baseball with competitive balance, owners would be hurting themselves by not providing it.

I believe this is why MLB operates as it does despite the fact that larger cities may have a revenue-generating advantage over smaller markets. As long as the advantage is not too large, fans still receive the competitive balance they desire from baseball. And any prolonged under/over performance by specific clubs might be the result of factors other than market size.

Attempts to limit the big market advantage are not without risk. Revenue sharing, the most popular solution to the problem, while giving low revenue teams more cash also creates a disincentive for winning. Thus, proposals to minimize competitive imbalance must be crafted with caution.

In order to determine whether or not there exists a problem that needs correcting we must know the exact magnitude of the imbalance and how important this imbalance is to the current distribution of wins across teams of differing market sizes.

In this article I present a study on the effect of population differences inherent to the current geographic dispersal of teams on the variance of wins across teams. I find that market size did affect winning, but that the effect was too small to explain the gap between the good and bad teams of recent history.

It is no surprise that MLB has at least one team in all but two (Portland and Sacramento) of the top 26 Metropolitan Statistical Areas (MSAs) in the US. If you are going to generate fans by tying your team to a locality, locating in bigger localities will generate more fans and more revenue for owners.

Extending the logic, this means that bigger cities ought to produce more revenue for owners than smaller cities. In an open market for players, the best players gravitate towards the teams with higher salary offers. Because fans generally prefer to watch winning teams, owners seek the best players to bring in more revenue.

Players who generate wins command salaries commensurate with the extra revenues they generate for owners in terms of increasing fan interest. Teams in large cities have a greater pool of fans to enjoy wins; therefore, wins are more valuable to big market teams. While an extra win per season in Kansas City may increase yearly attendance by 10,000 fans, one more win for a New York team could generate a million more fans in a season.

It appears that big cities ought to have an inherent advantage over small cities in attracting good players; thus, big cities can translate this advantage into wins on the field by purchasing the better players. (Though this result is quite intuitive, I want to acknowledge that it is strongly supported in the academic economics literature. For example, see “An Economic Model of a Professional Sports League” by Mohamed El-Hodiri and James Quirk in Journal of Political Economy, 1971.)

For baseball fans this is a troubling situation. The joy of competition is watching players on the field exploit all of their abilities to win the game. The uncertainty of the outcome is part of the thrill of witnessing sports events. The indeterminacy of the game is what makes the game addictive and fun.

If certain teams have advantages over other teams solely due to the population of their fan bases, the indeterminacy of competition disappears. Big market teams will always have the best players, and small markets can rarely host winners due to the inequality of revenue sources for clubs. Baseball’s Blue Ribbon Panel in 2000 devoted a significant portion of its analysis to remedying this potential problem.

According to the panel, the owners feel that it is important for every team to have “at least periodic opportunities for success,” in order to keep maximum interest in the game. If some teams have inherent advantages due to their market, this “standard” may be in jeopardy. In order to determine the magnitude of the problem, I tried to measure how much city size influenced the ability of teams to win games. Big cities may have an advantage over small teams, but how many games is this advantage?

I used regression analysis to estimate the impact of population size on the average wins of teams from 1995-2002. The eight-year average ought to be sufficient to cancel out abnormally high or low win years for teams. I measured market size with two measures: 1) the population of all metropolitan statistical areas (MSAs) of MLB teams in 2000 and 2) the number of Nielsen rating households in the MSA.

Unfortunately, the sample size of 30 teams was smaller than I preferred, but there was not much I could do about it. City sizes are fairly stable so not much would have been gained by adding multiple observations of teams in different years. [I could quibble all day about my empirical methodology, but that would be distracting. I would be happy to handle any specific concerns via e-mail.]

For simplicity, I present the regression estimates graphically on two scatter-plots. First are the estimates for MSA population, the second is for Nielsen households.

Each point in the figures plots the average wins for each team and the size of the city for the sample. The regression lines estimate the relationship between wins and population size. The lines are upward sloping, indicating that larger cities were associated with winning more games than smaller cities. Both of these estimates are “statistically significant,” which means that it is very unlikely that these relationships were the result of random chance. I do not attempt to say why, other than that big cities may have had more revenue to purchase free agents, coaches, management, minor leagues, etc.

However, the story does not end here; the real question is “how much was the big city advantage over small cities?” The regression estimates were:

Wins = 76.43 + .0526 (MSA Population in 100,000s)
Wins = 75.68 + .168 (Nielsen Households in 100,000s)

For simplicity I translate these estimates into wins. The MSA population regression estimated that every 1.9 million residents generated one extra win per season. For illustration, the largest market (New York) is expected to win 10 more games than the smallest market (Milwaukee). The Nielsen regression estimated that 600,000 households resulted in an extra win per season. The difference between the smallest (Kansas City) and largest (New York) markets of this estimate was approximately 11 games.

So what did this advantage mean? During the sample the Yankees won an average of 30 games more than the Kansas City Royals and 24.5 games more than the Milwaukee Brewers. The difference in market size explained 30 to 40 percent of the difference in wins between the top and bottom markets.

Thus, the other 60 to 70 percent of the difference was due to other things such as luck, the ineptitude of the Royals and Brewers, and the skill of Yankees owners, managers, coaches, and players. Additionally, the estimates had low R-squares of about .12 indicating that the estimated equation explains about 12% of the variance of wins.

To further the analysis I subtracted the total number of wins due to MSA population from the Actual Wins total to create the Population-Adjusted Wins per season. Population-Adjusted Wins reflect the results of the team due to factors other than market size. The table below lists all of the teams in MLB with the mean Actual Wins per season from 1995-2002, along with several measures that modify wins per season by removing the influence of population. I did not make similar calculations using Nielsen households because its correlation with MSA population is high (r = .97) and would yield nearly identical estimates.

Team            Wins  Pop-Adj  Predicted  Residual  Post-Season
                        Wins     Wins      Wins     Appearances
Atlanta         97.5    95.3     78.6      18.9         8
Cleveland       90.7    89.2     77.9      12.7         6
Seattle         88.6    86.7     78.3      10.3         4
Arizona*        88.0    86.2     78.1       9.8         2
NY Yankees      95.5    84.3     87.5       7.9         8
Boston          86.8    83.8     79.4       7.3         3
Houston         86.2    83.7     78.8       7.3         4
St. Louis       83.2    81.8     77.8       5.4         4
San Francisco   85.2    81.5     80.1       5.1         3
Oakland         83.3    79.6     80.1       3.2         3
Cincinnati      80.5    79.4     77.4       3.0         1
San Diego       78.7    77.2     77.9       0.8         2
Texas           80.0    77.2     79.1       0.8         3
Los Angeles     85.0    76.3     85.0      -0.0         2
Colorado        77.5    76.1     77.7      -0.2         1
Chicago WS      80.8    76.0     81.2      -7.0         1
Toronto         77.3    74.9     78.9      -1.5         0
Baltimore       77.2    73.2     80.4      -3.1         2
NY Mets         83.0    71.8     87.5      -4.5         2
Florida         73.8    71.8     78.4      -4.6         1
Anaheim         80.3    71.7     85.0      -4.6         1
Minnesota       72.8    71.3     77.9      -5.1         1
Montreal        72.8    71.1     78.1      -5.3         0
Philadelphia    73.3    70.1     79.6      -6.3         0
Milwaukee       71.0    70.1     77.3      -6.3         0
Chicago Cubs    74.2    69.4     81.2      -0.3         1
Pittsburgh      70.0    68.7     77.6      -7.6         0
Kansas City     69.0    68.0     77.3      -8.3         0
Detroit         65.7    62.8     79.3     -13.5         0
Tampa Bay*      63.6    62.3     77.6     -14.0         0

*Totals for Arizona and Tampa Bay are for 1998-2002.

The Population-Adjusted Wins are useful for analyzing how much of a team’s success was due to factors other than market size. It is as if each team played in a locality of equal size, and only chance and the skill of the owners, managers, coaches, and players influenced the outcome.

With these metrics the Yankees could not win more games than other teams because of a population-generated revenue advantage. Even without their big-market advantage the Yankees were fifth in Population-Adjusted Wins. While the Yankees were second in Actual Wins, it is clear that any big market advantage was only a small part of the success of this organization over the sample period.

Predicted Wins are the wins predicted by teams based solely on population size, irrespective of team success over the sample period. Viewed with Residual Wins (the difference between Actual Wins and Predicted Wins) this tells how good teams were at over/under performing given their population size. The Yankees operated in a big market, but they did many other good things to attain success, winning eight games more than predicted by the population advantage.

Similarly, the small market Royals did many bad things that contributed to their failure leading the team to win eight games fewer than predicted. Seven of the eight clubs that never made the playoffs during the sample years were among the bottom eight teams in Population-Adjusted wins and Residual Wins, with Toronto being the lone exception.

In summary, market size has played a role in winning in the recent history of MLB, but the market size effect was not the main reason for the dismal performances of several small market clubs. It was the “other stuff,” including poor management and bad luck, which explained more of these clubs losing ways. These teams would have been bad even without the influence of market size.

I would like to end by addressing a few possible objections to the study. There are a few questions I expect readers to ask, so I will try to answer them in advance. If you have further questions or find my answers unsatisfactory, please let me know.

• What about localities with multiple teams?

I did consider this and ran regressions using specifications that halved the population in New York, Los Angeles, Chicago, and San Francisco/Oakland. I also ran a specification leaving population as is and included a dummy variable equal to one for dual-team cities. The effect on the regression estimates was very small and the fit of the models was slightly worse.

• Why not use team revenue instead of population size to measure market size?

Revenue is related not just to market size, but also to the quality of team management. The problem is determining whether the failure to generate revenue is predetermined by the geographic structure of the league or due to bad business decisions. Cleveland and Seattle bring in large amounts of revenue with small metropolitan areas, while the larger markets of Philadelphia and Detroit do not.

The quality of business decisions is the main cause of this disparity, which means the effect on competitive balance is not inherent to the structure of the league. New management, not revenue sharing, is the answer to these problems. Of course teams with higher revenues will spend more on winning than teams with low revenue. I think that is a good thing, because it encourages teams to seek out new revenue sources by satisfying consumer demand.

• What about using other control variables in addition to population?

I am not sure what I would gain, while giving up very precious degrees of freedom. I considered including a few other variables, but I ultimately decided they were not worth the effort of gathering. If you suggest and supply some data, I would be willing to run estimates on alternate specifications and discuss the results.

Click here to learn about THT's download subscriptions.