Prospects and imperfect informationby Victor Wang
May 27, 2008
“We have the luxury of having much more information about our own players in the minor leagues than we do about amateur players, and we take advantage of this information. Not only is detailed information captured for all minor league players (to varying degrees depending upon the level) but we have our staff provide information that we feel is essential for evaluation. All of that goes into our internal player evaluation system.”
—Jeff Lunhow, May 20, 2008
“McDougal’s the player Billy’s giving up. McDougal’s also been dogging it during workouts. He’s conveyed to the A’s minor league coaching staff a certain lack of commitment to the game. But these things the Cleveland Indians are required to learn the hard way.”
—Moneyball, pages 213-214
Jeff Lunhow expresses what is thought to be a common belief among farm directors and general managers. This belief is that farm directors and general managers have more information about their than they have on other teams' prospects or draft prospects. For example, a team might know if a certain prospect lacks a proper work ethic or if a player has been visiting McDonald's a little bit too much lately. These things are harder to find out if a prospect is in another system or about to be drafted. Teams like the Atlanta Braves have been notorious for trading highly regarded prospects who have gone on to do little in the majors.
So do teams really have an information advantage when trading their prospects? And, if so, does that mean we should lower a prospect’s rating if he is traded?
Here is what I did to see if teams are trading the “right” prospects. I examined Baseball America’s list of top 100 prospects from 1990-1999. From there, I recorded all the prospects who were traded in a year when they were on a top 100 list and noted what their rating was. Using data I collected on a top 100 prospect’s average production (http://www.hardballtimes.com/main/article/the-bright-side-of-losing-santana/), I used a prospect’s ranking to find his expected production. To do that, I looked at the expected WAB (Wins Above Bench, Win Shares Above Bench divided by 3) for each prospect group.
For example, if a hitter was ranked in the top 10 when he was traded, his expected production was 10.9 WAB. If a pitcher was ranked anywhere from 26-50 when he was traded, he had an expected production of 4.44 WAB. If a player was traded multiple times and ranked in different groups, I took the average of his expected production.
After that, I looked up what the prospect actually produced in his first six years in the majors. This is a prospect’s actual production. If teams that are trading a prospect have an information advantage, then we should see that the actual production is lower than the expected production. Note that some prospects played a few games for the original team before being traded. I included these stats with a player’s actual production. This will have a very minimal impact on the analysis, since a prospect who played in the majors and was rated as a prospect the next year must have had very few at-bats or innings pitched. Here are the results:
Number of Prospects: 71
Expected Production: .82 WAB/yr
Actual Production: .89 WAB/yr
Number of Position Players: 41
Expected Production: .98 WAB/yr
Actual Production: .97 WAB/yr
Number of Pitchers: 30
Expected Production: .61 WAB/yr
Actual Production: .78 WAB/yr
Prospects traded multiple times
Number of Players: 6
Expected Production: .76 WAB/yr
Actual Production: .31 WAB/yr
From this analysis we can see that if teams did have an information advantage for their own prospects, they weren’t using it well in the last decade. A traded prospect’s actual production was actually higher than his expected production. While a hitter’s actual production was basically the same as his expected production, pitching prospects overachieved. This suggests that teams aren’t using their additional information properly.
However, this conclusion would assume a couple of things. First, this assumes that teams had more than one prospect that were regarded about the same. It assumes that they could determine which prospect was better, and that the team they were trading to was indifferent between the prospects. Also, since we are dealing with top prospects, these who were traded may have been essential parts of the trade.
Considering that this study was done on traded prospects during the ‘90s, we must also acknowledge that teams were more willing to trade top prospects in this era. Another factor is that prospects are very difficult to project and teams with additional information may not be using that extra information correctly. One thing interesting to note is that prospects traded multiple times fall well short of their expected production (sorry, Gio Gonzalez).
However, we are dealing with somewhat small samples in this study, especially with the prospects who were traded multiple times. This may suggest that if a top prospect is traded multiple times, we should lower our expectations for that prospect, but I wouldn’t make any significant conclusions from a sample of six prospects.
However, I think we can conclude a couple of things from this study. One is that prospects are difficult to project, which shouldn’t be a surprise. However, because of this, teams that may know more about their prospects may be correct in their thinking when they trade a prospect only to see him blossom and develop into a solid or even star player. Also, since top prospects are crucial parts of a trade, a team’s information advantage might not show up for top prospects.
It might be better to look at non-top 100 prospects, who would be much more difficult to evaluate, or wait until we can evaluate traded prospects in this decade, when teams are more reluctant to trade top prospects. Despite this, I can think we can say that if a top prospect is traded, it does not represent any new information toward the evaluation of that player.
Victor Wang's work on OPS has been featured in SABR's By the Numbers magazine, and was the 2007 recipient of SABR's Jack Kavanagh Memorial Youth Baseball Research Award. He can be reached via email here.