What to make of pre-ranks?

It’s hard not to be influenced by pre-ranks. A novice owner may simply defer to authority. An owner use a pre-ranking to confirm a preference for one player over another. And even the savviest of owners may be taken aback if there is a large chasm between their opinion of a player and his pre-ranking.

This week I want to dive into the volatility and accuracy of Yahoo pre-ranks and see if there’s anything we can learn relating to keeping or drafting highly pre-ranked starters, closers, and position players. In most of my keeper leagues, we don’t declare keepers until relatively soon before the season begins. That means pre-ranks are up already and may be a tool one uses when making decisions.

How reliable are those rankings? Are they more reliable for batters, starters, or relievers? Here, I’ve decided to take a snapshot of Yahoo’s top 100 players coming into the season and what the current top 100 actually looks like. This is hardly scientific, as it is only one season, which isn’t even complete yet. But, let’s see if there are any clear trends that emerge.


Player TypePre-ranked top-100 (PRT100)Actual top 100 (AT100)PRT100, but not AT100, not PRT100, but AT100PRT100 vs. not PRT100 as % of AT100 Success rate
Batters7562352265/3553<
Starters1928112028/7242
Relievers6103730/7050

For 2007 and 2008, I can’t access the pre-ranks, but here’s the final composition of the top 100 for each year:

2007

Player TypePercentage of top 100
Batters58
Starters28
Relievers14

2008

Player TypePercentage of top 100
Batters69
Starters19
Relievers12

Here are some trends that emerge from the data:

  • The pre-ranks are not particularly accurate. Overall, only 51 of its projected top 100 actually finished in the top 100. Removing the top 25 (surest things) from the equation, would substantially drop the already underwhelming success rate of the pre-rank predictions. The highest pre-ranked pitcher to fail to rank in the top 100 for reasons independent of injury is Cole Hamels, who was ranked at 45. For batters, it’s Jimmy Rollins at 14. (Rollins could easily finish in the top 100 though—the surest bet is B.J. Upton, who was pre-ranked at 16).
  • Although I don’t have the pre-rankings for 2007 and 2008, I feel confident presuming that relief pitchers are most drastically under-represented in pre-rank top 100s. In 2007 and 2008 there were a total of six non-closer relievers ranked in the top 100. This tidbit underscores positions taken by me and Marco Fujimoto regarding the value of elite non-closer relievers.
  • The average composition of the top 100 over the past three seasons has been 63 percent batters, 25 percent starters, 12 percent relievers, an approximate ratio of 5:2:1.
  • In 2009, the pre-rank was about 10 percent more accurate at predicting actual top 100 batters than it was predicting pitchers, though that may be entirely mitigated by the sample sizes.

At some point, the pre-ranks start to become nearly meaningless. Next week, I’ll delve into the 51-100 segment of the list, where presumably the bulk of the variation occurs, and explore whether the pre-ranks are a viable tool at all once decisions start to get a little more difficult.

Let me also add two quick notes before I close. First, I chose to remove players who didn’t perform at top-100 level because of injury, being that injury is a constant every year that is more or less random depending on the particular player. Two, if anybody has information about the year-to-year volatility of fantasy production for pitchers vs. batters, that would make a great companion resource to this piece.


7 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Andrew
14 years ago

Good point on middle relievers. I feel the Matt Thorntons and Michael Wuertzes of the game still don’t get their due respect. In leagues with large benches, it’s a no-brainer to own a few of those types.

dan
14 years ago

Great stuff!  I’m participating in a Yahoo league this year that used an autodraft (participants are spread out all over the country). Trades are important in such leagues as you can end up w/ unbalanced rosters.  The guys in the league (most of whom I don’t know, personally) took the yahoo rankings as gospel, no matter how stupid they were.

Phil
14 years ago

I’ve been playing on yahoo since 2000, and every year before this one the O-rankings would actually change, kinda like the idea of if the draft was held today, here are the rankings.  It has been weird this season as the O-rankings haven’t changed at all. 

Where this has messed up is the Can’t Cut List, which includes for example Lance Berkman, who is not bad, but in a yearly league should be allowed to cut.

JTin
14 years ago

I think you’re missing a major point here – Pre-Ranks are typically built for drafting purposes, NOT as a prediction of which players will finish as the top scorers. Position scarcity comes into play.

A good example are Kickers in fantasy football. Although several kickers will no doubt finish the year in the top 100, you’ll never see them on a Pre-Rank list because it would be lunacy to draft a kicker in the first 10 rounds.

dan
14 years ago

I don’t see a need for a Can’t Cut List, at all.

Jeremy
14 years ago

I love pre-ranks. It gives me a good idea of where most of the people in my league would draft a player. Then I can find the good values to draft late. This year it was guys like Ethier and Sandoval (who I got) and Werth (who I didn’t) in the 12th-15th rounds. Last year, Josh Hamilton had a prerank of around 170. I didn’t think he’d be a top 10 player, but I knew he was worth drafting at least a couple rounds before that.

That’s how to use preranks…not for your own purposes of ranking players, but for deciding which players are good values based on when everyone else is likely to start thinking about drafting them.

Derek Ambrosino
14 years ago

Jtin,

I’m not sure that is accurate; I’ve never seen a mission statement for the pre-rank system, though your football point makes sense. That said, why would they use an O-rank take positional depth into account and then place that side-by-side with an actual rank that does not?

Quickly looking, this year’s pre-ranks were distributed as such:

19 MIs
25 CIs
31 OFs
(there’s presumably a bit of overlapping)

This would indicate that they may have made some attempt to represent all positions and take account for positional depth. Nonetheless, all players projected at this level are expected to produce substantial raw value, regardless of position.

I think Jeremy’s point is important about pre-ranks, they are often best used to judge which of two players that you are interested in at your current pick has a better chance of being around next round. Or, for keepers, which player you have a better chance of being able to re-draft once put back in the pool.

A final note here, I do not actually develop my own pre-ranked list. (I actually think that for any individual fantasy player it’s an efficient use of his/her “study time.” [goes to file that away in the “column ideas” folder]) So, this performance by Yahoo may actually have been pretty decent. I’m not very sure of what benchmark would be laudable. Intuitively, this seemed like a pretty poor performance, especially considering it’s pretty hard to flub the top quartile of the sample size, barring injury.