Intuition vs. Quants

Before the season started I announced that I’d be playing in the inaugural season of the CardRunners Expert League. Thus far the league has been loads of fun with some great theoretical discussions happening over on the league’s blog. I’d highly recommend picking up its RSS feed. If the first week has been any indication, this will be a great sounding board for fantasy baseball ideas, and I’ll be posting over there frequently.

The big debate thus far has involved Rotowire’s Chris Liss and poker pros Eric Kesselman and Bill Phipps. The debate has unofficially been titled “Intuition vs. Quants,” with Liss advocating “intuition” and the poker guys on the “quants” side. It’s essentially a debate over the merits of quantifying everything and using projections to run your team as opposed to taking a more subjective (though no less rigorous) approach and researching players without needing a precise projection for each one.

I love these kinds of discussions, and I’m very pleased to see it being discussed so intelligently and openly here. A lot has been written so far, but it’s well worth catching up on. I decided to jump in today, and you can find my views on the subject here.

To trace the history of the discussion (if you have some free time, it makes for a terrific read), follow these links (in this order):

  1. Liss and Jeff Erickson
  2. Kesselman
  3. Phipps
  4. Liss
  5. Kesselman
  6. Phipps
  7. Phipps
  8. Liss
  9. Robert Dixon (Wall Street options trader)
  10. Liss
  11. Carty

If you have your own thoughts on the matter, feel free to comment here or over at the CardRunners league blog.

Print Friendly
 Share on Facebook0Tweet about this on Twitter0Share on Google+0Share on Reddit0Email this to someone
« Previous: Twisting Oliver: Gettin’ while the gettin’s good
Next: NL Waiver Wire: Week 2 »

Comments

  1. Mike Podhorzer said...

    Fantastic discussion, thanks for bringing it to our attention D-Cart. We need more of this type of stuff! So sick of reading the same articles about pitch and ditch and buy low and sell high guys after only 2 weeks of the season.

    For the record, I lean toward your side and the quants/agnostic way. It should be very interesting to see how the league goes, with the caveat that one season’s league results are nowhere near a good sample to mean much.

  2. Peter Kreutzer said...

    The issues here are 1) whether having an exact measure of inaccurate predictions is better than having a measured understanding of a player’s “value” and the market’s evaluation of him, and 2) whether the exact measure of inaccurate predictions (especially when these are massaged to include multiple risk scenarios and recalculated) is really a scientific approach or subject to the same gut-level manipulations that Liss more readily admits to.

    I’m pretty sure in re No. 1 that there are advantages and disadvantages to both. Having exact values might restrict your ability to adjust when necessary, while using a market evaluation method might cause you to misread a player’s intrinsic value.

    As for No. 2, I don’t think it’s possible to argue that many of the assumptions in the quant’s model aren’t, of necessity, subjective. How that degree of subjectivity measures up to that of the market evaluation is hard to measure.

    Edwin and Mike’s points about a suitable test for the two sides are important, too. This discussion might go on for awhile. I’m looking forward to hearing Mike’s thoughts about projections.

  3. Edwin said...

    I also added this as a comment to your CardRunners post, Derek.

    Please forgive me, a stranger stepping into a debate as developed as this. Here is my take, as an outsider:

    1) Projections:

    Everyone uses a model. Liss’s “intuition” is a model, albeit undefined. There is so much variance in fantasy baseball that the advantages of a defined model are not necessarily visible inside of one fantasy baseball season. While Liss’s supreme knowledge of the game, feel for breakout players and playing time situations, etc., all give him an advantage over the so-called “quants,” Liss would, in the long run, be an even stronger player with a tested and defined model that incorporated his knowledge into projection making. As Derek says and the traders agree, accuracy is advantage, more discernible in the long run. When you are trading thousands of shares per second, or whatever you crazy finance whacks do, the 0.5% (or 0.00000000005%) predictive advantage your model provides is all the more evident (thank you Thesaurus). 

    2) Valuing Projections

    Art Mcgee has done good work on the valuation of projections, and I assume our “quant” friends have as well, likely going into even greater depth. Valuing projections is not about accuracy. It’s about doing some math. Given league settings and final standings, we should be able to know exactly how much Andrus’s 2009 season was worth. If Manager X has the player valued correctly in the context of the league, and Manager Y is off by $5, X gains an advantage. Now, Liss’s point seems to be that this hypothetical $5 advantage is negligible because of the inherent fallibility of projection systems. While projection systems are fallible (25%-30%, say), if the valuation of my fallible system is also flawed, I become even less accurate and drop down into the 30-35% fallibility range. While there is a chance one’s flawed valuation of one’s projection turns out to be closer to the actual outcome, there is just as great a chance that it will be even further away, thus making the proper valuation of projections no less important. 

    3) Conclusion

    Liss’s intuition is a model. When he goes to $27 for Jeter, he is making calculations, based on the environment of the auction, position scarcity, his own budget, his own valuation of Jeter, etc. In the long run, when the “quants” define models (and test them) for all of the calculations Liss is making (not just in the draft, but throughout the season), they will win. We will have to look at a hundred thousand fantasy baseball seasons and the average place finish for each camp, not one or three. There are so many variables in the equation of a fantasy baseball season, however, that experience still wears down a “quant” over a full season, despite the inaccuracy in these calculations of experience. “Quants” might not have enough models yet, but theoretically, given equal knowledge, defined and tested mathematical systems will give “quants” the advantage over time.

  4. Edwin said...

    Well put, Peter.

    The sense I get is that if one is able to define the calculations one is making with mathematical expressions, one will be able to more efficiently and effectively evaluate, adjust, and improve said calculations.

    It makes me think of having to show my work back in math or chemistry class in high school. I could get the right answer with a few scribbles or by doing it in my head most of the time, but by going through step by step on paper, I could see if I had made any mistakes while going through the equation, and I could also go back and see where I went wrong if I didn’t catch it at the time.

    By translating one’s thought processes into defined mathematic expressions, one generates a historical record that can be reconsidered efficiently.

    What is interesting to consider is the scouting aspect of player evaluation. How do we assign numbers to how quick a players hands are, or how still his head is, and how his performance is affected? This is where I feel Liss has a valid point, and I would be curious to see what the “quants” think about applying models to these physical/visual cues, before the numbers happen.

  5. Mike Podhorzer said...

    “What is interesting to consider is the scouting aspect of player evaluation. How do we assign numbers to how quick a players hands are, or how still his head is, and how his performance is affected? This is where I feel Liss has a valid point, and I would be curious to see what the “quants” think about applying models to these physical/visual cues, before the numbers happen.”

    Edwin, why can’t these scouting opinions be just another set of data points that get thrown into a projection? If you scout Conor Jackson and note that his build, or whatever else, portend a potential increase in power, despite his historical performance that would suggest otherwise, why can’t this opinion lead to an increased isolated slugging percentage and/or HR/FB ratio projection?

    I honestly do not even understand the other side of the debate. The accuracy of the projections themselves are completely irrelevant. All else equal, of course placing a precise computer model-derived dollar value onto a projected stat line is going to give you an advantage! Am I missing something here?

  6. Edwin said...

    Hi Mike! What’s up?

    “All else equal, of course placing a precise computer model-derived dollar value onto a projected stat line is going to give you an advantage! Am I missing something here?”

    No, not here, I don’t think. I am not talking about valuing projections, but generating/updating the projections themselves. The Quants win every time in valuing projections, IMO. My apologies if we’ve misunderstood each other; the nuanced semantics of this discussion coupled with my low IQ do not necessarily engender clarity!

    As for generating/updating Conor Jackson’s projection based on a scouting opinion, I think it is possible to transfer an observation about Jackson’s swing into a data point. I’m just not quite sure how. I suppose what I should have said is that this is where Liss has an advantage, not necessarily where his argument succeeds or fails. It would be possible to view hundreds of hours of at bats, find correlations between hand quickness and results, and then apply them to Conor Jackson’s projection after viewing a series of his at bats. Quite interesting to think about, actually.

    In the long run, math wins, in my opinion. In the short run, however, it will have a difficult time relating the physical aspects of the game (that happen before the outcomes) into actionable models. The efficacy/reliability of scouting reports is debatable, to be sure, but these reports seem to be highly subjective and thus difficult to model effectively. 

    But yes, I agree that ultimately it is possible to model pretty much anything with conditions and a result. And certainly the initial models will reflect highly subjective interpretations of the relationship between these conditions and results, only to become increasingly objective over long periods of testing. Long live the scientific method!

    In the here and now, however, I would want to have models take care of all of my initial projections and projection valuations, and then perhaps tweak these projections a bit with some scouting observations. In essence, then, I suppose I am similar to Derek, in that I want statistical analysis and microeconomics to be doing most of my evaluation, with a few dashes of “genius” tossed in because I think it might give me a slight edge over those only doing number-crunching.

    Was that blabber at all provocative? I think I’ve gone cross-eyed!

  7. Peter Kreutzer said...

    Mike: “All else equal, of course placing a precise computer model-derived dollar value onto a projected stat line is going to give you an advantage! Am I missing something here?”

    The issue here, Mike, as far as I’m concerned, is the “all else equal” part. Sure, if we use the same projections and base our bid values on draft day on the prices derived from those projections, and your model for deriving dollar values is better than mine, you’re going to win.

    But where I have a problem is that I don’t think stat projections are the best, most reliable and accurate inputs for future dollar values. And in my experience I haven’t seen any evidence the are, in large part because of the myriad compromises one makes while makes while a set of projections. For instance:

    Do you project Jason Heyward for 150 AB with a .225 BA, .325 SA (failure mode), or 375 AB with a .275 BA and .450 SA (he struggles, gets sent down, but returns and is pretty good mode), or 550 AB with a .310 BA and .575 SA (he’s god mode)? What chance does he have of falling into each category, or somewhere between each? Depending on where he falls in each, his fate effects (and vice versa) the fate of Melky Cabrera, Matt Diaz, Jordan Schafer and others.

    I don’t see how deriving a dollar value from one or two or a compromise of potential prospective scenarios (otherwise known as projections) is more helpful than looking at the player’s potential, looking at the pool, looking at your team and saying that this price for this guy is worth the risk.

    I’m in no way opposed to using math and what we know to help us project and value the future, but I think the inputs of projections are detrimental to proper bid pricing and budgeting on draft day. I hope that gives you just a glimmer of what the other side of the debate is.

  8. Mike Podhorzer said...

    Peter, that definitely makes more sense and was a good explanation. However, I would argue that again, one could slap odds on every scenario, like PECOTA does, and come up with a weighted average projection, in which you would then calculate a value for. So although we aren’t sure of exactly what Heyward will do, we have to have some sort of projection in our head to determine how much we should bid on him.

    If my weighted average led to your 375 AB projection, should I bid more than my calculated value, due to his upside? I would say no, because then I am cutting into my profit potential if he did hit that upside, while increasing my downside risk because he is still a rookie with a short minor league track record.

    So my argument basically continues to be that although it is impossible to be completely accurate with projections and some players are more difficult than others to project, one could still take everything into account, all the various scenarios, upside/downside, etc, and formula a weighted average projection. That stat line has to be in your head at the very least when you determine how much to bid. And using a computer model to help you translate that weighted average stat line into a dollar value is better than hoping your brain alone could do so.

  9. eric kesselman said...

    Mike, I think the quants from CR are with you 100%. This is pretty much exactly what they’ve been advocating, and I think some of us are also having trouble seeing the other side of the argument. Would love for you to weigh in sometime over there too.

    Peter, I really don’t understand where you’re coming from when you say stuff like “I don’t see how deriving a dollar value from one or two or a compromise of potential prospective scenarios (otherwise known as projections) is more helpful than looking at the player’s potential, looking at the pool, looking at your team and saying that this price for this guy is worth the risk.”

    It really seems this is EXACTLY the same process the quants advocate, only the quants are (unsurpisingly) doing it in a quantitative manner whereas what you wrote seems to be this kind of Liss-ian argument where you just “look” at stuff and then “say” a bid. What’s the problem being a bit more quantitative about that same approach? Obviously the guesses for both the quality of the outcome (breakout, fail, etc) as well as their likelihood are subjective guesses. Why shouldnt we TRY to make some good guesses here as opposed to just winging it?

  10. Peter Kreutzer said...

    Mike and Eric,

    I’m clearly not explaining myself properly, so let me try another example:

    I make projections. These are based on formulas I’ve derived and tested over the years. I start out with a baseline calculated from the formula, and then I have tweaks that I apply manually to account for context changes, good or bad luck in the past, etc. I then scale these numbers for projected playing time.

    Showing what happens with pitchers will explain why I don’t think you should price off of projections. Pitchers are unreliable, except for the best pitchers, who are unreliable but don’t have any track record of it yet. So what happens:

    I project Roy Halladay to throw 225 innings like those he’s thrown in the past. This means his projection will be worth about $40.  But we all know that he’s going to cost $30. Do I tone down the projection, making it not look like a Roy Halladay line, though there’s no reason to? Or do I write down the price, in which case it isn’t based on the projection?

    And what do I do with scores of mid-tier starters whose projections are worth $30 if they last for 200 innings, but we know that many of them won’t? I can project all of them for 150 innings each, but those don’t really look like projections. It’s creating a risk pool, with lots of arms that look alike. We know a handful will emerge up, and two handfuls will crash and burn, but we have no idea which ones. So either all their projections look alike, or you pick the ones you think likely to succeed. If you do that, you end up with a $30 price tag on a $17 pitcher, which doesn’t help anyone at all.

    I hope this better illustrates the problem. A projection is an evaluation of what a player is most likely to do this year, but we know from experience it will be wildly inaccurate, and if you price from it you will have to make scores of adjustments to your prices to reflect the market and common sense.

    I’m working on a post for the Cardrunners blog that explains how I derive my prices in order to show that they are based on data and numbers, but not the translation of projected statlines. The short answer is they’re a market evaluation based on what a player has historically earned from year to year and what leagues have paid for him in the recent past, filtered through their age and injury history. I may project Ryan Sweeney to hit 16 homers this year, I may have good reason to project him to hit 16 homers this year, but I certainly shouldn’t pay for that. My method is an attempt to keep things real. I can say that the room is going to pay $10 to $14 for Sweeney. I choose where in that range I’m happy to drop out. Since Sweeney earned $14 last year as a 24 year old, and I think he’s likely to improve, that’s a good price for him for me. 

    It seems to me as I argue this point about projections that either I’m making a hash of the explanation, or I’m really not understanding what Mike’s process is, for instance. I think I take everything into account in my process, in as balanced and unbiased way as possible, while when I price using projections my prices reflect the input of a flawed system like player projections, and the outcome is equally flawed.

    I hope this helps.

  11. Natt Ringer said...

    Great discussion, but I also confess that I don’t understand Peter’s side.
    It seems to me that you have a pretty crummy model if it can not account for the volatility of the players.  Each player should have a unique distribution of possible values.  In the Halladay example, if the “normal” Halladay season is worth 40,  but 20 percent of the time he gets hurt or blows up and is worth 5, doesn’t that make him worth 33 ?

    These guys are options traders, right?  In the option pricing model, the most important inputs involve estimating the volatility and shape of the distribution.  Why would pricing players be any different?

  12. Peter Kreutzer said...

    Natt:

    Exactly. Pricing players is all about incorporating the risk.

    Projecting players is about describing what they are going to do (though I’ll concede, as you’ll below, that you can make projections that include risk). Let’s say I have five pitchers who are going to earn, to make it simple, $30, and each has a one in five chance of totally crapping out.

    The bid price on each of them is $24. That’s easy.

    But the fact of the matter is that four of them are going to earn $30 and one of them is going to earn $0, so if you’re making projections it looks more wrong to give each of them a $24 projection. You will test better in correlations of projections with this style of projection, but your users/readers/own heart will complain and say that you are a chicken or worse.

    My point is that a projection that looks like what your pitcher might do is going to give you a wrong price, and a projection that incorporates the broad random risk of playing is going to collapse the whole field down into the middle and not tell you anything specific about the player.

    Both have their uses, but I don’t see any advantage to using either for generating bid prices, and in fact think the projections can be more misleading than useful. That said, I do convert my projections to prices as a check to the prices I derive from the cost and earnings scans described above. Sometimes I realize a projection is out of whack, and sometimes I realize I’ve misgauged a player’s actual value, and I make adjustments. I do not incorporate the risk evaluation into my projections for this purpose.

    I got into this discussion in the first place because I thought the quants thought they were getting more precision in their pricing from the projections than they were, though the discussion has been sidetracked enough times by my inexactitude and by side issues that I still am not sure that’s true. The more I hear the more it sounds it isn’t. In any case, I have no problem with a sophisticated pricing model, but the only projection it makes sense to base that on is the risk averse collapsed set of projections. These are the one’s that are most accurate.

    I doubt that a model that buys in the middle and lower pricing tiers is going to beat a room full of sophisticated players who can also buy off the top as well as the middle and bottom, at last not very often, because the other teams will buy the top pitchers (who generate the most stats) and most will not get hurt. So, the team pricing off the averse model is going to have to buy some high priced players, too, or risk being the runner up over and over.

    But once they decide to buy high, the only way to buy the top players is to make a decision about which is going to stay healthy and effective, effectively stripping the risk aversion out of the projection. Once you do that, why work from the projection in the first place?

    What matters is the price the room sets for the player and how you feel about that (not as a “feeling” but as a result of your analysis of risk versus upside). There are better ways to decide about it than projecting statines and putting a price on them.

    Am I getting closer?

  13. KY said...

    Old post but yes, that’s closer.  Basically you are arguing that the league winner will have to buy players at above the price a regressed/risk model will price players at, and then have them deliver even more value then that, in order to win.  That a model can not buy a team that can take first.

    I don’t agree, but I think it explains your point best.  I don’t agree because of things you say above, “a projection that incorporates the broad random risk of playing is going to collapse the whole field down into the middle”.  Yes it will, but it will still divide the league dollars available to spend into the collapsed pool. 

    For example, say we have 2 teams and 4 of the $30 pitchers you mention above.  One of them will flop and produce 0.  We have $120 in the pool to spend on these players among some teams.  Said risk model will say they are all worth $24.  If you buy one of them for $28, I’m going to get one of them for $20.  Leaving me more then you to buy other players with.  We already agreed they all have an equal chance of producing $30.  So why would I want to be the one who paid more then $24 for that opportunity?

  14. Peter Kreutzer said...

    My point wasn’t that you should pay $28 for the pitchers, it was that you should pay $24. My point was that if you project each of the pitchers to earn $24 (make their numbers worth $24) you’re going to be “wrong” on all four of them.

    If you project them to be worth $30 each, you’re going to be “right” on three of them, and way wrong on the fourth.  BUT, if you price them based on the projection you’re going to overpay for the ones you buy.

    Which brings me back to my original point, which is that the projected statline is a bad tool for setting a price, which is better determined by historical earnings and a read of the market.

  15. eric kesselman said...

    I don’t understand how your third paragraph follows from the previous two.

    You agree you ‘should pay $24’, which is a price derived from our projections after accounting for volatility.

    That seems to be the way we should arrive at prices, and you seem to agree.

    Then in the third paragraph you conclude that we should price it off historical earnings and a read of the market. How did you get there?

  16. KY said...

    Asked another way, how could I better spend the $120 on those 4 pitchers if I have no idea which of them will be the ones to earn 30 and which will be the 0?  Using another method then risk or whatever.

  17. Peter Kreutzer said...

    Eric: The projections are either right by being wrong (adjusted to $24) or wrong by being right (left at $30 for all four). They aren’t of any real help informationally. You’re either manipulating them to align with your evaluation, or you’re ignoring them. What matters is the price you give to the player, how that aligns with the market and with his potential. The projection can be a flag that something’s out of whack. I like the way Bill Phipps uses them to tell him whether his thinking is askew or not, but the actual projection doesn’t help you price.

    KY: If your price on all four players is really $30, and so you adjust them down to $24 to recognize the risk, you won’t get any of them. Other teams in your league will ignore the risk and grab them all for about $30 each. One team will be crushed, and the other three will do just fine. You don’t have to worry about it.

    In our mano a mano hypothetical, if we really both agree that all four pitchers are of equal talent and risk, there isn’t anything to do at all. Where it gets interesting is when we test if we really agree that all four are equal, by bidding. If one is really superior, it would make sense to bid $119 for him. If one is really inferior, it would make sense to let the other guy have him for a buck.

  18. KY said...

    Yeah I respectfully disagree with that.  “you won’t get any of them.”  Absolutely not.  I have to get some of them.  There are $120 in the pool and 4 players to spend it on.  This is a limited example, but in a full league there is a pool of players and a limited number of dollars to spend.  If 3 people spend $119 on those 3 of those pitchers, all other players remaining equal, I will get the last pitcher for $1.  And that last pitcher has an equal chance of producing a $30 season.  Because a league is a fixed system, you can not spend more on some players without spending less on others.

    I’m not sure about the mano e mano part because I was not proposing that any team had 120 to spend individually or that there were not other players to buy.

  19. eric kesselman said...

    I don’t see how you can say that knowing a player has an 80% chance of producing a 30$ season is of no informational help.

    It really feels a lot like Chris Pikula’s comment on the CR comments that your complaint is that an estimate of 3.5 for a six sided die roll is useless because the die just always fails to roll 3.5.

  20. Peter Kreutzer said...

    Maybe I misunderstood. What I meant is that in a league with $3120 chasing 108 pitchers and 168 hitters, the $24 difference between the four pitchers being priced at $30 and $24 is meaningless. Other teams will spend the $30, and you’ll be left getting other pitchers.

    There is plenty of money there to chase, and lots to skimp on later.

  21. Peter Kreutzer said...

    Eric:

    When you make projections you have to make certain assumptions. Let’s take the example of AB.

    In the preceding three years, a player has 590, 610, and 600 AB. How many AB will you project him for?

    The right answer is 540. That incorporates the risk that he might get hurt. But nine times out of 10 he’s going to have 600 AB.

    If you derive his price from 540 AB you’re going to get a much different price than if you derive it from 600 AB. So, would you rather be 90 percent right 10 times in a row or 100 percent right 9 times out of 10 (and totally mistaken the other time)?

    Settling on a price for future performance is based on weighing the difference between these two scenarios. You figure out what you’re willing to pay for the player, based on your belief that he’s going to succeed or fail THIS year. This is based on what he’s done in the past, your analysis of how he might improve in the future, and your assessment of your league and how it bids for guys like this guy.

    You can then jigger your projection to meet this price. But that isn’t the same thing as basing your prices on the projection. (And I’m not saying that the price isn’t a projection. It most definitely is. I’m saying that basing your draft prices on your projected stats is wrong because it will get you worse results than doing it a different way.)

  22. eric kesselman said...

    Let’s play a different game as an analogy.

    The game goes like this:

    You pay me X dollars to play.

    I roll a 6 sided die. Whatever value I roll, I pay you that value in dollars. However, twenty percent of the time, I don’t roll the die and just run away with your money (I’m a former lawyer remember)

    What is your projection for the die roll?

    How much would you pay to play the game?

    The average roll of a die is 3.5. So 80% of the time, I roll an average of 3.5 and 20% I effectively roll a zero. Now we can quibble about whether the ‘projection’ should be 3.5 or 2.8, or whether it should be 3.5 ‘if the game runs’ but the bottom line is that we know the weighted average of this is 2.8, so I can profitably pay less than $2.80 to play this game.

    I suspect we all agree on this. I don’t see how this isn’t incredibly informative about what I should pay.

    I don’t think anyone is going to argue that it is suddenly about ‘my belief’ that the next die roll will be good or not, nor about past earnings. Regardless of the history of past rolls we still know the 20% chance of ruin is there and account for it, even if it hasn’t happened.

    Why is fantasy baseball any different?

  23. Peter Kreutzer said...

    Well, that’s the first step. You’ve described the pricing of one player, without discussing the way the projection for a player changes because of the risk. That’s because your example deals only in percentages, not in cumulative effect.

    I suppose if you do this for 276 players divided among 12 teams spending $3120 between them, and you give each “player” 600 PA or 200 IP, and you have a drop dead effect, that means any “player” could shut down totally—or change its odds randomly because of a muscle pull or a sick child or some other random factor—at any time, you’d get a little closer to modeling the issues with making prices from projections.

  24. KY said...

    I think Peter’s point will apply the more shallow the league is.  Especially one with a bench.  If I am in a head to head league I basically care about getting the end of the season with a team full of stars. 

    In a 14 hitter 9 pitcher 12 team NL only league with no bench it will be less of a point.  I will have injured players if I draft 25 players.  Spreading out that risk has value. 

    I also do not agree that I if I am outbid for those 4 pitchers it will be a bad thing.  The dollars are divided into the talent.  If someone pays more dollars that the league rate of dollars to talent that leaves less dollars to spend on the un-drafted talent, those who did not do so, will be able to draft more overall talent with their remaining dollars then those who spent more then $24.

  25. Peter Kreutzer said...

    My point has to do with a particular procedural point. Even if you make the best projections in the world, they are a poor model for the prices you pay for players on draft day. This isn’t because the projections are wrong, but because they measure a different thing than a price does. That was my only point.

    As for getting outbid for the four pitchers being a good or bad thing, I think on the math of it in the small sense, you’re right, though I’m not so sold on the efficiency of the market. But in a game with one team versus multiple opponents, is a strategy with less upside and downside risk better for winning than a riskier strategy that has a higher upside and more downside?

    I’m just asking.

  26. Derek Ambrosino said...

    KY,

    …that presumes there are other pitchers worthy of spending that money on (you could say you’d just spend that money on hitters, but you still need pitching stats to win).

    I think part of Peter’s argument, and he articulated in a few other places, is that by fully accounting for risk, you bow out of the competition for the truly elite (potential) producers. Now, if you can maximize the return on your investments closer to the middle of the talent pool, you may be able to close the gap on those who bought stars and didn’t get burnt (and you’re likely ahead of those who did and got burnt).

    But… a)you can still get burnt on your more moderate investments; b) even a moderate return on your investment at the middle of the talent pool won’t beat, in absolute production, an equal return on a high-ticket item and; c)if you wind up with surplus dollars for those stats, you’ll just wind up overpaying for guys anyway, just not the top guys.

    Perhaps a glimpse into real baseball and pricing is of some worth here. Is A-Rod’s contract of 25M+ per season a good price? Well, not on a dollars per win share basis. But, you also have to consider the fact that only a very, very few players can produce the amount on win shares he can. So, when you have a fixed roster of producers, not every unit of production commands the same cost. You pay a premium for those last 6 homers and those last 12 RBI, that only two or three other guys will produce. The fact that Josh Willingham hit more homers per dollar is immaterial, because if I’m smart I’m going to have a bunch of those guys on the cheap too!

  27. KY said...

    Sorry, I don’t agree with that.  Its a large pool of pitchers and when you divide your dollars you do so by assigning some of them to pitchers.  If your team ratio matches what you set for the league, you will have pitching talent available at a better value if someone ahead of you pays more then you would for some of the talent pool.

    I don’t agree with the second part because you are not behind anyone by staying away from stars.  If I buy 4 middle weights and you buy two stars and two scrubs we are thus far even.  Now if you say but I can spend the season improving my scrubs and i can’t, then a) I presume that my team still has some scrubs even if I bought 4 middle weights and b) i think I’ve written to you before that replacement level should be set at draft replacement plus in season replacement.  In effect, players that I can replace in season are also worth $1, even if there is no one better at the time of the auction.  This pushes up the value of players who make it above the auction replacement + waiver wire replacement line.  Its effectively doing what you describe and rewarding players that can truly differentiate themselves.  I’m wondering if anyone has any further thoughts on that.

    On getting burnt, I agree.  I can see this as an argument to buy more expensive players.  If the chance of injury for all players were equal, I would have a potential for a higher total score and some of my players lose nothing when replaced from the waiver wire if I have stars and scrubs.

    I don’t think the real baseball argument applies because the $ pool is not fixed.  Would A-Rod’s 25 mill make sense in a salary cap world would be the question?  Perhaps given the above.

  28. Peter Kreutzer said...

    I’m totally agnostic on the question of Stars and Scrubs versus a spread the risk strategy. Both have advantages and both have disadvantages. The ROI work that Alex Patton for many years and Mike Gianella more recently have done, show that buying players in the midrange and at the top end is equally efficient. And buying on the low end is really a matter of luck, though I think we all think we have a special talent for it.

    But I think KY is overstating the case when he says that “you will have pitching talent available at a better value if someone ahead of you pays more than you would for some of the talent pool.” That’s true insofar as everyone is playing the same game with the same prices, but there are many strategies employed that disrupt any single set of prices.

    For instance, if I chuck saves and redistribute the saves money to the other categories, I end up with different prices. The same is true for someone who bails on steals, or someone who dumps BA. Or someone who chucks HR and RBI. All of these strategies are intended to beat someone who plays for the middle. If everyone is fighting over the same 10 pieces of pie, there’s a good chance each team is going to get a smaller piece of each pie, and there’s going to be a clump of teams in the middle. And a team that dumps a category or two can sneak in using a totally different set of prices than the one KY is talking about.

    So while a dollar wasted is a help to the other teams, it’s hard to tell which dollars are actually wasted. We’re also nowhere near efficient enough to make someone pay for adventuring a bid a few dollars higher than our price list says. There’s just too much flexibility in the game, and too many $1 lottery tickets that give a team with better players (whether stars and scrubs or of spread risk makeup) a big advantage. And, it’s important to remember, that you’re playing against 11 or 12 of them.

    As to the in season replacement value added to the baseline of a player, I can see this having great potential in shallow leagues. In AL and NL only leagues, the only replacement players of note are traded in from the other league or promoted from the minors. They’re more of the lottery variety than a replacement player, so I don’t think it has much application there.

  29. KY said...

    Yeah, I’m sorry, but I don’t agree with that either.  You just said that the problem with drafting a lot of middle weights because of watered down projections was that you don’t get any stars.  Drafting expensive stars necessitates drafting corresponding cheap scrubs since you have a limited supply of money.  So I don’t see how they can be divided.  If you have a talent for finding good scrubs that just means your league is bad at it relative to you and failed to price cheaper players properly. 

    On the punting, I don’t understand why that wouldn’t play to your advantage.  If a team punts a category then they are going to spend more money on the categories they did not punt making the punted categories cheaper for those who did not punt.

  30. KY said...

    “I said drafting from projected stat-lines gives you less precision about who and what you draft than setting prices based on the market.” 
    I can see why one might think a large market is more accurate then a projection but in an individual league where the market is 12 people I don’t see why one would think that.  If you buy based on the market, how can you beat the market except via luck?  Can we agree that auction leagues are won by paying the same price as other teams ($260) for more total production?

    So is the term intuition just semantics?  If I change my projections based on what I know, that is not intuition, that’s just a formula that happens to reside in my head.

    I misunderstood the scrubs point, thank you.

    I think if a team punts a category it makes it easier for other teams to gain more then one point in that category, not just the team that punted.

    I don’t agree that if they draft off a different price list those bargains will not come.  If a model has accounted for all the dollars in the league among all the players in the league, then without fail every time a player goes for more then that model lists them at another player will go for less then that model was going to pay.

    Yes winning steals by 40 can become a problem, but most people are able to trade surpluses without losing much value.

    If Liss’ strategy wins then his team paid the same amount of money as other teams for more production in that particular league.  For the most part, the only way to do that is to know better how players will perform.  There are other factors but isn’t that the main reason for winning besides luck?

  31. peter kreutzer said...

    What I’m saying is that projections, the statlines the are presented from them, are different than prices. If I ask you to project what Roy Halladay is going to do, you’re going to say he’s going to throw 225 innings of 2.96 ball, with 17 wins, 205 strikeouts and a whip of 1.13. I’ll ask you what you’re going to pay for him, you’re going to say $32. I’m going to tell you your projection is worth $40. What are you going to do?

    Your choices are either to raise the price, which would be silly, since Halladay is a $32 to $35 pitcher. That’s what he costs. Or you’re going to reduce his projection.

    But why would you do that? The numbers above are what Halladay has produced each of the last two years and there are no signs he’s hurt or aging. In fact, he’s moving from the toughest division in baseball to an easier league. Are you going to say he’s going to pitch 165 innings at the above rates instead of 225?

    Are you going to say his ERA is going up to 3.50? His WHIP to 1.25? That he’s only going to win 14 games? That’s what they were when he was hurt in 2007, but he isn’t hurt now?

    I just don’t see how you get meaningful information out of whatever projected statline you give Halladay that helps you determine his price.  We know his price without the statline, his price has the risk that he’s going to get hurt again built in, and our only decision is to decide whether we want to pay at the high end or not of his possible range, if we have to.

    To buy or not is a reasonable choice. To change Halladay’s obvious projection because he might randomly get hurt seems as nonsensical as paying for the full projection you hope for (that’s the one “you” offered up above, that was worth $40) if he doesn’t get hurt.

  32. KY said...

    “…you’re going to say $32. I’m going to tell you your projection is worth $40. What are you going to do?”  You lost me here.  Why are you telling he is worth more then 32?  Why would I change his projections exactly?  Why can’t I just “not buy him” if his price goes above 32?

    Since you used an NL player I can say that I had Halladay down for 2.9 220IP 16.5W 200Ks which in my league was $38.50 by my calculation.  Seems I should have given him another win or so, but I digress.

    “We know his price without the statline”  I think you are saying we know his market price.  But his market price is not the same thing as the value of the stats he is likely to produce in a particular league, is it?  The rest of the league setting the price of Halladay to $40 does not make that a wise move.  It just makes it “what the league did”.

    If his statline doesn’t have the risk that he is going to get hurt in it its not a very good stat line in my opinion.

  33. Peter Kreutzer said...

    This discussion has been about the fallacy of basing bid prices on projected statlines. If you’re going to ignore the projected statlines and use another price (not based on a projected statline), in fact based on whatever you know that makes you not want to spend more than $32, then we don’t disagree. That’s what I think we all should do, and in fact what we all do.

    Though I would bid $33.

    You say, “I think you are saying that we know his market price. But his market price is not the same thing as the value of the stats he is likely to produce, is it?”

    Maybe I wasn’t clear. I think it’s really important to know what a player earned in the past. And I think it helps to know what players who earned like that in the past at similar ages earned in the future.

    I also think it is important to know what leagues paid for those players. In my magazine you find all that information. This is not a plug, I’m just pointing out that the information is available and valued by many tens of thousands of readers.

    Combine what a player earned from year to year, what leagues paid for the player from year to year, and what other players with similar profiles did moving into the future, and you have a good idea of what a player is going to cost and what he might earn in the coming year.

    None of which has to do with projecting his stats. Which is how I got involved in this discussion. Oh, setting a price for a player is like projecting his stats, but it isn’t the same thing. Statlines are statlines, and they mean a particular thing. In the case of the previous post, your statline projection for Halladay earned $38.50, but you weren’t going to spend more than $32 on him.

    I’m not challenging the validity of your projection, or the efficacy of your price, but what does one have to do with the other?

  34. eric kesselman said...

    Where you lose me entirely is when you say stuff like “We know his price without the statline”

    How?

    You say you get no meaningful information from the statline, so where does it come from?

    You keep suggesting the answer is historical earnings and market price. Historical earnings used as a guide to the future is really just a projection in disguise. And saying we just know the market will bid Halladay $35, so he’s $35 is obviously fraught with all sorts of problems.

  35. eric kesselman said...

    Looks like you beat me to submission by a couple minutes and addressed some of my questions.

    I agree looking at the market data you suggest will give you a good idea of what a guy will cost. I don’t see how it gives you any idea of whether or not you should pay it or not.

    And isnt looking at past earnings the same thing as making a stat line projection, just in another form? You can say Halladay’s past 3 years are XXX ips, XX ks, XXX wins and therefore I project him to a similar season next year (which is worth $40) OR I say past 3 years Hallday earned 40 39 41 so he’s worth $40 next year.

  36. Peter Kreutzer said...

    If KY gives me a projected statline that’s worth $38.50 and then says he’s not bidding more than $32 for Halladay, what is the use of the statline for determining price?

    I know that Halladay earned 19, 39, 39 the last three years.

    I know that Halladay cost 29, 23, 32 the last three years.

    What am I going to pay for him?

    If you use my projection you’re going to pay $40. If you use KY’s projection you’re going to pay $38.

    I say you should pay no more than $33. KY says you should pay no more than $32.

    What good are the projections for determining price?

    This goes to another issue, too. If you set a price list for your league, and your projection sets Halladay at $38, even though you’ll only pay $32 (and the most fanatic owner will gladly take him for $33), you’ve just unloosed $5 or $6 that has to be accounted for elsewhere. A truly functional price list will not only account for the prices you want to pay for players, but also account for what the other owners will pay for players, because that’s what eats up money out of the budget.

    None of which, like a rusty clock I sound again, has anything to do with projected statlines determining prices.

  37. Peter Kreutzer said...

    My friend Alex Patton has described this in simple terms that may help clarify this discussion:

    “Projections are predictions. Bids are bets.”

    They’re different things.

  38. eric kesselman said...

    Sure, they’re different things. Predictions are what I need in order to figure out how much to bet.

    I still don’t understand your resistance to saying my bid = my projection – risk discount. My 38$ projection has a lot to do with my bid. It’s why I don’t bid $25 or $42. I don’t understand why you keep suggesting a $40 projection basically requires a $40 bid or its meaningless. As you noted they are different things, but they are VERY related.

    Your point about ‘unloosed’ money is a good one, and I think Bill is going to address it on CR.

  39. Peter Kreutzer said...

    Eric: “And isnt looking at past earnings the same thing as making a stat line projection, just in another form? You can say Halladay’s past 3 years are XXX ips, XX ks, XXX wins and therefore I project him to a similar season next year (which is worth $40) OR I say past 3 years Hallday earned 40 39 41 so he’s worth $40 next year.”

    It’s the same thing if all you do is look at earnings. If you look at what people are paying, you can figure out the discount.

    If you calculate the actual Halladay projected production, you might decide he’s worth $40. But that will be wrong. Your market research will tell you that although he’s earned 39 the last two years, he’s likely to go for 32 or 33.

    Of course there is an implicit projection in setting a price, but since we play the game with prices (not projections) the price is much more known and of much more importance than the translated value of the projection. (Which, as I’ve tried to explain rather endlessly above, is much more shaped by assumptions the projector has to make than the player’s actual value in any given fantasy league.)

    (The above raises a really good point… If a league’s values get out of whack, someone who can figure out the distortion could step in and win. In fact, that’s what I try to do in all the leagues I play in. Smart players don’t mean perfect prices.  And, of course, not all leagues are equal, especially since many of the interesting ones have unique rules.)

  40. Peter Kreutzer said...

    I have no problem saying my bid equals my projection minus whatever discount there is for risk. That’s what my bid is.

    I have a problem saying that Halladay will pitch 175 innings of 3.25 ball, with a 1.21 WHip and 16 wins, thus earning his $32 bid price.

    They’re different things. And if you don’t think they’re different, you’re missing an important part of the game. (Of course, I know you aren’t missing that part, so I’m uncertain why you aren’t getting my point. But that’s for later.)

  41. Peter Kreutzer said...

    You can disagree, but I didn’t say that the problem was drafting middleweights. I said drafting from projected statlines gives you less precision about who and what you draft than setting prices based on the market. Projections are inevitably wrong (not because of variance (though that’s true too), but because they measure different things, depending on whether you’re trying to gauge what all players are going to do or what one player is going to do) and setting prices based on them is a mistake. Projections are a prism, and the angle you look at them from changes what comes out.

    Since you haven’t argued that pricing from projections is a good idea, I don’t think that’s an issue for you. But I certainly didn’t say that it was an inherent problem drafting middle weights. I said that if you priced from projections, you would inevitably draft mid-price guys, and there are good argument for saying that’s not always the best way to put a team together. And if you priced from projections and changed them to incorporate the sort of intuitive opinions we have about players each year, which aren’t contained in any formula, then you’re doing the same thing Chris Liss does. You’re making it up based on what you know, which has come to be known here as intuition.

    I also said no one had a talent for good scrubs, though we all think we do.

    Finally, if a team punts a category, it makes it easier for other teams to gain a point in that category, and harder for all teams to gain points in all the other categories. There’s no inherent advantage to punting, except disrupting prices and forcing teams to ad lib off their price lists. I brought it up here to point out that there is not one price list for all players. You can say, If someone spends more than your bid price, that you’re going to get bargains later, but if they’re drafting off a different price list, those bargains may not come. Or you may feel like you got bargains and then you end up winning steals by 40. Pfft.

    Oh, another finally: Drafting some expensive players doesn’t necessarily mean buying cheap scrubs. Liss’s strategy, enumerated more than a few times at the cardrunners blog, is to buy great players to get stats, buy midlevel upside guys, and then wait until the price on regulars with playing time go for cheapish prices, to fill out the lineup with AB. No $1 players, but some $30+ guys.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>