Scouts, statisticians and wizards

I was about to leave a comment at The Book Blog in a thread titled When to go from the eyes to the numbers when I realized it was probably worth delving into here as well. It opened a philosophical door in my mind that occasionally opens and closes. It seemed a good time to take advantage of the open door and get my jumbled, innermost thoughts peer reviewed (or at least committed to paper for future reference)!

In post No. 8 of the thread I linked above, Nathan asked:

I guess the real question is, can anybody, just by watching 20 games, tell the difference between a .275 and a .300 hitter? I’m referring to batting average here which I know is lame, but the point is that the difference over 80 at-bats (roughly 20 games) is 2 hits! Can an observer notice 1 hit every 10 games?

Now, the fact that two hits separate .275 from .300 indicates that stats aren’t great in that small of a sample, but really, what can a scout see, can he see that this player is hit every 10 games good?

My response is No. 9 and you can view it for yourself if you like. In fact, I highly recommend taking a look at the whole thread. I’ll hit the highlights a little later. Really, it’s unfathomable to me how any person can divine the difference between a .275 and .300 major league hitter when he’s in high school, rookie ball, or even Double-A. The rate of attrition among minor leaguers could mean that perhaps they can’t, at least not with any real degree of accuracy . The problem I see with figuring it out is that there are so many different inputs that make up a good hitter. Coordination, reflexes, strength, eyesight, reaction time, mental toughness, intuition, temperament, focus, etc. all have some bearing on whether one player is one hit per 10 games better than another.

Numbers, with sufficient sample size, of course, allow us to proxy the net product of all the myriad inputs that make a player a player. But the sample size is the limiting factor. Beyond that, Cliff Lees abound in the baseball world, players whose skill sets undergo such massive changes that the previous data becomes nearly worthless. And anyone who watches the game knows that other difficult to explain phenomena occur. Raul Ibanez and Adam LaRoche come to mind.

It seems to me that scouts and statisticians are asking similar but ultimately different questions. The scout’s job is to learn the player, to become familiar with his mechanics, his strengths, his flaws, how he handles himself under pressure, how he spends his time off the field, how he relates to his family and loved ones. By doing this, the scout tries to paint as detailed a picture of the player as humanly possible so he can convey to his employers how much that player is worth. He judges the quality of the player’s skills. Knowing the quality of those skills and knowing which ones can be improved, he can estimate where a player is at now and where his ceiling is.

Statisticians do something else entirely. We ignore the majority of the inputs and focus on the measurable output. When we look at numbers (or at least when I look at numbers—maybe I’m being presumptuous in using “we”), we’re trying to quantify a player’s skill in a succinct and tidy manner. We don’t care if Chris Coste has a godawful swing, that it’s of poor quality and so very unlikely to stick at the major league level. We care that he produced a .316 and .326 wOBA in ’07 and ’08 respectively. And we care that he was a catcher, making him above average for his position. We don’t care that Milton Bradley’s temperament is at best questionable. We just care that he can mash the crap out of a baseball when healthy. (Yes, I’m simplifying.)

I must admit, that philosophical door in my mind that I mentioned earlier rarely stays open long and now it is closing quickly. I hope I got my views across clearly enough for some good dialogue. Oftentimes I’ve heard the work of saberists referred to as statiswizardry (which can be intended to compliment or disparage). Ultimately, I think it’s the scouts who do the magic by divining the quality of a player’s individual skills.

And I think that partially explains why some casual fans are resistant to openly accepting saberist ideas. All we have is charts and graphs and output from R and Stata. It’s all very convincing and useful stuff to statistically oriented minds, but the scouts have something more popular with the masses: Magic.


You can follow me on twitter @BaseballATeam
8 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Nathan
13 years ago

So the scout can see how the ball was hit, can’t batted ball stats do that? I’m not trying to start a scouts vs. stats argument, I’m really just wondering where the scout comes in. I guess I feel the only real purpose is to project. Example: I was a great high school hitter, but looking at my frame (a little like Tim Lincecum) I was clearly never going to be able to develop the power a major league hitter needs.

Brad Johnson
13 years ago

Dan,

I never truly jumped all the way onto the stats bandwagon myself. Or perhaps a better way of saying that is I never abandoned the idea of scouting. What I term magic is truly experience and insight. If anything, I think scouts are under utilized in baseball. Were I a GM, I’d find out which club had the most scouts and then I’d hire 30 more than that. Incidentally, I’d also try to them a little about the numbers…

Nathan,

You act as though projection is a minor thing. A scout’s job as I see it is to project, identify when a player CAN make a change, and identify when a player HAS MADE a change. That’s highly useful at every level of baseball.

One of the problems with stats right now is that they don’t adequately capture the fluidity of a player’s skill. A scout can give insight into that at real time while we require weeks, months, years of data. The increasingly loud call for error bars only partially solves the problem.

Dan in Philly
13 years ago

For years, I was a stathead.  After reading Moneyball and going from there to Baseball Prospectus and other such resources, I was totally on board to the saber guys and totally not with the scouts.  Like you, I termed what they do as magic.  I did this because I did not understand what they did, they couldn’t really explain it to me, and often they said their gut have them a hunch.  This seemed like so much magic to me, and I discounted it as mere superstition.

However I have since reconsidered my opinions and I now am a little (ok, a lot) less certain that sabers are better than scouts on this.  Sure you can call what they do magic, but it isn’t.  Instead, it’s experience.  Read the book “How we decide” by Johan Lehrer.  Seriously, read it.  It may change your views on the value of an experineced scout, manager, or GM.

I now believe there’s a place for sabers in baseball.  However ignoring those who have eaten, breathed, and lived baseball their entire lives is a mistake, a huge mistake.  They have internalized far more baseball than you can imagine and that “gut” feeling they get is the unimaginably complex human mind compiling countless individual pieces of data and projecting a likely outcome in what can only be described as a “hunch.”  Wise men pay attention to these hunches.  I have learned to as well.

Pat Andriola
13 years ago

Dan in Philly, I’m about to start “How We Decide” soon. I’ll post back after I’m done to see if I agree with your comment.

Derek Carty
13 years ago

I think I’d have to disagree a bit with you, Brad, or at least take a not-so-pessimistic look at scouting.  I’ve got a lot of thoughts on this, but I’ll just try to give a few so I don’t ramble on for paragraphs and paragraphs.

I believe that scouting is very important and definitely not magic, especially for those who truly know what they are doing.  I think your example of a scout picking out a .275 vs .300 hitter over 20 games is a little misleading (and I believe Bill James or someone made a similar point long ago, so I’m not just picking on you smile ).  Sure, it would be hard to recognize one hit every 10 games, but the scout isn’t counting hits.  We have stats to do that for us.  The scout is analyzing the hitter’s stance, load, timing, swing plane, bat speed, body type, overall approach, etc to determine what he thinks of him as a hitter.  And that kind of information, if done correctly, is valuable for a big league club, especially for hitters at the lower levels.  Even for hitters in the majors, identifying changes in these kinds of things can shed some light on why a hitter’s statistical profile may have changed.  Stats are descriptive – they tell us what happened – but they can’t always explain the why.  Scouting can help to do that.  Of course scouting is imperfect and has flaws, but so do stats.

I’d also vehemently disagree about teaching scouts some things about stats.  If I’m running an MLB team, I don’t want my scouts to know anything about stats.  That’s not their job.  It’ll bias their views.  I want my scouts to focus on scouting and my numbers guys to focus on the stats.  Then the job of the sabermetrician is to put the two together.

For me, I do care that Chris Coste has a “godawful swing” (hypothetically – I have no idea if that’s true) or that Milton Bradley has a bad temperament.  And, of course, I care about the wOBA and all the numbers too.  It’s all a part of the big picture, and it can all inform our projections for the player going forward.

So, I guess I did ramble a little.  Sorry about that.

Brad Johnson
13 years ago

Clearly I’m not a good writer. I think what scouts do is the most important part of player evaluation and I don’t think that’s what my piece has conveyed based on the comments. My use of the word magic was meant to complement their ability to divine a player’s potential usefulness from many disparate and seemingly unrelated things.

I used the 2 hit example because someone else used it. I agree that it’s more than a little misleading.

Brad Johnson
13 years ago

I’m having trouble finding it for under $7 after shipping (the self imposed price line I’ve set for myself on books while jobless). Ironically I’m spending far more on books now than when I was buying them new since I have time to read them.

Prices change all the time and $.01 books appear randomly, so I’ll get it eventually.

Paul E
13 years ago

It’s probably not totally related to your discussion, but I do recall Bill James insisting that Mike Stenhouse was going to be a guy who developed into an Edgar Martinez type of hitter (high OBA – 25 Homers)based on AA and AAA numbers. But, then again, he apparently was the fourth overall pick in the draft….so some scout and organiztion had to be fooled as well