---

Posted by David Gassko at 12:44am

Wednesday, October 29, 2008

My player evaluation philosophy

I had planned on talking about my second mock draft of the year today, but after some of the participants pulled out at the last minute, it never materialized. Therefore, I thought that I might discuss with you today my general philosophy when it comes to player evaluation.

It's the start of the off-season, and I know that some of our new readers are probably unfamiliar with a lot of the stats that I use in my articles. I'd like to discuss my feelings on stats in general and on evaluation methods that aren't stats-related (quick note to those reading this and thinking "I don't want to hear from a guy who is in love with stats and doesn't consider anything else"—that's not me, so keep reading).

What are "sabermetics"?

I'd first like to start out by giving my interpretation of the term "sabermetrics." Sabermetrics is not—at least in my view—necessarily the same thing as "statistics." Sabermetrics is about truth, regardless of the form it takes. In most cases, this includes statistics, but non-statistics can also be a part of the process if treated carefully.

Statistics

This isn't, however, an implication that all statistics are good. In fact, the vast majority of them are either flawed or useless, for fantasy purposes or otherwise. This is why I am extremely selective about the stats that I use. I sometimes get questions about why I don't use a particular stat. The answer is almost always because there is a better one out there (which I am using). And when I'm not convinced that a sufficient stat already exists, I invent my own, if at all possible.

I don't proclaim to be all-knowing, but I do believe that every stat I use is the best that is currently available (although I'm always scrutinizing them and looking for ways to improve). If you ever have a question about any stat that I use or why I don't use a particular other stat, with your question is greatly encouraged. I'd be happy to explain my thinking to you, or admit that I'm wrong if you have a better idea.

Stats aren't the end all, be all, however. There are some things that really can't be put into numerical form that should be considered. This is where scouting comes in.

Scouting

Scouting can give us information that stats simply cannot. Combining the two can give us a fuller view of a player and can provide insight that neither could do alone. As an example of this, last off-season I examined Geovany Soto, who put up a tremendous 2007 season in the minors, pretty much out of nowhere. If looking solely at the stats, we would say that Soto should see some heavy regression (could even be a fluke) and should be expected to hit far fewer home runs in 2008, especially jumping to the majors.

If we considered some qualitative scouting information, though, we would be a little more optimistic. John Sickels of Minor League Ball said that Soto had added strength and changed his swing to add more loft to the ball, which were the cause (in part) of Soto's power surge. Taking this information into account, as a rough estimation, I predicted a pretty sizeable 15.0 percent HR/FB for Soto this year. His actual HR/FB turned out to be 14.7 percent.

If some things that usually fall under the "scouting" head can be quantified, though, they should be. While things like mechanics, injury analysis, and (as in Soto's case) added muscle mass really can't be put into (or aren't readily available in) numerical form, recent innovations like PITCHf/x and HitTracker (both of which are used by THT Fantasy Focus) allow us to examine some things that formerly were only able to evaluated using traditional, eye-ball scouting methods.

As far as non-quantitative scouting information goes, I'm not an expert and don't proclaim to be. When information like this is applicable, I'll simply quote someone who knows what they're talking about, such as THT's own Chris Neault or Alex Eisenberg or something like Tom Tango's Fan Scouting Report (as a side-note, look for an article later this week or next relating this to stolen bases).

The one thing we need to be careful of is that qualitative scouting information isn't easily tested. It would be very difficult to go back and systematically see whether or not the process worked. We are trusting that those we are relying on know what they are talking about, and even then, we can't say for certain how accurate this information is. I don't have a big problem with this as long as I trust the person in question and as long as we realize that using this type of information is inherently more risky, although we do need to keep it all in the back of our minds.

Sabermetrics

Sabermetrics, in my opinion, is simply about truth, regardless of the form. Combining stats and scouting is often the best way to achieve truth, despite how few people seem to be willing to admit it.

Stats need to make up the majority of our analysis, though, because they can capture in a precise manner exactly what a player did (or should have been expected to do). The stats are testable and we can know with relative certainty just how accurate they are. The scouting information needs to supplement the stats, if necessary. While I believe scouting is much less important than stats (and might not be necessary in many cases), it can be a useful tool for us.

None of the above

While many writers are against the use of stats, this isn’t to say that they are “scouting guys” by default. They may (and probably are likely to) call themselves this, but we must consider what a scout actually is. Scouts go to scout school or at least study scouting methods and techniques on their own. They examine mechanics and biomechanics and things of that nature.

I believe there is a distinct difference between this type of scout and someone without such training who watches a baseball game, posts to a website or blog, and gives their opinions under the guise of it being a “scouting perspective” simply because numbers aren't involved. I don't really believe that feelings, guesses, and hunches fall under the "scouting" heading, and they need to be made and accepted with a great deal of caution.

(Please note that this is different than something like Tango’s Fan Scouting Report, which deals with the opinions of people who may be doing no more than simple eye-balling. When you can combine the opinions of a large enough sample size, that opinion is likely to be quite valid, even if selecting any one of participant could give you completely incorrect information. In economics, this concept is referred to as the “wisdom of crowds.”)

Intangibles

So where do things like team chemistry, hot streaks, slumps, clutch hitting, and things of that nature come in? Sabemetrics does not deny the existence of these things, contrary to popular belief. These things do very much exist, and saying that they don't is simply incorrect. The thing is that they either can't be quantified or aren't repeatable skills. Because of this (and because the impact, in my opinion, is relatively small compared to the actual talent of the player, anyway), they aren't included in the process.

We could make guesses, but they would be just that: plain, shot-in-the-dark guesses. It's been documented that people who do that don't tend to predict things very well. At least when we (someone who is qualified, that is) look at mechanics and make an assessment, it is based on some logical process (even if it can't be quantified).

How do we know, for instance, who is a good clubhouse influence, though? There's no logical method that is available to the majority of us to come to a reasonable conclusion. Unless we talk with the other players in the clubhouse (and are certain we're getting a truthful response), we simply don't know.

We can listen to beat writers and reporters, but most of the time they are simply writing whatever makes a good story. That's their job, and that's fine, but we can't trust those stories for our evaluations. The media creates an image of a player, and writers far too often fall back on that image, which simply perpetuates it.

A great example of this is Carlos Beltran, who is often abused by the media and the blog community for not being a leader or for being too soft-spoken. But they don't know what goes on in the locker room, and a number of Beltran's teammates have actually said that he is a good leader (not that we can take those statements at face value). Of course, this is of little consequence to the media and blog community, who continue to talk about Beltran as if he is a hindrance to the Mets.

There is a smaller sect, however, that actually think Beltran is a good leader. They simply say he's a quiet, lead-by-example guy. These opinions on leadership (and on most other related things) are wholly subjective, so much so that you can come to complete opposite conclusions and still be able to support them. They both can't be true, though, and neither can be measured. They're simply guesses, and these kinds of judgments just can't be trusted.

So absent any truly credible information about these kinds of things, they must be ignored. If you have two players ranked very closely, sure, use it as a tie-breaker. But realize that this sort of thing is of minimal importance when compared with the player's actual baseball-playing skills.

Summarizing the four approaches

In summary (in my view), there are four approaches.

At one extreme are the stats guys who use stats and nothing else. This category can be broken down into two sub-categories.

One sub-category looks at surface statistics like RBI or ERA and irrelevant splits and trends, and the other (more preferable) sub-category looks at underlying numbers and indicators like BABIP and LOB% (the one warning I'll give here is that we must make sure the advanced stats we're using are actually capturing what we're trying to measure. A number of sites try to use advanced stats but use them incorrectly or use stats that aren't actually relevant. I try to be very careful to explain exactly what the stats I use mean so you guys understand where I'm coming from).

At the other extreme are the scouting guys who base their judgments on mechanical evaluations of player and some eyeballing of results.

Sabermetrics (my view of it, anyway) combines the two, relying heavily on statistics and supplementing them with scouting information when applicable.

Lastly, there are those who don't really rely on any of this and are essentially guessing, or who combine guessing with some basic numbers (like those used by the first sub-category of stats guys).

Be critical

When you’re reading a fantasy baseball analyst, always be sure to ask yourself how the analyst in question is coming to his or her conclusions. If they’re falling back on this pseudo-scouting (i.e. guessing) position or don’t give the basis for their opinions whatsoever, this might be something you’d want to take into consideration before trusting their advice.

Invitation

If you like my methods, I hope you begin reading my work (if you aren't already). While some of the stats I use will be entirely new to you, guess what that means? It means the only people who will be reading about them are those who read this site, since many of these stats can be found nowhere else. That is a huge advantage you can gain over your competition!

If you're serious about winning your league, it is well worth it to take a few minutes to learn about these stats so that you can follow along going forward (click here for a summary of every stat you'll see me use). You'll be happy you did. And if you ever have any questions, please feel free to , and I'll explain whatever you need.