What we can say for right now is that there appears to be a relationship—slight, but statistically significant—between viewing height and line drive rate. Further study—and a better model for line drive rate—could perhaps show the relationship is stronger or weaker than this preliminary study has. I do want to emphasize that the relationship we are seeing in the data is what we should expect to see based upon what we know about optics, although depending on how you feel about the matter you could consider that either another data point in support of the idea or something that's encouraging confirmation bias for me.

So let's look at ground balls for a moment. A ground ball is much simpler to study. We don't care too much about the trajectory of the ground ball. We're simply interested in knowing the direction it's headed, typically expressed in the angle it travels from home plate. So how well do we know that?

What we think we know

First, just to be sure we're on the same page, let's define a ground ball as (practically) any ball that lands before it reaches the outfield, or a ball that at least we would expect to land before reaching the outfield. (The exception comes when someone drops or misses an infield fly ball; say he loses a popup in the sun.) There may be a few cases where a low line drive and a grounder can be confused, but they're subtle and rare and probably don't affect our analysis much. Suffice it to say that everyone probably has the same conception of when a batted ball is a ground ball.

The next question is, where does it go? Typically, the field is divided into "zones" that each ball is assigned to, like so:

That chart is based upon the one for Project Scoresheet, the largest source for hit location data available freely and the source for Retrosheet's play-by-play data from 1989 to 1999. Those are the data we'll concern ourselves with for the time being.

The process of collecting that hit location data is simple: In its most basic form, a human being sits there and makes a judgment call. This is true for almost any source of hit location data we have available. The question isn't whether humans make errors; we know they do. The question is about bias: Do those errors occur in ways that don't wash out of our sample with enough repetitions?

Looking for park effects

The most obvious control over where a ground ball goes is the handedness of the batter: A left-handed batter will tend to hit the ball toward the first base side of the field, while a right-handed batter will tend to hit the ball toward the third base side.

We already have some reason to believe that pitchers have little to no control over where their ground balls end up. It doesn't even appear that they have a significant impact on which side of the field a grounder is hit to, once you account for the handedness of the batter. (Of course pitcher handedness is a crude proxy for batter handedness in the aggregate, so if that's all you have it's better than nothing.)

So here's what we'll do. Let's take the average rate that a grounder is hit in each zone, broken down by batter handedness. Using that, we can calculate an "expected" rate of balls in zone for each park from '89 to '99. Then, let's compare that with the actual and see what we get.

Looking at the root mean square error between expected and observed rates in each zone, we get an average of .020, with an average of about 2,000 balls in play at each park. Looking at expected random variance, we see that it should explain an average error of only .008. In other words, over an extended period (sometimes 10 years), we see a difference in the distribution of ground balls between zones that we can't easily explain through random chance.

What does it mean?

As far as I can tell, there are these potential explanations for this difference:

• Differences in the field of play. For instance, a "crowned" field will see more ground balls at the sides than up the middle (relative to the average) because the slope of the field will tend to push grounders toward the foul lines.
• Individual scorer bias. For instance, some scorers may have been more likely to score a ball in the 'tweener zones, like 56 and 34, if it passed between the two fielders than others.
• Observer position bias. Similar to the problem with line drives, the viewpoint of the scorer could impact what zones he assigns a ball to. So in some parks, a scorer might be more likely to score a ball as being toward the line, or toward the center.
• Particular hitting styles. Any one individual hitter shouldn't have a significant impact on our total figure. But if a team has a certain hitting philosophy and is able to impart it to all of its players efficiently, that could move the needle a little.

Given the data at hand, it's hard to say for sure how much each effect acts upon what we see in the data (and on the field of play). I think at the very least it's safe to say there is a significant difference in the recorded distribution of ground balls in play from park to park, one that does not seem to be attributable to random error. What we don't know yet is how much of that is attributable to observer error and how much of that is attributable to actual differences in batted ball distribution.

Simply put, parallax is a phenomenon in which the apparent position of an object can change based upon the location of the observer. Let's consider a very modest change in location—moving from the driver's seat to the passenger's seat of a car. (Do not attempt this while the car is in motion, please.) Observe how the gas gauge looks in each position:

In this instance, the passenger would tend to believe that there was less gas in the car than the driver would.

Baseball fans will be most familiar with the effects of this phenomenon when it comes to calling balls and strikes off the television feed; most broadcasts use an offset camera angle that distorts our view of the strike zone. But what else could it affect?

Most of what we know about batted balls comes from stringers who score games. Typically those stringers are given a spot in the press box. Now, because of the way different stadiums are constructed, the view from the press box shifts around. This, for instance, is the view from the Wrigley Field press box:

Compare to Citi Field:

It's a bit more of a dramatic difference than simply moving over a seat in a car, isn't it?

So let's test a theory—that the placement of the observer has an effect on how that observer determines the trajectory of a batted ball. Let's focus on air balls—fly balls, line drives and pop-ups. Based upon what we know, we should expect that the higher the observer, the flatter a batted ball looks and the more likely it is to be scored a line drive.

Grading the parks

Figuring out press box heights is not a simple task. I did the best I could given the tools I had. But the heights I collected are at best estimates. This is especially true for stadiums where press boxes have multiple levels. And for some parks I gave up on trying to get a decent estimate at all. I collected data on 27 parks in use from 2005-2009. The entire list, including the estimated heights, is available here. Parks are coded with Retrosheet park codes. I excluded one park from consideration, Coors Field; its inflated line drive rate caused by the high elevation makes it unsuitable for this study.

Then I calculated line drives per total air balls (flies, liners and pop-ups) as per the batted ball data available from Retrosheet from 2005-2009. Those data are based on the observations of MLB Gameday stringers. To avoid having a league bias, I removed all at-bats from pitchers. And because a team often uses the same hitters over a period of years, I looked only at the visiting team batting. Yes, there may be some persistence of pitcher line drive rates across seasons, but it's a minor effect compared to the persistence of hitter line-drive rates.

We do in fact see a slight correlation between press box height, about .16, after weighting for the number of years a park was in use during the sample:

(Each park gets its own data point, but the correlation—and the linear trend line—are based on the weighted data.)

If that's all there was to it, we could probably table the matter as perhaps real but not especially significant. But let's focus on the extreme parks a minute—those 40 feet or lower and 70 feet or higher:

The blue-coded points are parks that are either extremely low (the Oakland Colliseum and Shea Stadium) or extremely high (Fenway Park, Turner Field, PNC Park and Citizens Bank Park). They don't seem to follow the trend line at all; they actually seem pretty centered on the median. My hunch is that in those parks, scorers aren't relying on their view from the press box. Instead, they are looking at the TV feed. If we look at the trend with those parks excluded, the relationship becomes much stronger, with a correlation of .38:

Running a regression analysis, we see that a change in observer height of one foot is worth nearly .002 points of line drive percentage. That's a significant effect, for my money.

The implications

So far we've looked at a theory about how batted balls are observed and provided some evidence to support the claim. What are the implications, if this theory is true?

The professional data providers, BIS and STATS, certainly should provide better data than what the Gameday stringers provide; they take more care with the data, provide cross-checking, etc. But they are still unable to provide a consistent point of observation in every ballpark. (STATS uses a primary scorer in the press box; BIS has no in-park scorers but relies on video feeds.)

The implication of this is that we could see an effect where fielders are over- or underrated by defensive metrics based upon that scoring data, even over a period of years, because of an error introduced by a persistent bias. What I can't tell you—at least not without a lot more study—is which players, by how much or even the magnitude of the potential effect.

This isn't a repudiation of current defensive metrics, mind you. But people get the impression that they are based on a cold, calculating computer. But all current means we have of measuring defensive impact are based on human observation. We don't have a perfect means of evaluating anything— hitting, defense or pitching. It doesn't mean we don't strive for perfection, though.

I'm here to tell you that in fact there is—the relational database. And I'm here to help.

First, a warning: I am not a professional database administrator. I am a hobbyist. I may not follow "best practices." I will not be explaining a lot of relational database theory or lecturing you on the importance of the normal forms. Frankly, I don't know too much about that myself. And if you have your own database setup and workflow already, your setup may differ from mine. I hope you can still derive some value from this.

Also, I run Windows, so this tutorial will be written for Windows users. If you use a Mac, the people at Beyond the Boxscore have some helpful hints.

Most modern databases use something called the Structured Query Language, or SQL. It is, yes, a programming language. But please, don't let that scare you. It's simply a way for you to "talk" to a database and ask it questions. We'll get into that more in a minute. First, we need to set up a database.

Installing the software

There are many databases to choose from. I use MySQL. It's free, which is nice. And there's a large user base, which means there are plenty of tools and tutorials for you to look at.

First, you need to download and run the installer for MySQL 5.4 CE Essentials. Choose a "typical" install. You'll typically have to wait a few minutes for files to copy. Then there will be some ads. Skip over them. Make sure that "Configure the MySQL server now" is checked. (I uncheck the box about registering the software with MySQL, but that's up to you.) Chose "Detailed configuration."

Then you'll go through a set of pages asking you questions. Here are the answers:

• Developer Machine
• Non-Transactional Database Only
• Decision Support (DSS)/OLAP

Continue on, using the defaults. When prompted, either add a password or uncheck the box to alter security settings. (This will leave your root password blank.) Once you're finished, you shouldn't have to start MySQL at all—it should be ever vigilant, waiting for you to command it to do your sabermetric bidding.

(If you get an error message that says "This installation package could not be started. Contact the application vendor to verify that this is a valid Windows Installer package," you need to download and install Windows Installer.)

Again sticking with the theme of "absolutely free," I use SQLyog. You want the "Windows-Binary" from the Community Edition. Installation is straightforward—just download it and click "Next" a few times. Then run SQLyog.

From there, you'll be presented with a screen asking you for the settings to connect to the database. Everything should be correct. If you selected a password, type it into the password blank. Then hit "Connect."

And... voila! You've been connected to, well, to an empty database. I suspect you want some actual data in your database.

Putting the "data" in database

There are obviously a lot of different things you could want to have in your database, depending on your area of interest. Let's start off modestly, if not exactly small. The Baseball Databank is a record of pretty much every official statistic for every baseball player in the history of the majors. (Right now, it's current through 2008; the latest update is expected soon after the awards voting is finished.)

First, download the ZIP file. Extract the .SQL file from the ZIP, and remember when you put it. Now go into SQLyog.

In SQLyog there will be three panes. The left-hand side should list your databases, the top is where you write queries, and the bottom shows the results. Let's add a database. Right-click in the pane, and select "Create Database." Call it "BDB," leaving off the punctuation. Leave the rest as the defaults, and click okay.

Now, let's populate the database. Right-click on the database, and select "Restore from SQL Dump." There should be an button with an elipsis (...) in it. Click that, and select the file we extracted from the ZIP. Select Execute. It'll ask you if you want to execute the queries in database BDB. You do. This should take a little while.

Once that's done, go into your database and click on the folder that says Tables. If nothing is there, try hitting "F5" on the top row of your keyboard to refresh the browser.

Go ahead and click on batting, and then slide on over to the right and click on "Table Data." Scroll around for a bit. If you've ever used a spreadsheet or looked at a table before, the layout should feel familiar. Now it's time to play around with the database a little.

Asking the database questions

The fundamental unit of the database is the table, which is further broken down into the row and the column. Each row is a "record" in the database; you can have as many records as you like. Every row has several fields which contain the data you're interested in. A column is a group containing all of the fields of a certain type in the table.

You won't be entering formulas into cells, like you would a spreadsheet. You interact with the data in the spreadsheet by writing queries. Let's start off by looking up the hitting stats of the 1990 Royals.

Slide up into the query editor and type in this:

There is a green play button in the upper left-hand corner; press it. (Or just hit F9.) Most of the column headers should be pretty self-explanatory: H is hits, SF sacrifice flies, etc. And playerID is each player's unique identifier, taken from the first five letters of the last name and the first two letters of the first name, plus a number. Some are easier to figure out than others.

To explain:

• SELECT is your bread-and-butter SQL command; it tells the database to fetch records out of the table.
• * is a wildcard; it tells the database to fetch all columns.
• FROM tells the database what table to fetch from.
• WHERE tells the database to filter the information.
• AND means you want the database to match multiple columns.
• The semicolon tells the database that our instruction is complete.

The equal sign is one way to compare two values in SQL. The quotation marks are important; if you don't use quotation marks, MySQL will assume you're referring to a column of that name.

In some results you'll see a value of (NULL); that means the field is empty. In the batting table, NULLs are generally used to indicate:

• Players who appeared in a game but had no official plate appearance (generally relief pitchers, pinch-runners and defensive replacements).
• Statistics that were not tracked that year (generally sacrifice flies, double plays, etc.).

In this case, we really only want to see players who went to the plate. And let's say we also want to know the "sabermetric triple crown" stats of batting average, on-base percentage and slugging percentage. (Tango's sabermetric wiki is a great source of formulas to try out, incidentally.) We can add to our query a bit:

You don't have to break up the different lines; I do that for legibility. You do want to separate each thing you want to select by commas. I put the commas at the start of each individual line, but that's a personal style preference. SQL simply ignores carriage returns (that is to say, line breaks), tabs and extra spaces, so feel free to "pretty up" your code in a way that makes it easy for you to read.

SQL will do any simply arithmetic with the records you provide. (You can also do a lot more complex math with it if you really want to.) The AS command is very helpful—otherwise the column will end up being named "(H+BB+HBP)/(AB+BB+HBP+SF)," which simply isn't very convenient.

Let's say we don't want every column, we just want a few columns:

Bored with the 1990 Royals yet? Let's try something different.

ORDER BY tells the database to sort the list; Desc tells it to go from large to small. (Asc is the corresponding command to sort from small to large.) LIMIT tells SQL to return only the first 50 records.

So that's the top 50 home run seasons. (Supply your own asterisks where you feel it necessary.) But what if we want career totals?

The SUM command will sum everything in the column unless you stop it. GROUP BY tells the database to combine everything based upon player ID.

Now what if want to look at the top 50 career batting averages? We obviously don't want to see some pitcher who lucked his way into going 2-for-3 and never saw the majors again. So let's say we want a minimum 3,000 career plate appearances.

Because we're using the SUM function, we have to use the HAVING function instead of the GROUP BY function. Note the >= - that's greater than or equal to.

So, what's that COALESCE doing there? Remember, sacrifice flies weren't officially recorded until 1954. For years prior, those values are NULL in the database. Trying to sum a NULL value chucks everything out. Using the COALESCE function in this way tells the database to substitute 0 in the place of a NULL. Otherwise, players like Ty Cobb would be excluded from the list (precisely the sorts of players we don't want excluded!)

Now let's try writing a query to list the 50 lowest career earned run averages, minimum 1,000 innings pitched. The Databank doesn't list IP for pitchers, simply outs. Remember to divide by three:

    SELECT playerID
            , SUM(ER)/SUM(IPOuts/3)*9 AS ERA
            , SUM(IPOuts/3) AS IP
    FROM pitching
    GROUP BY playerID
    HAVING IP >= 1000
    ORDER BY ERA ASC
            LIMIT 50;

If you run this query, you'll have a lot harder time recognizing the top leaders than the top batting average leaders. A lot of these guys were deadball-era pitchers. So let's find these guys' names.

Working with joins, a first glance

Often, all the data you want to use won't be stored in a single table—you'll have some data in one table, and some data in another. That's where joins come into play. A join lets you hook two (or more) tables together for querying.

There are two ways to do a join: explicitly and implicitly. When I say an explicit join, I mean a join where you use the JOIN command. An implicit join is one where MySQL is just supposed to figure out that you want it to join. For now, we'll start off with the implicit join.

In this case, what we want are the first and last names out of the master table. In this case, what we're really after is a player's full name. So let's put that together:

CONCAT is a fun command for string manipulation. It lets you take multiple strings and combine them. In this case, we took the first name, a space (surrounded by quotes) and a last name, separating them with commas. CONCAT does the rest.

Instead of joining to the entire master table, we're going to join to the results of that query using an alias.

In SQL, you use the period separator to indicate what table you want to take data from, like so:

table.column

where table is the name of the table and column is the name of the column. You can also assign an alias to a query when you use it in the FROM command, like so:

So now, instead of seeing that walshed01 is the career ERA leader, we can see that it's Ed Walsh. It's an improvement, right?

The key is in the WHERE clause, which matches up a column from one table with the other. The equivalent explicit join is:

    SELECT p.playerID, Name, ERA, IP
    FROM (SELECT playerID
                , SUM(ER)/SUM(IPOuts/3)*9 AS ERA
                , SUM(IPOuts/3) AS IP
        FROM pitching
        GROUP BY playerID
        HAVING IP >= 1000) p
    JOIN (SELECT CONCAT(nameFirst," ",nameLast) AS Name
                , playerID
                FROM MASTER) m
            ON p.playerID = m.playerID
    ORDER BY ERA ASC
            LIMIT 50;

You want to be careful when writing the WHERE or ON clause of a join—if you aren't careful, you'll end up doing what's called a Cartesian join, or essentially joining every row in one table to every row in another. For those of you keeping score at home, that would mean joining 39,016 rows from the pitching table to 17,264 rows from the master table, giving you... 673,572,224 rows as a result.

You do not want that.

We're far from done with joins—they're quite possibly the most powerful (and most dangerous!) tool in your SQL arsenal. But for right now let's move along to some other things. We'll be back.

Dealing with data types

I want you to see this at least once before it creeps up on you by accident. I do not, on the other hand, want to be the sort of tutorial writer who hands you a code sample and then, when it doesn't work, shouts "Gotcha!"; So I'll be clear with you: This doesn't work right. Let's say you want the top 50 player seasons with more runs scored than runs batted in.

Try running this:

    SELECT playerID
            , yearID
            , teamID
            , R-RBI AS R_RBI
    FROM batting
    ORDER BY R_RBI DESC
            LIMIT 50;

We are being lied to here. Nobody, but nobody, has 18,446,744,073,709,551,615 more runs than RBI in a single season. I don't even know how to say that number.

The problem is with data types. SQL has an awful lot of data types, but we're only going to look at a few of the more common ones.

Text is generally stored as "varchar," or "Variable Character Field." What that means is simply that you can store any length of text in there up to the size of the field. There are some other text formats, but nothing to really be worried about right now.

There's a much greater diversity of number formats, which can be broken down into three types:

1. Integers, or whole numbers.
2. Decimals, for precise fractional values.
3. Floats, for approximate fractional values.

These can be further subdivided: you'll see values like "tinyint"; or "smallint"; thrown around, for example, as ways of describing the size of the number you can store in that column.

There's also the issue of signs, which becomes important here. For SQL, what we mean by signs is the plus/minus sign—whether a number is positive or negative.

You can store numbers in MySQL either as "signed"; or "unsigned";. Unsigned values cannot be negative. The Baseball Databank stores all numbers as unsigned.

(Why? Because it saves storage space and time.)

So in order for you to deal with unsigned numbers where values may be negative, you may have to change the datatype before doing any calculations. We do that with a CAST function:

The heart of the CASE

There are times when you want your query to be able to make some decisions on its own. Let's say , for instance, that we want to look at pitchers between the ages of 25 and 29, post WWII. Remember, a player's age is generally listed as his age come July 1. So, a simple little query here:

The CASE statement allows you to have different outputs based upon the underlying data. This could have easily been written as an IF statement as well:

I like CASE because it feels cleaner, personally. You can also use multiple WHEN statements inside a CASE.

Saving your work

Let's say you've written an amazing, bad-ass query. You know you're going to want to look at this one later. You have two options:

Simply put either one of those at the front of your query.

So what's the difference? A view:

1. Updates whenever the underlying data does.
2. Preserves the query used to generate it.

Creating a table:

1. Does not update to incorporate changes in data.
2. Does not run the query every time it's called.

A view is more flexible; a table is quicker.

Also, I recommend creating a new database to store tables in—multiple databases if you have multiple projects. So let's say we wanted to start a new database for a project that does Marcels projections. Right-click in the pane on the left and select "Create Database." Call it Marcels and click on it to make it the active database. To run queries on tables in the Baseball Databank, simply prefix the names of the tables with bdb., like so:

bdb.batting

bdb.pitching

Well after that little slip, it would hardly be fair to send you on your way without some Marcels projections, wouldn't it?

First off, I just want to thank Tangotiger, who came up with the Marcels, and Sal Baxamusa, whose spreadsheet versions of the Marcels were invaluable as learning aids for me.

I tried my best to reimplement the Marcels as faithfully as possible, but there are (likely) differences between Tango's work and mine. All credit goes to him and Sal, all blame for mistakes and errors goes to me.

Here's the code for hitter projections. Have fun with it, play around with it. Try to make it better, try to break it, try to take it apart and see how it works.

That should be enough to get you started. If you have questions, don't be afraid to ask! And most of all, have fun.