The Physics of Statcast Errors

Even though I eventually earned a PhD in Physics, I struggled mightily with my first physics course. I’ll never forget the trials of Professor Endar Linzcor. I immediately nicknamed him “Dr. Line Score,” consistent with my life-long obsession with baseball.

Line Score’s inordinately strong but otherwise non-descript accent added another dimension to the challenges of understanding the nature of science. Nonetheless, Line Score taught me some key ideas for dealing with the complexities of data collected in the pursuit of knowledge.

I recall one pretty simple experiment during which we were supposed to use a stop watch. We lifted one side of a ramp and set it on a block. Then we used a stop watch to measure the time it took a toy car to roll down a ramp. Next, we added another block to increase the angle of the ramp and measured again. We did this about five times.

It was Professor Line Score’s rule that once we had our data, he had to look it over before we could go on to write up our lab reports. I dutifully reported our measurements. Dr. Line Score’s reaction included a furrowed brow, a chin scratch, and the words, “Dozent pash the schniff text.”

“Schniff text?” I echoed. The doctor’s crinkled up nose and a few sniffling sounds helped me realized we had failed to pass the sniff test! There was something funny about our numbers. After looking them over with my lab partner for the next twenty minutes, we realized our times were getting longer as the ramp was getting higher! I had forgotten to reset the stopwatch between measurements and thus failed the “Schniff Text.”

That brings me to Statcast, which completed its first full year of operation in 2016. It is a truly bold vision and a remarkable implementation of sophisticated radar and video technology. However, all science and technology is subject to errors and mistakes. Hence the importance of the Schniff Text–Sniff Test–to all such endeavors.

For example, let’s take the Statcast Leaderboard for 2016 sorted by the exit velocity off the bat–you know, the “Stanton Index.” Sure enough, Giancarlo Stanton is on top at a blistering 123.9mph. This particular data was recorded on June 9. Statcast claims the ball had a launch angle of -4.83˚ and traveled 141 feet before reaching the ground.

This data has the odor of a Sniff Test failure. It is extremely unlikely a baseball could possibly be hit with a downward trajectory and go 141 feet before hitting the ground. The video of the event shows the ball on the ground before it leaves the batting circle. Now the odor is rather feted, and we can conclude the Statcast data are wrong.

Second place on the exit velocity leaderboard is Jonathan Schoop at 121.6 mph on July 22. The rest of the Statcast data report the ball was launched at 36.22˚ and traveled 329 feet before reaching the ground–a homer in many situations. However, the event is reported as a ground out. Unfortunately, the only way to view the video is to watch the archived game at MLB.TV. It was indeed a 6-4 force play. Sniff Test failed.

The third event on the Leaderboard is Tyler Flowers’ 118.5mph single off Jeurys Familia on June 24. This event is completely consistent with the video. Of course, now we are down to exit velocities of fewer than 120 mph–in the Sniff Test range.

I point out this consistent result to remind the reader that my intent here is not to denigrate Statcast. It is wonderful and amazing technology that I hope will improve through the widely applied scientific maxim of the Sniff Test.

Statcast data clearly state when it’s system is unable to report results. For example, Kris Bryant made contact with the ball 452 times in 2016. The Statcast system reported the dreaded “null” result for the hit speed off the bat only 41 times, less than ten percent.

As a physicist, I can’t help but speculate as to the science behind many Statcast issues. Understand that I make these comments based only upon my knowledge of physics and the publicly known information that the system uses radar to track the ball. I have no inside information about the proprietary technology used by Statcast.

I have written about the physics of radar guns previously in THT. The Trackman radar is located directly behind home plate, mounted up on the face of the second deck in most parks. It is the large black square to the right of the words “Dodgers Stadium” in the photo below.

As any fan knows, this is the best spot for viewing the game because it is close enough to the game and high enough to get a full clear view of the entire field. That’s why the broadcast booths reside there. So, naturally it is the best spot for the radar.

However, radar signals are the easiest to deal with when the object is moving directly away from or directly toward the radar detector. So, balls hit to center field generally will be moving away from Trackman, while pop-ups will be moving mostly parallel to the face of the detector. Indeed, nearly half Bryant’s null results were on pop-ups.

Over one-third of Bryant’s null results were on grounders, including infield outs, errors, and singles. The problem with the radar in these situations is most likely the fact that the detector is having trouble sensing the reflections from the ball as opposed to the reflections from the ground itself.

As a physicist, I have a naïve suggestion that must have been considered by the engineers that designed Statcast. There could be more than one radar unit in the park. After all, PITCHf/x uses three video cameras and very rarely misses a pitch. Since the engineers that build Statcast are smart people, there must be some reason multiple radars are not as simple to implement as multiple cameras. It may be that added radar units would be unworkable because the signal one emits would be collected by the others, leading to more confusion rather than less.

As far as casual fans are concerned, I suspect Statcast is a success. The graphics generated with its data add a new dimension to broadcast baseball. After all, fans find the speed off the bat for a homer interesting, but the speed of a pop-up…not so much. However, as a scientific data collection system, Statcast can best be described as a “top prospect” as opposed to a Hall of Famer. However, it does take time to convert the former into the later.

References & Resources

• Baseball Savant, Statcast leaderboards
• David Kagan, The Hardball Times, “The Physics of Radar Guns”