The Physics of Replay Reviews

I guess I am a curmudgeon. When I first heard that baseball was adopting replay I saw it as the end of civilization. However, now that it is firmly in place I have to admit that MLB has done an excellent job implementing the replay review system. A  team of umpires reviews controversial calls from the Replay Operations Center in New York instead of at the ballpark – a major improvement over the NFL system.

The umps at the park, now absolved of responsibility for the decision, implement the call from the replay center. We are spared the once inevitable and time wasting arguments on the field. Finally, the decisive video is shared with the crowd,  providing some chance of maintaining civility in the stands.

Nonetheless, there are physical limits to what you can seen in video replays. While researching this article, I attempted to get information about the technical details of the cameras and the signals they provide to the Reply Operations Center. This proved to be impossibl,e as MLB seems to guard information more closely than the NSA. As a result, the data presented here are based upon typical sports video production cameras, not the specific ones used by MLB and its broadcast partners.

Let’s start with a basic description of the typical video available from broadcast television. Only a limited number of broadcasts use cameras specifically designed to produce slow motion replays. So, we’ll deal with just standard speed cameras where the slow motion is generated by just showing the images at a lower rate.

There was a day when TV broadcasts were simply sent to your set as 30 still pictures every second. Since it takes your eye and brain take longer than a 30th of a second to make sense of the image, the video appears smooth and continuous. Today, the situation is wildly complex.

The data your TV gets depends upon whether you collect your signal off the air or from cable/satellite and your provider. The image you see is determined by the software in your set. The still images may be presented at 30 frames a second (fps), 60 fps, or a combination known as “interlaced.” The frames are composed of horizontal lines. Interlaced signals are made from every other line of the last image and the rest of the lines from the current image. Bottom line, most cameras produce an image every 60th of a second.

So, one of the physical limits is associated with what happens between images. In other words, what can change in a 60th of a second? Table 1 describes several situations where motion is occurring.

Distance traveled between 1/60s frames

Description	Typical Speed	Distance
A ball thrown by an infielder	80 mph	2.0 ft
A runner at top speed	20 mph	0.5 ft
A sliding runner	10 mph	0.25 ft
Home run coming down	50 mph	1.2 ft

On a close play at first base, the ball has about two feet between images and the runner has covered six inches. These numbers alone explain why the umps usually have it right when you see the replay. They have a surprising amount of room for error that cannot be detected by typical cameras.

Then there are those homer calls where we need to know if the ball hit above the yellow line. The ball travels about 14 inches between camera frames. If the collision is with a solid wall, the ball will be in contact with the wall for much less than a 60th of a second. So, you are unlikely to have a still frame where the ball is touching the wall.

A second consideration is the time it takes to produce a single frame. This is called “shutter speed.” Shutter speeds can vary depending on the amount of light available for the camera. In the bright sunlight of a day game the shutter speed might be 1/1000s. At a night game it might be lower, say 1/125s. Table 2 illustrates the distance objects move during the creation of a single frame at both speeds.

Distance traveled during a single frame

Description	Typical Speed	Distance 1/1000s	Distance 1/125s
A ball thrown by an infielder	80 mph	1.4 in	11 in
A runner at top speed	20 mph	0.35 in	2.8 in
A sliding runner	10 mph	0.18 in	1.4 in
Home run coming down	50 mph	0.88 in	7.0 in

The motion during the time the image is constructed explains the blur of the ball or the runner in a slow motion frame. A ball traveling at 50 mph will look like a thick horizontal line more than twice as long as it is wide.

The third physical limitation is the camera angles. The angle from which a play is observed is crucial, so much so that getting a good angle is central to the training of umpires. I would contend that a large fraction of blown calls are due to the umpire having a bad viewing angle whether because of his poor choice of position or the vagaries to the play.

Major league broadcasts use between seven and 12 cameras. The high number is from the Yankees’ exceptional quality YES broadcasts. Not all cameras follow every play, so there seem to be rarely more than three distinct angles to review.

In summary then, there are three physical limitations for replay reviews:

1. The 60th of a second between video images.
2. The time to construct a single video image, the shutter speed, which can vary.
3. The limited number of camera angles.

On May 20, Tyler Colvin of the San Francisco Giants hit a ball down the left field line ruled foul by the third base umpire, who was in perfect position to make the call. The video illustrates all three limitations – I’ll wait a few minutes while you view the clip.

There seem to be three camera angles. The one from high above home plate shows the play at normal speed at the beginning of the clip. The view from the camera high above first base is shown at 19 seconds into the clip, and several times later at different levels of zoom and varying speeds. A camera near the left field foul pole looking in toward home is 32 seconds into the clip and is shown in slow motion.

The slow motion views from both the high first base and the left field cameras clearly show the effects of the time between the frames and the blur from the shutter speed. By a stroke of good fortune, the high first base camera actually has an image of the ball striking the ground, but the shutter speed leaves the call ambiguous.

The left field camera seems to clearly show the ball hit the ground foul, but careful analysis reveals the camera doesn’t have an image of the ball hitting the ground. It has one slightly before and one slightly after.

The decisive image is really due to the perspective provided by the high first base camera seeing the chalk or paint or whatever it is spray from the collision between the ball and the ground. This spray was along the baseline, so the high home and left field cameras couldn’t distinguish it from the white background of the line itself. The third base ump had the same problem. It turned out he had a bad angle for viewing the spray.

I hope this discussion illustrated the physical limitations of replay reviews based upon constraints of video cameras. Given these limitations, it is surprising over the first two months of the season, almost half the challenged calls had been reversed. I will try to be less of a curmudgeon in the future, but that may just be wishful thinking.