Improving motion detection

Motion detection is an important feature in SprintTimer. In Video Finish it helps to reduce the number of frames that you have to shift through. In Live Finish it is the key factor for measuring time. So I thought that explaining how it works might help you apply it successfully. The first thing to realize is that it not really is motion detection, but rather change detection; the motion creates changes in the image that are detected.

SprintTimer captures a video frame and calculates the color distribution within the detection zone. Then it captures the next frame, does the same calculation, and compares the two. If the difference is large enough it assumes that there has been a motion and saves the frame. This can be illustrated by the simplified figure below. A is one frame captured by the camera. B is the next frame, say 0.1 s later (if we are running at 10 fps). In C the difference between the two images is shown in grey to make the difference clearer. If we look in the yellow detection zone, we can see that about 10 % of the pixels have changed from white to blue. This means that the color distribution has gone from 45 % blue and 55 % white to 55 – 45. These numbers can be used to calculate a correlation between the two, let us call this number BC*. Here BC = 1 means perfect correlation, i.e. no change and thus no motion. BC = 0 means complete change and a lot of motion.

If you go to the Preview in Video and Live finish, you will find a slider with a value above it. This is the BC value that the app uses to indicate motion. The default value is 0.85, which means that the difference has to be large enough to create a BC below 0.85 for the motion detection to be triggered. So pushing up the slider will increase the sensitivity (less change needed) and setting a lower value will make it less sensitive. When the app detects motion, the detection zone will change color and there will be a second number above the slider. The latter shows the BC value for the latest detection.

If we zoom in on picture C and test some scenarios, we get the following figure (click to make it larger). Let us assume that I am standing 2 m away and the camera runs at 10 fps. The first image is the same as the one above. In the second I have doubled the frame rate. This means that the runner has moved half distance and the number of pixels that have changed is only half. In the third image, the frame rate has been kept low, but the distance to the runner has doubled. The effect is roughly the same as with an increased frame rate. In the last image, both the distance and the frame rate have been doubled. You can see that the BC value rapidly increases towards 1, which means that the chance to capture the runner is greatly reduced.

All this leads to some practical suggestions to avoid missed competitors when using Video and Motion Finish:

  • Moving closer, 2-4 m is a recommended distance for runners.
  • If you are unable to get close enough, go to the preview and zoom in to get the competitors to better fill out the frame.
  • Lower the frame rate, you seldom need 60 fps, unless you are dealing with very fast objects. If you want to increase the accuracy of the timing, you can use the Dual-mode when evaluating the race.
  • In the preview, you can increase the detection sensitivity (but standing closer and/or lowering the frame rate is better).
  • You use Human detection as an alternative.
  • In extreme conditions, consider turning off the motion detection.

Why use this technique?
An advantage of the color distribution method is that it not is sensitive to a small overall motion, i.e. you can hold the iPhone by hand if you hold it reasonably still. Another advantage is that it is fairly fast, which means that it can be run on a smartphone 60 times/s while recording video. It is also independent of which type of object is measured and what direction it is taking.

A disadvantage is that it reacts to all kinds of changes, for example, shifts in the light. This becomes especially pronounced in water sports like swimming, rowing, and kayaking, where glitter in moving water can trigger the detection. Another case is that some lamps like LED have a light that flickers faster than the eye can see, but that the camera will react to. It also works best when the measured object contrasts against the background, for example, are black bike tires against black asphalt not easily detected.

I also want to emphasize that Video Finish was designed to complement Photo Finish in cases where the finish is spread out and where you usually can stand fairly close by. Applying it to distant or spread-out finish lines is not optimal.

*BC for Bhattacharyya Coefficient