Analyzing Videos for Fun and Profit

Analyzing Videos for Fun and Profit


Joey Bernard
Wed, 07/05/2017 – 05:46

People’s phones and all of the various sensors that may be built in to them
is a
source of scientific data logging that almost everyone carries around.
Although the selection of sensors varies from phone to phone,
they almost all have a camera. In this article, I take a
look at a piece of software called Tracker that can be used to analyze
videos you take of experiments.

You can download Tracker directly
from the project page.
A lot of good documentation is available at the main website, including
examples of how you might be able to use it from the community of other
users. Tracker is written in Java, however, so you also need to
have a JVM installed on your system before you can use it. For example,
on a Debian-based system, you can install a very good JVM with the command:

sudo apt-get install openjdk-8-jre

You then can download the relevant installer for your system and run
it from a terminal window. You probably will need to make the installer
executable with a command like this:

chmod +x

Don’t forget to use the sudo command so that you have the correct
permissions to do the installation.

Once it’s installed, you should have a new entry in your applications
menu system. Starting Tracker will give you a new empty project for
your video analysis.

Figure 1. When you start Tracker, you get a new empty project.

Although you could start analyzing your own videos immediately, you may want
to use simpler videos while learning how Tracker
works. When you ran the initial installer for Tracker, it should have
asked you whether you also wanted to install example files. Assuming you
said yes, you now can load one of those within Tracker. When you click
on the menu item File→Open File, you’ll see a new window where
you can select either a movie file or a Tracker file.

Figure 2. The open file dialog box allows you to open movie files or
Tracker files.

To begin with, let’s look at one of the Tracker files from the
experiments folder. In Figure 3, I have loaded the experiment
file named BallToss.trk. The main window displays the
movie that is being analyzed as part of the experiment. The pane in the
bottom right-hand side shows a table of x and y coordinates for a series
of time units. A similar table is generated for each tracked object within
the movie. In this example experiment, the only tracked object is the
ball. The top right-hand pane contains the associated plot for the data
stored in the table. This way, you can have a visual representation of
the tracked motion.

Figure 3. Loading a Tracker file opens all of the parts of a previous

So, what can Tracker actually track? At the bottom of the movie display is
a set of controls. If you click on the green arrow, the movie will start to
play. As it does, Tracker actually registers the movement of the tracked
object and also updates the data panes on the right-hand side.

Figure 4. As the movie progresses, Tracker updates the data panes on
the right-hand side for the tracked object.

Although these panes do give you lots of raw data, there really isn’t any
analysis being done yet. Clicking the View→Data Tool menu item pops
open a new window where you can use this raw data to figure out what is
happening in the video.

Figure 5. The Data Tool window provides a set of analysis tools to
look at the raw data pulled from the video.

The Measure and Analyze buttons at the top of the plot window provide
a set of different sections that you can display. This gives you several
different analysis options for your data. Clicking the Measure
button gives you the options Coordinates, Slope and Area. When you
select one or more of those options, values are displayed
at the bottom of the plot representing each of the selected options.

When you click the Analyze button, you get a drop-down with the
Statistics, Curve Fits and Fourier Spectrum options. Each of
those options actually opens a new pane in the Data Tool window. The
exception is the Fourier Spectrum option. This option actually opens
a new window where a Fourier analysis is done on the raw data from the
tracked object.

Figure 6. Tracker also can do a Fourier analysis of the observed

The Statistics option gives the data displayed in the top right
pane. This is the set of descriptive statistics—that is, the maximum,
minimum, mean and standard deviation, among other values.

Selecting the Curve Fits option gives you the pane at the bottom of
the plot window. Here you can select what type of curve is being fitted
to your raw data. You can use a number of default functions
to try to fit your data. In this case, the linear function is the
obvious choice for a fitting function. You can see the function being
used, x=A*t+B in this case, along with the calculated values for the
parameters A and B. If you are tracking an object that has a really odd
motion, you can click the Fit Builder button to pop up a new window
where you define a new fitting function.

Figure 7. You define your own fitting functions as part of the data

What do you do as part of a new project? Let’s say you want to analyze
the pendulum video from the sample video folder. You can set some
calibration items within the video by either selecting the relevant
buttons at the top of the window or selecting the items under the menu
entry Track→New→Calibration Tools. You can set up items like a calibration
stick or a reference set of axes.

Figure 8. You can set up calibration tools within your video to be

One way to do the analysis is to step through the video, frame by frame,
selecting the object to be tracked within each frame. This may be the only
way to collect the raw data, but humans are lazy. If the object being
tracked is relatively consistent across the duration of the video clip
in question, you can try using the autotracker. Clicking the autotracker
button will pop up a new window where you can control what is being

Figure 9. The autotracker tries to follow an object
from frame to frame automatically.

Tracker helpfully displays reminder information on how to
use the software. In this case, it reminds you that you need to
press Ctrl, Shift and left-click the mouse button on the object of
interest. If you then click the search button within the autotracker
window, it will move through each frame, looking for the object in
question. Once it is done, you should have a full set of data from the
tracked object. From the plot in Figure 10, you can see right away that
the pendulum follows the expected motion. You also can track multiple
objects by creating a new track for each object.

Figure 10. The autotracker can generate all of your raw data

Now that you’ve taken a quick look at Tracker and the kind of analysis
you can do with it, you should be comfortable enough to start
experimenting with it. Don’t forget to share any interesting ideas you come
up with on the community page and add to a great resource for citizen