/ agile

Facilitating TDD: Automating GTest

This isn't specific to embedded stuff (or even GTest) - but it does seem to work well for it. In this article I'm going to describe the setup I use for automating the build and execution of gtest tests.

For a long while now I've been using gtest for my testing framework. It's popular / actively maintained, and I like that mocks are built-in and easy to use. Also parallel execution looks sweet and is on my list of things to try. Gtest comes with CMake files for setting it up for your build system, but it also has a vanilla makefile buried within the repo - that's what I have used for small projects. I customize it for my tests, since usually I only want it to make the tests for the code that I'm working on.

Motivation

All by itself gtest is great, but I really wanted a setup where every time I make changes to a file:

  1. the relevant tests are compiled
  2. those tests are run
  3. the results are immediately displayed and I can give myself a pat on the back for not breaking anything (or go fix my mistake, which NEVER HAPPENS)

I was used to this when doing Rails work for my site - using Guard/minitest. But Guard seemed like it was too big for a small project I was working on (and it was an embedded C++ project) so I tried to find a more lightweight solution. Enter when-changed.

When-Changed

when-changed is a Python tool for monitoring one or more files / directories and then executing a command when it detects a change. The usage is simplicity itself:

$ when-changed FILE [FILE ...] -c COMMAND

I created two bash scripts - "watchtests" and "runtests" - to make my life a little easier. The first (watchtests) is to call when-changed with the proper arguments. The FILE arguments are the source/test directories (you can use -r for recursive watching), and the command that is executed is another bash script (runtests) that runs make and the executable(s) produced from it.

I start up a terminal, cd into my project's root directory, then:

$ ./watchtests.sh

and set the terminal window in one of my side monitors. Now every time I save a file my tests get run and I can immediately see the output. With this setup, it's very easy to get into a TDD rhythm.

You have to be a little careful when using directories with when-changed because every time a change is detected, your command will be executed. Depending on your editor and the usage of temporary files, this results in double (or triple/quadruple) the output that you would expect, which is really annoying. Luckily there is an option flag built into when-changed for just this sort of thing. The following is part of the help text for the tool:

-1 Don't re-run command if files changed while command was running

Add the -1 flag to your call and voila!

The gif below shows the setup in action.

Automatically run tests when a file is changed

Below are the contents of my bash scripts. Happy testing!

runtests.sh

#!/bin/bash

UUT=matrix_conversion_unittest
printf "\n\n"
  
make ${UUT} || exit
  
./${UUT}
  
printf "\n\n"

watchtests.sh

#!/bin/bash

SOURCES_DIR=./src
TESTS_DIR=./tests
  
when-changed -r -1 {${SOURCES_DIR},${TESTS_DIR}} -c ./runtests.sh

Addendum: Installing When-Changed

Per the readme for when-changed, these are the requirements:

  1. Python (I've used both 2 and 3 without any issues)
  2. pip (make sure it matches your version of Python)

You then use pip to get when-changed and all of its dependencies:

$ pip install https://github.com/joh/when-changed/archive/master.zip

This seems pretty simple right? Well, I managed to screw up my Python installation between using my OS package manager (both apt and homebrew) and then using Python's easy_install to install the Python package manager pip. Don't do this. If you download Python from the official website, you should download pip from that site as well. If Python comes with your OS, try to use your OS package manager to install pip. In the course of writing this article, I came to find out that's what the Python organization recommends anyway.

I'm still not sure what the Pythonically-correct way to install everything is, but getting everything through my OS package manager worked for me in the end. The important thing is to be consistent with how you install Python and pip.