Welcome back to DevOps like It’s 1999, an ongoing series about integrating modern DevOps tools into a semi-ancient online multiplayer game codebase. In the last article, we added unit test support for the codebase, and integrated it into Travis-CI. In this article, we will be setting up code coverage reporting in Coveralls in order to stay on top of our unit tests.
Coveralls is a code coverage dashboard that tracks how much of a codebase is covered by unit tests. Code coverage tracking is fascinating. The basic idea is that it analyzes the code in our codebase, and then when we run our test executable, it compares which lines of the code end up getting run. As the code changes, it is reanalyzed, and the change is recorded and displayed in the Coveralls dashboard.
Coveralls can also integrate with GitHub pull requests by providing status checks before the PR gets merged. This is a common feature amongst unit testing services, but is especially useful as a way to ensure all new code is covered by new tests.
So, how exactly do we set up our code coverage analysis in this codebase? It’s actually relatively easy! Gcov is a C source code coverage analysis tool that will allow us to profile our code. Luckily for us, gcov is available via the Ubuntu package manager as well as the package manager that Travis-CI uses.
Before we can send any data up to Coveralls, we first need to update our test suite Makefile to generate our gcov profiles. This can be accomplished by adding a few things. The first is to add -fprofile-arcs -ftest-coverage to our c flags, and the second is to add -lgcov -coverage to our l flags. These flags, when coupled with our test executable, will generate coverage profiles for our code that can then be used to send data up to Coveralls.
Now that we can generate our test coverage profiles, we need to update our Travis-CI config to install and run the Coveralls analysis after we run compile and run our test suite. Adding the following lines to our .travis.yml will accomplish this:
- pip install --user cpp-coveralls
- coveralls --exclude tests --gcov-options '\-lp'
Basically, what this snippet does is install the Coveralls C tool before compiling and running our test suite, and then actually runs the Coveralls tool if the test suite succeeds. Now, every time our tests are run in Travis-CI, a corresponding coverage check will be run, and the results will show up in the Coveralls dashboard. Now that we have unit tests and code coverage analysis for OASIS, this project is starting to look new again. We have a long way to go, though, so stay tuned for more attempts at sticking round pegs in square holes.