Sunday, March 20, 2016

Unit Testing FRC C++ with CppUTest in Eclipse

Introduction

Participating as a programming mentor in this year's FIRST Robotics Competition has been a lot of fun, but I found myself wishing, once again, for better ways to test our robot's code. The existing methods are the traditional direct tests on the actual robot hardware, and simulation using Gazebo. Directly testing the robot is often difficult to even get started with, since the robot is not constructed for much of the 6-week build season. Once it is constructed, though, deploying builds and manually working the controls to verify functionality is slow and error-prone. Working with the simulator also has its own shortcomings. A realistic SolidWorks model is required to drive in a virtual arena, and the simulator itself consumes a lot of processor power and memory.
  Fortunately, there is another way, and that way is unit-testing. Unit tests are small, simple tests written in code (often in the same language as your production code) that set up inputs, exercise some part of the production code, and verify the outputs that are produced. These tests can run directly on the programming computer, no robot hardware required, and execute very quickly. For Java and C++, there are several great frameworks available for free to make the process of writing unit tests easier. In this post, I will describe how the CppUTest framework can be used to make unit tests for a C++ robot program based on WPILib. CppUMock, which is a part of CppUTest, is also used to create a mock version of WPILib that can be precisely controlled in the unit tests. Example code based on the 'Arcade Drive' sample project is available at GitHub.
  Please note that this strategy does not require any modifications to your existing robot code. The only change to your project will be an additional build configuration that allows it to be tested this way. Switching between the default configuration and unit-testing configuration (described below) is easy and fast.

Requirements

The following utilities are required to develop FRC programs with CppUTest.
  1. Eclipse with C/C++ development plugins (eclipse.org)
  2. CppUTest (github.com/cpputest/cpputest) (release 3.7.2 was used for this example)
  3. MockWPILib from the example code (unless you want to write your own mocks)
  4. Unix development tools (if developing on Windows, use Cygwin (cygwin.com)
    1. autoconf
    2. autogen
    3. automake
    4. make
    5. gcc-core
    6. gcc-g++
    7. libtool

Building CppUTest

The first step is to acquire the CppUTest sources either from a release package or the git repository. If you're using Cygwin, the sources should be placed somewhere in your Cygwin installation path (ex: C:\cygwin64\home\user). Then, using a bash shell (or Cygwin Terminal), navigate to the CppUTest root directory and run the following commands, in order:
  1. ./autogen.sh
  2. ./configure
  3. make
If you have all of the packages listed in the requirements section, this should generate the files libCppUTest.a and libCppUTestExt.a in the lib/ subdirectory. If the process is interrupted with errors, then one or more packages are probably missing and should be installed before continuing.

Building WPILib Mocks

The next step is to use CppUMock (included in CppUTest) to begin writing the mock classes you need for your tests. These will be located in a separate project in the same workspace as your main robot project. To create your own mocks project, select 'File > New > C++ Project' and set the project type to 'Static Library > EmptyProject'. For Windows users, set the toolchain to 'Cygwin GCC'.


Click 'Next' to get to the 'Select Configurations' window. Then, click on 'Advanced Settings' to go to the Tool Chain Editor (the Tool Chain Editor for an existing project can be accessed in it's Properties window).


Make sure you're in the 'C/C++ Build > Settings' section (in the menu on the left). Under 'Compiler > Includes', add an include path that points to the 'include' subdirectory of your CppUTest installation (ex: C:\cygwin64\home\user\cpputest\cpputest-3.7.2\include). Click 'Apply' at the bottom of the window.

For Windows users, go to the 'Environment' section (select 'C/C++ Build > Environment' in the menu on the left). There should be an environment variable with the name 'CYGWIN_HOME' with an empty value. Set the variable to the path of your Cygwin installation (ex: C:\cygwin64). Leave the other environment variables unchanged.


Click 'OK' at the bottom to leave the properties window and click 'Finish' to create your mock library project. You may now begin writing the code for your mocks. To learn how to create mock classes, see the CppUMock documentation at the CppUTest site (cpputest.github.io/mocking_manual.html). For examples, see the MockWPILib project. In addition to mock classes, the START_ROBOT_CLASS macro should be defined as empty (#define START_ROBOT_CLASS ()).
  If users want to use and extend the MockWPILib project instead of creating their own mocks project, it can be acquired from GitHub and imported into the Eclipse workspace. To import a project, select 'File > Import > General > Existing Projects into Workspace'.


The build properties explained above should be checked after importing the project to make sure that the correct include path for CppUTest is set. If Cygwin is being used, the Cygwin installation environment variable should be set appropriately as well.

Robot Project Configuration

The next step is to add a build configuration to the main robot project that uses the mock library instead of the real WPILib library. First, go to the project's properties (right-clock on the project and select 'Properties'). In the 'C/C++ Build' section, click on 'Manage Configurations' on the right. In the 'Manage Configurations' window, click 'New' and give a name and description for the new configuration. Copy the settings from the 'Debug' configuration.


Click 'OK' to create the configuration, and then select it and click 'Set Active' in the 'Manage Configurations' window. Click 'OK' to go back to the properties window.


In the properties window, go to 'C/C++ Build > Tool Chain Editor' and set 'Current toolchain' to 'Linux GCC' for Linux users and 'Cygwin GCC' for Windows users. For Windows users, the Cygwin installation directory needs to be set as explained above in 'Building WPILib Mocks'.


After setting the toolchain, go to the section 'C/C++ Build > Settings'. Under 'C++ Compiler > Includes' add an include path for the mock library.


In the 'Project References' section, check the mock library project to indicate that the main robot project now references it.


Click 'OK' to exit the properties window. The main robot project can now be built with the new testing configuration. Be sure that all necessary mock classes and functions are defined in the mock library project to avoid errors. To switch the build configuration to link in the actual WPILib and deploy to the roboRIO, right click on the projects and select 'Build Configurations > Set Active > Debug'.

Writing Tests

Now that the mock library is written and the robot project is configured for unit testing. the unit tests themselves can be written. These tests will reside in a separate project. Create a new project by clicking 'File > New > C++ Project'. Set the project type to 'Executable > Empty Project'. Windows users should set the toolchain to 'Cygwin GCC'.


Click 'Next' and select 'Advanced Settings' to go the properties window. For Windows users, the Cygwin installation path should be set as explained in the section 'Building WPILib Mocks'.
  In the section 'C/C++ Build > Settings', include paths for CppUTest, the mock library, and the main robot project should be added under 'C++ Compiler > Includes'.


In the same section under 'C++ Linker > Libraries', library names and search paths should be added for CppUTest, the mock library, and the main robot project. The library for the main robot project will be called 'FRCUserProgram' and is located in the project's subdirectory for the testing configuration. For CppUTest, both the CppUTest and CppUTestExt libraries should be linked.


After the compiler and linker are configured, go to the 'Project References' section and check both the main robot project and mock library project. Then, click 'OK' to exit the properties window, and click 'Finish' to create the project.


The unit tests can now be written in the new tester project. To learn how to write unit tests with CppUTest, see the documentation on the CppUTest site (cpputest.github.io/manual.html). For examples, see the 'Arcade Drive Test' project on GitHub. Remember to define the 'main' function with a call to 'CommandLineTestRunner::RunAllTests'.
  Once the unit tests are written and built, the tester project can be executed to run the tests. Right-click on the tester project and select 'Run As > Local C/C++ Application'. The test results should show up in the console in Eclipse. If a test fails, the exact location of the failing assertion will be given.


  If an error concerning a missing library appears when running the tests on Windows, then the Cygwin libraries may be missing from the dynamic linker search path. To fix this, go to the 'Run/Debug Settings' section in the project properties. Select the executable in the launch configurations list and click 'Edit' on the right. In the 'Edit Configuration' window, select the 'Environment' tab and click 'New' to add an environment variable. Set the variable name to 'PATH' and the value to the 'bin' subdirectory of the Cygwin installation (ex: C:\cygwin64\bin). Click 'OK' and make sure that 'Append environment to native environment' is selected at the bottom.


Click 'OK' to exit the 'Edit Configuration' window. Click 'OK' to exit the properties window. The tester project should now be able to run.

2 comments:

  1. Just found this and have yet to dig in, but excited. Started mentoring this year and was hoping for something like this. Do you know if this works across years? Was specifically curious on the MockWPILib?

    ReplyDelete
    Replies
    1. Thanks for reading! The setup process should be pretty much the same for all years. I think MockWPILib needs to be updated, though, as well as filled in with the rest of the WPILib API. It's pretty bare since it only just has enough for the given Arcade Drive example. I've been spending more time on the RedBot-based FRC prototyping project lately (see http://www.botbakery.net/2015/06/introducing-learning-and-prototyping.html) since that seems more appealing to students than traditional C++ unit tests.

      Delete