Driving Boost.Test with CMake

Jun 02, 2015 - C++ Boost CMake

Boost.Test is one of the more popular C++ based test frameworks on the market. With it being a boost library, chances are its already installed in the local environment, or can be easily built. This article doesn’t aim to show how to use Boost.Test. Rather, it shows how to incorporate building and running those tests with CMake, one of the most popular build systems for C++ projects. This article assumes that a CMake based project already exists.

To begin, Boost.Test must be included in the project.

find_package(Boost COMPONENTS unit_test_framework REQUIRED)

Next, create a text file called BoostTestHelpers.cmake. This file will house a CMake helper function, add_boost_test which will be used to generate a test runner usable by CTest from a single test source file. This function takes a single source file containing Boost.Test unit tests and optionally a library dependency to link against. In cases where the library being tested is header-only, the dependency parameter is not required. In this file, create a skeleton of the add_boost_test function.

function(add_boost_test SOURCE_FILE_NAME DEPENDENCY_LIB)

	# Function body here

endfunction()

This function takes the base name of the given source file and uses it as the executable name of test runner. For example, if SomeUnitTests.cpp is passed to the add_boost_test function, then the test runner executable will be named SomeUnitTests. Then the executable is added and linked to the library being tested, if it’s not a header-only library, and the Boost.Test library.

get_filename_component(TEST_EXECUTABLE_NAME ${SOURCE_FILE_NAME} NAME_WE)

add_executable(${TEST_EXECUTABLE_NAME} ${SOURCE_FILE_NAME})
target_link_libraries(${TEST_EXECUTABLE_NAME} 
                      ${DEPENDENCY_LIB} ${Boost_UNIT_TEST_FRAMEWORK_LIBRARY})

By default, the CTest runner executes and reports on the tests being run at the executable level. This means if the example SomeUnitTests.cpp file contains 50 tests, CTest only reports whether the entire file passes or fails. An example is shown here.

Running tests...
Test project /home/projects/testproj/build
    Start 1: SomeUnitTests
1/5 Test #1: SomeUnitTests ...   Passed    0.01 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.02 sec

While of some use, it would be more useful if CTest reported the names of each test it runs. To accomplish this objective, the test file is read and a regex used to search for all Boost.Test test cases and scrape the names. The tests names are stored in the array FOUND_TESTS where they can be used to add the individual tests.

file(READ "${SOURCE_FILE_NAME}" SOURCE_FILE_CONTENTS)
string(REGEX MATCHALL "BOOST_AUTO_TEST_CASE\\( *([A-Za-z_0-9]+) *\\)" 
       FOUND_TESTS ${SOURCE_FILE_CONTENTS})

Finally, a loop is performed over all of the found tests. Any excess characters are stripped from the test name and then finally the test is registered with CTest.

foreach(HIT ${FOUND_TESTS})
    string(REGEX REPLACE ".*\\( *([A-Za-z_0-9]+) *\\).*" "\\1" TEST_NAME ${HIT})

    add_test(NAME "${TEST_EXECUTABLE_NAME}.${TEST_NAME}" 
             COMMAND ${TEST_EXECUTABLE_NAME}
             --run_test=${TEST_NAME} --catch_system_error=yes)
endforeach()

Adding the tests to CMake in this manner will generate more informative test output. For example, if SomeUnitTests.cpp contains three tests labeled Test1, Test2, and Test3, the output from tests added via add_boost_test will look similar to the following.

Running tests...
Test project /home/projects/testproj/build
    Start 1: SomeUnitTests.Test1
1/5 Test #1: SomeUnitTests.Test1 ...............................   Passed    0.01 sec
    Start 2: SomeUnitTests.Test2
2/5 Test #2: SomeUnitTests.Test2 ...............................   Passed    0.01 sec
    Start 3: SomeUnitTests.Test3
3/5 Test #3: SomeUnitTests.Test3 ...............................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 3

Total Test time (real) =   0.04 sec

The final product is a simple CMake function that can be called as simply as follows, where SomeUnitTests.cpp is a file containing Boost.Test unit tests and my_library is the library containing the code being tested.

include(BoostTestHelpers)
add_boost_test(SomeUnitTests.cpp my_library)

Below is the completed add_boost_test helper function. Additional improvements would be to support an arbitrary number of dependency libraries, however, the function as is can be a useful tool for easily incorporating Boost.Test unit tests into a CMake build environment.

function(add_boost_test SOURCE_FILE_NAME DEPENDENCY_LIB)
    get_filename_component(TEST_EXECUTABLE_NAME ${SOURCE_FILE_NAME} NAME_WE)

    add_executable(${TEST_EXECUTABLE_NAME} ${SOURCE_FILE_NAME})
    target_link_libraries(${TEST_EXECUTABLE_NAME} 
                          ${DEPENDENCY_LIB} ${Boost_UNIT_TEST_FRAMEWORK_LIBRARY})

    file(READ "${SOURCE_FILE_NAME}" SOURCE_FILE_CONTENTS)
    string(REGEX MATCHALL "BOOST_AUTO_TEST_CASE\\( *([A-Za-z_0-9]+) *\\)" 
           FOUND_TESTS ${SOURCE_FILE_CONTENTS})

    foreach(HIT ${FOUND_TESTS})
        string(REGEX REPLACE ".*\\( *([A-Za-z_0-9]+) *\\).*" "\\1" TEST_NAME ${HIT})

        add_test(NAME "${TEST_EXECUTABLE_NAME}.${TEST_NAME}" 
                 COMMAND ${TEST_EXECUTABLE_NAME}
                 --run_test=${TEST_NAME} --catch_system_error=yes)
    endforeach()
endfunction()

See Also