TDD for embedded platforms – Practical application and framework comparison
Two frameworks with different approaches enable test-driven development for embedded applications: GTest and CppUTest.
(Image: Bogdan Vija / Shutterstock.com)
The development of embedded software is becoming increasingly complex. Thanks to more powerful microcontrollers, networking and higher demands on security and reliability, firmware projects often hardly differ in their complexity from classic software projects on a PC. However, the special framework conditions for embedded applications still apply: strict resource limits, direct hardware dependencies, real-time requirements and the need to make a system stable and error-free even in very early development phases.
This is where test-driven development (TDD) can help. TDD is a development practice from the agile environment that deliberately places the writing of tests before the implementation of the actual functionality. The basic idea: whoever formulates the test first knows exactly what the code should do later. As soon as this test fails (red phase), you implement the minimum necessary code (green phase) to make the test pass. The code is then improved and cleaned up (refactor phase). This cycle is repeated over and over again. The result is clean, comprehensible and well-tested modules that feel more secure and have fewer errors.
Videos by heise
Test first, then implement
TDD is a technique from the field of agile software development in which the test is written before the actual code implementation. This sounds contradictory at first – How can you test something that doesn't even exist yet? But this is precisely the strength of TDD: the test serves as a requirements definition, as a specification of the functionality. The initial test describes how the function should behave. Logically, this test fails at first because the function has not yet been implemented. This is the red phase.
Next, you write the minimal code to make the test pass. This phase is called the green phase. As soon as the test is green (i.e. has passed), the refactor phase starts, in which developers improve, simplify and optimize the code. The tests ensure that the functionality remains unchanged.
This cycle – Red-Green-Refactor – is run through again and again. The result: the code is covered by tests at all times, the requirements are clear and comprehensible and the development team gains confidence in the quality of the software. This confidence is invaluable, especially in the embedded environment, where debugging on real hardware is often time-consuming or only possible at a late stage.
Test-driven development offers the following advantages:
- Early error detection: bugs are detected before they propagate through the code.
- Structured way of working: The requirements (tests) are clearly in front of you, no "programming away" from the actual goals.
- Better maintainability: The tests serve as living documentation. Other developers (or you yourself after a few months) understand more easily how functions are intended.
- Higher code quality: As you only write as much code as is necessary to pass the tests, there is less unnecessary and error-prone ballast.
Special features of embedded development
Unlike desktop or web applications, embedded software is often tied to specific hardware. A sensor, an actuator or a specific peripheral register determines whether a function works correctly. This makes testing more difficult, as the query of a GPPIO pin cannot be easily executed as code on the PC.
This is where an important TDD practice comes into play: abstraction and mocking. Instead of accessing hardware registers directly, interfaces are defined. For production, this interface accesses the real hardware, and in the test environment, a mock or stub replaces the hardware. As mocks simulate the environment, the same code can run under test on the PC. This allows many errors to be found early and independently of the target device.
Embedded systems often have little RAM and flash memory. A comprehensive testing framework cannot run directly on the target system, but needs a PC as a host. The code running in the tests is abstracted to such an extent that it can run both on the PC and (theoretically) on the target. This means that large parts of the logic and algorithms can be tested early and conveniently, even without an embedded debugger.
Real-time requirements and timing are further aspects: Many embedded functions depend on timing sequences. TDD helps here too: timers and timing functions can be mocked to perform deterministic tests.
Google Test (GTest) and CppUTest in comparison
Two different testing frameworks are used in this article: Google Test and CppUTest. Both enable test-driven development of embedded software, in particular by executing the tests on the host. However, the frameworks have different objectives and design philosophies.
Google Test (GTest) is a C++-based unit test framework that was originally designed for desktop and server applications. It offers an extremely powerful, but also complex API. The assertions are numerous and finely granular (e.g. EXPECT_EQ, EXPECT_NEAR, ASSERT_THAT with matchers) and there are mechanisms for testing exceptions (death tests), parameterized tests, test files, test fixtures and much more.
GTest was originally aimed primarily at C++ developers. As C is often used for embedded projects, C source code has to be integrated via C++ test drivers. This is not a fundamental problem, but an additional expense, as C headers have to be included in extern "C" blocks. In addition, GTest usually requires a C++ standard library and a more modern compiler. As both are often not available for embedded targets, GTest usually only runs on the host.
Integration into the project
GTest can be integrated into a project in three ways:
- Package manager or predefined packages: Many build systems such as CMake offer
FetchContentorExternalProjectmechanisms to pull GTest from the GitHub repository directly into the project. - System-wide installation: Under Linux distributions, GTest packages can often be installed from the repositories. Under Windows, Vcpkg or Conan are available.
- Vendoring: The GTest repository can be integrated into your own repository as a submodule.
Teams must ensure that their compiler and build environment are designed for C++. The GTest library is extensive and usually requires the standard library. This is not a problem on the host, but it certainly is on a pure embedded target. The tests should therefore run host-based.
Ideally, you should work with a CMake structure in which you separate test directories and link the GTest library in CMakeLists. The following code shows a typical procedure:
FetchContent_Declare(
googletest
URL https://github.com/google/googletest/archive/refs/tags/release-1.12.1.zip
)
FetchContent_MakeAvailable(googletest)
add_executable(tests test_led.cpp test_sha.cpp)
target_link_libraries(tests gtest_main)
If you want to use the GMock mocking framework, which is part of GTest, to simulate hardware interfaces, you must convert the functions to be tested into a form suitable for GMock – often classes and virtual methods. For pure C functions, this can mean more effort.
GTest can output test results in JUnit format, which simplifies integration into CI systems such as Jenkins or GitLab CI.
Best practices and stumbling blocks for GTest
Fixtures are practical, but you should use them sparingly and not make them overly complex. Simple SetUp/TearDown methods and tests that are as isolated as possible help to keep an overview.
Parameterized tests are great for testing algorithms against lots of input data, such as cryptographic functions with different test vectors.
GTest is rarely the first choice for low-resource targets that need to run tests directly on the device. It is extensive and requires C++ runtime functions.
Beware of over-engineering: With GTest, there is a risk of creating extremely complex test hierarchies. This can be counterproductive for embedded modules that need to be tested quickly and easily.
Lean testing with CppUTest
CppUTest is a lean testing framework for C and C++ that is particularly popular in embedded development. It comes with minimal dependencies, does not require an extensive standard library and is easier to port to embedded-like toolchains. The assertions are simpler (e.g. CHECK_TRUE, LONGS_EQUAL, STRCMP_EQUAL), but flexible enough for most use cases. CppUTest is deliberately kept minimalistic.
Another advantage is the close integration with CppUMock, an easy-to-use mocking framework for C functions. This proves to be particularly useful in embedded development, where mocking hardware accesses is elementary. With calls such as mock().expectOneCall("ReadAdc").andReturnValue(123); you can simulate functions without having to write complex wrappers.
Integrating CppUTest into the project
CppUTest is usually integrated by downloading or cloning the CppUTest repository. The best practice is to add CppUTest to a project as a submodule or static library.
CppUTest works together with CMake. Teams can drag the source code of CppUTest into their project and then integrate it with add_subdirectory(cpputest). The following code also links to the test targets:
add_subdirectory(cpputest)
add_executable(tests test_led.cpp test_sha.cpp)
target_link_libraries(tests CppUTest CppUTestExt)
If you want to run tests on a special toolchain, for example for an embedded Linux target or even directly on the microcontroller, you can try to build CppUTest with a cross toolchain. This is often easier than with GTest due to the sparse dependencies.
CppUTest provides fewer special assertions, so that highly complex comparisons may require their own auxiliary functions.
CppUTest is simpler than GTest when it comes to reporting. JUnit reports are possible, but may require their own scripts. If you need complex reporting functionalities, you may have to invest in manual work.
CppUMock is ideal for mocking C functions. Mocking is carried out via a central instance (mock()). When implementing the test functions, it is important to ensure that the function signatures match exactly. Variadic functions or inline functions, which are more difficult to mock, are sometimes a stumbling block.
Best practices and stumbling blocks for CppUTest
Keep it simple: Just like the framework, the tests should remain simple and comprehensible.
Mocking should begin as soon as hardware-dependent functions exist in the project. It is important to clearly define the interfaces.
CppUMock allows you to define expected calls and parameters. Teams should use this consistently to ensure that their code calls the expected functions with the correct parameters.
In the event of failures, CppUTest displays very simple error messages. To localize problems quickly, test names and outputs must be unique. Don't be afraid of your own assertions: If you need to test complex data structures such as large byte arrays or cryptographic outputs, you should write your own auxiliary functions or assertions so that the comparisons are meaningful.
