Staying on top of state-of-the-art software testing methods is important for our work on functional safety. Therefore, innovation is one of the four core values at Solid Sands.
We achieve this by continuously delivering improvements and new developments to the market, while internally, giving students an opportunity to collaborate with us on thesis projects. Working with interns on thesis projects allows us to experiment with new ideas and projects we wouldn’t prioritize otherwise. One other exciting benefit is that interns can move into full-time roles. For example, five of our eight software engineers started as thesis project interns.
An example of such a project was the development of the test framework for functions in the <atomic> header from C++. The <atomic> header requires special attention in testing because its functionality appears only in multi-threaded programs, and multi-threading introduces non-determinism. Non-deterministic test results are hard to reproduce and hard to analyze.
An intern from the University of Amsterdam worked on designing and testing this new framework. Our goal was to develop an implementation-independent framework that could also be executed on the target device. Due to this requirement, the approach couldn’t depend on methods that rely on emulation or any form of scheduler manipulation. So, instead, it was decided to focus on measuring the thread interleavings observed (when the interesting action happens) in the testing program. Then, the framework could repeatedly run the test program until it reaches the desired number of observed interleavings. It generates reproducible results and, we believe, is simple and small enough to be analyzed in case of errors.
Academia and industry goals don’t always align. While we are satisfied with a straightforward working solution, practical implementation alone is not sufficient for a Master’s thesis research. So, with the intern, we concentrated on demonstrating that when the test framework detects interleaving, the threads were indeed interleaved. We introduced a bug in the musl library’s implementation of the compare-and-swap operation which triggers only when multiple threads attempt to access it simultaneously. Then, we measured how many failures occurred in the program runs, detected as “interleaved” compared to “non-interleaved” runs.
To learn more, visit the bottom of the Our Team page, where you can find this project and other thesis intern projects we have completed.
Our in-depth test suite for the C++ <atomic> header is ready for use in SuperTest 5.1 and SuperGuard 1.3.
Vlad Yaglamunov, Software Engineer