As Qualification Lead Engineer, my main role at Solid Sands is leading compiler qualification projects as part of our Compiler Qualification Service. In the four years I have worked here, I have seen many compilers, many compiler defects, and many customers surprised to learn that their compiler is not entirely up to scratch. But how does our Compiler Qualification Service work? And how do we help our customers to use their compiler safely, despite its defects? In this blog, I will walk you through the process and share some important lessons learned.
Matching your software development environment
When qualifying a compiler on your behalf, our qualification environment needs to match your software development environment as closely as possible. We don’t need your application’s source code, but we do need to use your compilation environment (the SDK) and the target execution environment for your application. You could, for example, choose to send us a small evaluation board containing your target processor, but we also regularly work with full-fledged industrial machines that occupy large areas of desk space, as illustrated in the picture above. As long as we can run the test executables generated by the compiler and verify the results, we will make it work.
Each compiler option impacts the behavior of the compiler and consequently its safe use. Compiler qualification therefore needs to be performed using your exact compiler configuration – your compiler’s target use case. While a compiler can be qualified for any use case, we often advise our customers to add some specific compiler options that contribute to the safety of the compiler. This minimizes the number of workarounds that may need to be put in place to safely use the compiler.
After specifying the use case, it’s time to put the compiler through its paces. Strictly adhering to the relevant ISO C or C++ language specification, we verify that the compiler correctly implements each and every detail of the language. But we don’t just test for compliance. We also run our exhaustive data-model-specific arithmetic tests, our optimization tests, and our calling convention tests to make sure that we really test the compiler inside out.
Safety manual
The end product of our Compiler Qualification Service is a comprehensive qualification report that matches the requirements of the applicable functional safety standard*. However, don’t expect this report to categorically state that your compiler works flawlessly, because that is not the goal of compiler qualification. The goal is to make you aware of any compiler defects so you can ensure they don’t impact your application. Each qualification report comes with a detailed safety manual that lists the compiler defects that we found, together with mitigations (workarounds) you can put in place to use the compiler safely.
Don’t worry, the safety manual won’t come as a sudden surprise. Throughout the qualification process, we’ll keep you up-to-date with our findings and verify that you agree with our proposed mitigations. It means that by the end of the qualification process you’ll be ready to implement the compiler safety manual as smoothly as possible.
Start early
Having qualified many compilers, one of the most important lessons I have learned is that you really shouldn’t let compiler qualification wait until the end of your application development. While the qualification process itself only takes around 6 to 8 weeks, implementing the compiler safety manual takes time. The earlier you are aware of compiler defects, the less effort it will take to mitigate them. By starting early, you won’t run the risk of missing a deadline because some unexpected compiler error turned up and needed mitigating.
Here’s what one of our compiler qualification customers had to say about the service:
“It was a very good collaboration, not just in terms of performing the compiler qualification but also in terms of explaining how everything worked. Solid Sands also identified very early on that if we changed the optimization level in the compiler, it could drastically reduce the number of rules we needed to adhere to and they were quite happy to stop work for several days while we sought internal approval for the change. When we analyzed the qualification report they provided, they also offered us support and advice when we had questions.”
* Over the years, we have performed many compiler qualifications following functional safety standards such as ISO 26262 (automotive), EN 50128 (rail) and IEC 61508 (industrial).
Remi van Veen, Qualification Lead Engineer
Compiler Qualification Service
Subscribe to our monthly blog!