To many, this statement sounds like an open door. Compiler generated code is more complicated because it includes duplicated source code for better performance. But how come, when it comes to safety testing according to the ISO 26262 standard, they are often seen as the same?
Safety is paramount in the automotive industry. That is why the ISO 26262 standard, which is the international standard for functional safety of electrical and/or electronic systems in production cars, has extensively detailed the testing of safety critical elements, both for hardware and software components.
One of the regulations in the ISO 26262 stipulates that application software tests must have the same behaviour on the (simulated) test environment as well as on the target hardware, with the compiler ‘in-the-loop’. It should verify that all statements in the source code are tested, as well as the branches in the code. Together, statement and branch coverage testing is called MC/DC analysis.
These are strong testing requirements that help get the application right. But they are not strong enough to guarantee that all possible errors in the compiler are detected.
Always qualify compilers
Even though application testing is done rigorously, you should still qualify the compiler. For the simple reason that source code isn’t compiler generated code.
Compiler generated code is much more complicated, because optimizations and transformations do not preserve the structure of the source code. To improve performance, code is duplicated and specialized, for example by loop unrolling. The generated code is larger and has a much more complicated branch structure because of it.
As a result, tests that are sufficient for MC/DC analysis of the source code and application models, will not provide sufficient instruction and branch coverage of the generated machine code. This is because the compiler generated code can contain instructions that are not verified by the testing framework. In other words, errors in this code will not be detected by this type of testing. For this reason, compiler qualification is needed to make sure that the generated code has no errors.
This has other benefits as well, such as reducing the emphasis on on-target testing in the application test-process. Compiler qualification can be separated from application testing to reduce the critical path to application deployment.
SuperTest
That is why Solid Sands recommends the SuperTest validation suite for your compiler qualification needs. If you want to know more about testing and compiler qualifation, contact us. We will be happy to explain more extensively why you should qualify C and C++ compilers independently, and what other benefits it has.
Because you want to make sure that you find errors before they become a problem.
Subscribe to our monthly blog!