It may sound as too obvious, but there is a big difference between the source code and the compiler generated code for an application. For one, compiler generated code is more complicated, because it often includes duplicated, and specialized, source code. So, why is it that when it comes to safety testing code according to the ISO 26262 standard, they are often treated as the same?
One of the most important things in the automotive industry is safety. That is why the industry introduced strict regulations and guidelines on how applications should behave. They are specified in the ISO 26262 standard; the international standard for functional safety of electrical and/or electronic systems in production cars. It details extensively the testing of safety critical elements, both for hardware and software components.
One of the regulations in the ISO 26262 stipulates that application software tests must have the same behaviour on the (simulated) test environment, as they will have on the target hardware. And that the compiler is ‘in-the-loop’. Furthermore, it should also verify that all statements in the source code are tested, and that the branches in the code are as well. This coverage testing of statements and branches is called MC/DC analysis.
Always qualify compilers
These rigorous testing requirements help you get the application right and safe. But they are not rigorous enough to guarantee that all possible errors in the compiler code are detected too. So, no matter how rigid applications are tested, you always should qualify the compiler independently. Why? Because source code is not the same as compiler generated code. It is more complex and therefore more prone to contain errors.
That is because optimizations and transformations, which are made by the compiler, do not preserve the original structure of the source code of the application. For instance, to improve performance, parts of the source code are duplicated and specialized for one specific task that loops. The generated code becomes larger and will have far more complicated branch structures because of it.
Tests that are sufficient to perform MC/DC analysis of the source code and application models, do not provide sufficient instruction- and branch coverage of the generated machine code. One reason is that compiler generated code might contain code sequences that are not verified by the testing framework. In other words, errors in this code will not be detected by this type of testing. That is why compiler qualification is necessary to make sure that the generated code has no errors.
Testing compiler generated code has other benefits as well, such as reducing the emphasis on ‘on-target testing’ in the application test-process. Also, compiler qualification can be done separate from application testing. This reduces the critical path to application deployment.
This is why Solid Sands provides the SuperTest validation suite for C and C++ compilers for your compiler qualification needs. It detects errors in the generated code and helps your application meet the ISO 26262 standard. The new optimization test suite in SuperTest is built specifically to achieve the highest generated code coverage in the presence of code transformation. If you want to know more about testing and compiler qualification, contact us. We are more than happy to explain more extensively how to qualify C and C++ compilers independently, and what other benefits there are.
Because you want to make sure that you can find errors before they become a problem.