As programmers, we all make mistakes. Writing first-time-right C or C++ code is very unusual. One important function of a compiler is to spot these errors, issue diagnostics and prevent object code generation. If a compiler fails to detect source code errors they will propagate to the application, where your test department or worse still your customer will eventually find them. In safety-critical applications it could have serious consequences.
In the context of software tools, Section 11.4.5.2(a) of the ISO 26262 Functional Safety Standard states that the intended usage of the software tool shall be analysed and evaluated to determine the possibility that a malfunction of a particular software tool can introduce or fail to detect errors in a safety-related item or element being developed.
In other words, you need to be confident that your compiler will generate a diagnostic message if your source code includes an error type that the compiler would be expected to find. One such error is the violation of a syntactic or semantic restriction of the language definition – normally referred to as a constraint violation. Despite the fact that both the C and C++ language standards define these constraints, we have found that many compilers, especially open source compilers or compilers used with their default settings, don’t always detect if they are violated. We know that’s the case because SuperTest includes a comprehensive suite of constraint violating tests, and we’ve found no compiler on the market, commercial or otherwise, that correctly diagnoses all of them. Every compiler we’ve tested exhibits ‘false negatives’ – errors that are not diagnosed – which poses a risk to safety.
Static Analysis Tools (and SuperTest!) to the rescue
Of course, most software developers don’t rely solely on the compiler to catch source code errors. They typically augment the compiler with static analysis tools to mitigate compiler false negatives. As part of the overall tool chain, these tools must also be verified, which is why we are already seeing SuperTest’s role expanding beyond the world of compilers.
SuperTest also contains an extensive suite of diagnostic conformance tests for the MISRA-C:2004 and MISRA C:2012 software development guidelines, so you can also use it to verify that your MISRA checker functions correctly. A false negative in a MISRA analysis is just as much a safety hazard as it is in a compiler, so your MISRA checker also requires verification.
SuperTest detects false negatives
As a software developer, your reputation depends on building applications that run flawlessly. So it’s absolutely essential that you verify your entire tool chain so that nothing introduces errors. However, it’s equally important to know that components in your tool chain that you would expect to catch source code errors do catch them, minimizing the chances of ‘false negatives’ to trip you up later. SuperTest not only contains tests for the correct functioning of the compiler, it also makes sure that the compiler alerts you to detectable errors in your source code. It does the same for static analysis tools and MISRA checkers. Because if you want your applications to strictly conform to the necessary standards, your tools also need to conform to them.
Subscribe to our monthly blog!