What Our Users Say

“…Right from the start we believed that the test coverage of SuperTest was significantly better than other compiler validation suites on the market, and certainly better than the GCC test suite. Over the past few years that has proved true, because SuperTest has identified several generic bugs in new releases of GCC that were not picked up by the GCC suite, some of which were relatively severe code generation issues.”

“…Our customers already know about SuperTest and its reputation for meeting ISO 26262 requirements, so when we say that our tools are verified by SuperTest that definitely gives us an advantage during negotiations.”

“…SuperTest’s strengths lie in its breadth of coverage of language features and what it allows you to test, making it particularly useful when you are bringing up a compiler from scratch with a custom back-end. Its ability to do that and its general utility in that space is why we come back to using it again and again. It’s almost an industry-standard.”

“…One of the biggest advantages of SuperTest is that whenever it detects a failure, isolating the test case where the failure occurred is a breeze, because the way SuperTest is structured makes it very easy to isolate an issue so that you can focus on solving it.”

“…Configuring SuperTest to do what we wanted was extremely straightforward, which meant that we were up and running with compiler testing within a few days.”

“…Although it’s not SuperTest’s primary function, we also use it to measure code size and performance. The beauty of using it for this purpose is that all of SuperTest’s validation tests are relatively small in terms of program size, which means that discovering why a section of code has changed either in size of performance is relatively straightforward.”

“…We appreciate that many customers, especially when they get closer and closer to the target architecture, prefer to write those ‘close-in’ segments of code in C. So having a mix of SPARK, Ada and C in the final solution will not be uncommon. The important thing is that thanks to SuperTest we can now provide the same level of tool assurance for all three languages.”

“…SuperTest allowed us to robustly test the code generation flow, and during testing we discovered several bugs. Being able to find them early in the development process was key to fixing them before product release.”

“…We recognize that SuperTest is an extremely useful suite of tests, so we run it every few days as an integral part of our regression testing. We use it to check programs for functional correctness, run-time failures and code size, as well as using it to benchmark other tools in terms of correctness and code size.”

“…Thanks to SuperTest, the quality of our compiler has increased significantly.”

“…Yes, we are believers that SuperTest catches far more than just conformance issues. We recently encountered an extremely serious code generation issue, it would have been an absolute “show stopping” bug.  We do heavy testing with hundreds of test suites, real life code bases, benchmarks, plus more, and the only one that caught this bug was a SolidSands SuperTest case.”

“…SuperTest helped us to find the problems in our compiler far more quickly than otherwise would have been possible. In comparison with the license and maintenance fees, we saved more than twice the amount in our efforts in improving the compiler quality.”

“… SuperTest is my primary regression test bench. I’d start using it very early in future compiler projects. It is a really valuable tool for us. The verbose mode is wonderful for pointing me at the failing section of code. SuperTest found us implementation holes, implementation bugs, documentation errors, simulation bugs and many regressions.”

“…GNU testing alone doesn’t provide anywhere near enough test coverage. SuperTest not only provides significantly better coverage, but also does it with less run time and better reporting.”

“…We found your compiler validation suite to be superior to the others.”

“…We were able to easily integrate the SuperTest compiler tests into our continuous integration pipeline, helping us to commit to the high-quality level of compiler analysis  that our customers expect.”

“…One of our main concerns was validating the GCC C++11 compiler. SuperTest offers us a huge and very rich test suite that we can run against the compiler to verify that it does meet the language standards. GCC comes with its own test suite, but it’s not by any means certified for automotive use or guaranteed to meet the standard in all areas.”

“…We definitely think Solid Sands can help us to maintain the quality of our software tools, so when we pitch for new business we absolutely highlight that SuperTest is part of our strategy to deliver products with the best possible correctness and competitiveness.”

“…When we upgrade the compiler, we use SuperTest to check it and enforce internal rules such as optimizations that we should not use, so that we no longer have the problem of having to revert to a previous version.”

“…While our compilation tools have a good reputation, we would like to further increase the test coverage for our customers and SuperTest is the right tool to secure the excellent quality of our products.”

“…Since we started using SuperTest, not a single customer has reported finding a code generation problem with the C compiler.”

“…The test cases are well documented and tell you exactly where the problem is.”

“…SuperTest detected a lot of errors during our developments. Running the test suite successfully gave us enough confidence about the quality of each compiler version to be released to a next stage.”

“…One of the strengths of SuperTest was that it delivered on its promise of being an ‘out-of-the-box’ solution.”

“…The other big advantage in terms of timing was SuperTest’s fast run-time, which allowed us to achieve over 99% test coverage within a few weeks. That meant it offered very good value for money.”

“…SuperTest provides confidence in the scope and stability of our compiler. We view it as the prime step into testing our compiler. Regular use of SuperTest identifies regression problems in a timely manner. We use SuperTest at least weekly.”

“…Through its systematic testing, compiler stressing and corner cases, SuperTest revealed possibly hundreds of issues in our compilers.”

“…Many of these bugs are run-time failures, but SuperTest has also found a lot of the bugs in our LLVM front-end.”