Continuous Improvements of Compiler Toolchain for Synopsys ARC Processors

Synopsys  – Solid Sands

The Synopsys ARC® Processor IP portfolio includes an extensive range of ARC-VTM RISC-V based processors,  along with a corresponding set of optimizing compilers.  The user configurability of these cores − allowing for the introduction or omission of hardware blocks, the adoption of various memory schemes, and the addition of custom instructions to meet specific performance, power and area requirements − significantly complicates compiler toolchain testing. Irena Phironova, Senior R&D Manager  of Synopsys ARC MetaWare Development Team, provides  some insights into why they use the SuperTest™ Compiler  Test and Validation Suite to improve their compilers and runtime libraries.

“One of the biggest advantages of SuperTest is  that whenever it detects a failure, isolating the  test case where the failure occurred is a breeze  because the way SuperTest is structured makes  it very easy to isolate an issue so that you can  focus on solving it.”

“Our customers are very conscious of hardware die size,  and our highly configurable ARC cores enable them to adjust configuration settings to tune performance,  power, and area tradeoffs for their specific requirements.  For example, designers can choose to include certain  RISC-V extensions, opt for a floating-point configuration,  configure different amounts of closely coupled memories  (CCM) or cache memory, and define the number of  interrupts. However, this flexibility presents a real challenge, as we need to ensure that our compilers generate code that matches those configurations. Consequently, we run our regressions across numerous configurations to ensure the proper coverage,” explains Irena Phironova.  “SuperTest provides the coverage we need in this cross-configuration testing. It also runs fast, so we don’t face any issues with the compute time. We run SuperTest overnight as part of our daily testing routine.”

However, according to Irena, meeting Synopsys’ test coverage and run-time performance characteristics is not the whole story. “One of the biggest advantages of  SuperTest is that whenever it detects a failure, isolating the  test case where the failure occurred is a breeze because the way SuperTest is structured makes it very easy to  isolate an issue so that you can focus on solving it.”

“SuperTest provides the coverage we need in this cross-configuration testing. It also runs fast, so we  don’t have any issues with the compute time.”

For ARC MetaWare Toolkit, SuperTest’s capability to identify and isolate compiler issues also makes it a key weapon in keeping pace with rapidly changing compiler technology. “The compilers for our ARC cores are based on LLVM1 technology with a ‘Clang LLVM’ front-end, which means that we need to periodically synchronize with a new version of LLVM and cope with a new wave of potentially critical bugs,” explains Irena. “It’s not just in the front end that bugs can occur. Changes in the optimization paths at the back end of our compilers can also create new problems. SuperTest identifies a lot of the issues, not only in the front end but also in the back end.”.

Irena even has figures to prove SuperTest’s effectiveness.  “We have a bug reporting system that makes it very easy to track the source of each bug and the tool used to identify it. Upon analyzing all the reports from the last  six months, I found that SuperTest was responsible for finding around 10% of the bugs. Considering the extensive scope of our testing framework, this is a remarkably high percentage for a single tool, making SuperTest one of the most productive tools we have,” explains Irena. “Many of  these bugs are runtime failures, and SuperTest has also  identified some in our LLVM front-end, showcasing its  comprehensive detection capabilities.”

In addition to regression testing of the compiler, the Synopsys ARC MetaWare Development Team uses SuperTest for functional testing and as a conformance test for C and C++ runtime standards. It is essential to guarantee that each new release of the toolchain includes runtime libraries that strictly conform to the latest standards. This involves two key tasks: first, ensuring that recent changes in the libraries and the compiler do not introduce regressions, and second, validating the implementation of new versions of the C and C++ standards as they are updated.