Trusting the Machine That Builds the Machine

Humanoid robots, collaborative robots and autonomous mobile systems are fundamentally different from traditional industrial machinery. They continuously process large volumes of sensor data and adapt their behavior through perception and planning algorithms. However, that adaptive intelligence ultimately depends on the correctness and predictability of the underlying execution logic. These systems therefore rely heavily on deterministic low-level software responsible for motor control, communication buses, and real-time safety functions.  

High-level AI systems may determine where a robot should move, but lower-level software, typically implemented in C and C++, ultimately determines how it moves, how quickly it reacts, and whether it remains stable and safe. In these systems, software defects do not remain confined to screens or logs. They directly affect behavior in the physical world. Safe and reliable interaction with the environment therefore depends on deterministic control software whose correctness can be trusted and that trust must extend beyond the application layer to the entire software stack.

Compilers and standard libraries are responsible for translating software into machine behavior, which makes them, and the entire toolchain, an essential link in the functional safety chain. A defect introduced by a compiler or library can systematically affect the final executable, even when the application code itself is correct. Companies developing safety-critical software must therefore not only demonstrate that their source code follows safety guidelines, but also that their development tools operate correctly. Especially in robotics, where software directly controls physical movement and interaction with the environment, ensuring toolchain integrity is critical to preventing unsafe outcomes.

A further challenge for safety is that robotics platforms evolve continuously through software updates, middleware integration and changing hardware targets. The software toolchain also continuously changes, if only because security updates cannot be skipped. Every change potentially invalidates previous qualification assumptions.

Emerging regulations are making these issues unavoidable. The EU Machinery Regulation, the AI Act, and the Cyber Resilience Act increasingly recognize software and software toolchains as explicit elements of safety and compliance. In particular, robots that use artificial intelligence or machine learning in safety-related functions are increasingly classified as high-risk machinery, requiring demonstrable, up-to-date evidence that the underlying software infrastructure is trustworthy, controlled, and verifiable. 

Traditional qualification approaches struggle to keep pace with this new reality. Toolchains cannot realistically be frozen across the entire lifecycle of modern autonomous systems, as they used to be. Qualification must evolve alongside the software itself! 

Continuous Qualification, as discussed in our previous blogs, treats toolchain assurance as an ongoing engineering activity rather than a one-time certification step. Through continuous compiler and library verification, organizations can maintain confidence that their software toolchain remains within the qualified safety envelope as the system naturally evolves. The future of robotics depends not only on intelligent machines, but on trusted software foundations.

At the Robotics Summit & Expo 2026 in Boston, our Chief Product Officer Sjoerd van der Zwaan will present a session entitled “Trusting the Machine That Builds the Machine”. The presentation will scrutinize what we believe is one of the most overlooked risks in modern robotics: the assumption that the software toolchain simply works as intended.

See you on Wednesday 27 May, 10:15 EDT in the Engineering Theater.