Solid Sands
Download as PDF
Executive Summary
Artificial intelligence (AI) is changing the robotics landscape. Autonomous mobile robots (AMRs), Automated Guided Vehicles (AGVs), collaborative robots (cobots), industrial humanoids, and intelligent production systems are increasingly capable of perceiving their surroundings and making complex decisions. Together, these systems exemplify the rise of Physical AI (also commonly referred to as Embodied AI) in which AI enables machines to sense, reason, and act autonomously in the physical world. This new generation of intelligent systems promises major gains in productivity, flexibility, and autonomy.
However, as these systems become more autonomous, the need for robust governance, safety, and accountability grows in parallel. As robots move beyond fenced production cells and begin interacting directly with people and other machines, failures no longer remain confined to computer screens or log files. They can result in unintentional physical movement, equipment damage or, in extreme cases, harm to people.
Much of the discussion around Physical AI focuses on AI models, perception, decision-making, and application-level functional safety. Yet beneath every autonomous system lies another critical layer that receives far less attention: the software development toolchain. Compilers, standard libraries, linkers, and build systems translate source code into executable machine behavior. If these tools introduce defects, even correctly written software can behave unexpectedly.
At the same time, robotics platforms are evolving continuously through over-the-air (OTA) updates, leaving both application software and the underlying development environment in constant flux. Traditional qualification approaches, which rely on freezing software and toolchains before certification, are becoming increasingly impractical.
In addition, the regulatory landscape is evolving. The EU Machinery Regulation (EU) 2023/1230, together with the AI Act and Cyber Resilience Act, extends regulatory expectations to software, AI, and cybersecurity across the product lifecycle. With that, compliance has evolved from a one-time certification milestone into an ongoing engineering discipline maintained throughout the product lifecycle.
This white paper explains why toolchains are such a fundamental part of the functional safety chain, explores the toolchain qualification challenges that come with continuously evolving autonomous systems, and outlines why continuous qualification is emerging as a practical strategy for maintaining trust in software-defined machinery.
“Intelligence alone is not enough. Trusted execution matters.”
Today’s autonomous machines bear little resemblance to the industrial robots that first entered factories decades ago. Traditional robots operated inside fixed work envelopes, repeatedly executing pre-programmed movements within highly controlled environments. Their behavior was deterministic and largely isolated from human interaction, allowing safety to be achieved primarily through physical guarding, interlocks, and conventional control logic.
Physical AI presents a fundamentally different engineering challenge. AGVs, AMRs, cobots, warehouse automation systems, and industrial humanoids continuously interpret and adapt to their surroundings while interacting safely with people. Achieving this requires the integration of multiple software technologies and capabilities, including:
Together, these technologies underpin the ‘Sense-Think-Act’ model.
Figure 1. The Sense–Think–Act architectural model. AI determines what a machine should do, while the software toolchain translates engineering intent into executable code that ultimately controls physical behaviour.
Sense acquires safety-relevant information from cameras, LiDAR, radar, force sensors, and other inputs.
Think combines AI inference, planning algorithms, and decision-making software to determine the appropriate response.
Act converts those decisions into precise control of motors, brakes, steering systems, and other physical actuators.
Although AI attracts much of the attention, it is only one part of a much larger software ecosystem. Intelligent decision-making has little value unless it is translated into safe, predictable physical behavior. Put simply, intelligence alone does not move machines. Trusted execution does. This distinction becomes increasingly important as robots grow more sophisticated.
Unlike a conventional robot arm fixed to the factory floor, an industrial humanoid must maintain balance, compensate for gravity, coordinate dozens of joints and respond instantly to unexpected events.
Similarly, an AMR navigating a warehouse must detect people, calculate safe routes, communicate with surrounding infrastructure and continuously adapt its behavior in real time.
In every case, software determines physical action. Unlike conventional IT systems, where software failures typically result in inconvenience or financial loss, failures in Physical AI can have immediate real-world consequences. Software has therefore become an integral part of the machine’s safety case rather than simply a feature that enhances machine capability.
Much of this software stack continues to be implemented in C and C++, particularly within embedded systems responsible for deterministic, real-time control. High-level AI may determine where a robot should move, but lower-level software determines how it moves, how quickly it responds, and whether those actions remain safe.
Confidence in autonomous systems therefore depends not only on validating application software, but also on ensuring that the underlying software infrastructure performs exactly as intended. That infrastructure includes the software development tools themselves.
“Confidence in safety-critical software cannot end at the source code. It must extend to the entire software toolchain.”
For many engineering teams, compilers and standard libraries are viewed simply as engineering utilities. Once selected, they are assumed to function correctly and receive far less scrutiny than application software or system-level testing.
Yet every line of source code requires translation into executable machine instructions before a robot can perform any actions.
Compilers, together with standard libraries, linkers, and other toolchain components, perform that translation. In effect, they convert engineering intent into machine behavior. However, the process harbors the potential to introduce defects:
Crucially, these faults originate below the application layer. An engineering team can rigorously verify source code, follow recognized coding standards, and perform extensive testing, yet still produce an executable that behaves incorrectly because a defect was introduced during compilation or linking.
As every software build uses the same development tools, such defects are systematic rather than random. They can therefore propagate consistently across an entire software product without any change to the application code itself.
For safety-critical robotics, this creates a silent risk. Toolchain defects often remain hidden because engineers naturally assume the development environment itself is trustworthy. As autonomous systems become increasingly software-defined and adaptive, however, that assumption becomes harder to justify.
Functional safety engineering has long recognized that software development tools can influence the integrity of safety-related software. However, this consideration has traditionally received less attention than application-level verification and system testing. As Physical AI increases the amount of software involved in machine behavior, the development environment itself becomes an increasingly important part of understanding and controlling systematic risk. The compiler is therefore no longer merely a productivity tool; it has become part of the functional safety chain.
For organizations developing Physical AI systems, the focus extends beyond asking, “Is our application software correct?” They must also be able to answer: “Can we demonstrate that the tools used to build our software are behaving correctly?”
“Modern autonomous systems must evolve continuously, but every update risks invalidating the evidence that demonstrates their safety.”
One defining characteristic of Physical AI is continuous evolution. Software improvements have become both an engineering expectation and a commercial necessity. It enables manufacturers to:
Although most software updates improve functionality or performance, a small number may unintentionally affect safety-critical behavior. The challenge lies in identifying which changes require renewed assurance.
Manufacturers historically addressed this uncertainty by qualifying a software development toolchain once, freezing it, and avoiding further changes wherever possible. This approach worked well when software changed infrequently. Today, however, connected robots require regular cybersecurity updates to remain protected against emerging threats. AI models are continually refined to improve perception and decision-making, while new sensors, middleware, and hardware platforms are integrated to extend system capabilities.
The toolchain evolves alongside these technologies. Compiler vendors release bug fixes, improve optimization, expand language support, and address security issues. Standard libraries and build environments receive similar updates. For example, widely used open-source toolchains such as GCC and LLVM/Clang are under continuous development, with annual major releases complemented by frequent maintenance updates, bug fixes, and security patches. Even organizations that upgrade infrequently must continually evaluate whether adopting these improvements affects the validity of previously established qualification evidence.
Together, such changes create the update paradox. Autonomous systems must continue evolving to remain safe, secure, and commercially competitive. Yet every change potentially weakens confidence in the qualification evidence used to demonstrate compliance.
Put simply, freezing the software environment is increasingly impractical, while updating without maintaining assurance is not acceptable. The challenge is therefore not to prevent change, but to manage it. The evidence supporting confidence must evolve alongside the software itself, making assurance a continuous process.
“Regulation is not changing the engineering challenges. It is changing what organizations must be able to prove.”
The engineering challenges described so far exist independently of regulation. Software is becoming increasingly complex, autonomous systems evolve continuously, and software development toolchains directly influence machine behavior.
What regulation changes is the expectation that these realities must be explicitly managed and demonstrably controlled. The most significant regulatory driver is the new EU Machinery Regulation (EU) 2023/1230, which becomes fully applicable on 20 January 2027, replacing the Machinery Directive 2006/42/EC.
The Regulation reflects a fundamental shift in the role of software within machine safety. Software no longer merely supports safety functions; it increasingly performs them.
This is explicitly recognized in Annex III of the Regulation, which defines the essential health and safety requirements (EHSRs) that manufacturers must satisfy. These requirements include provisions addressing the safety and reliability of control systems, as well as protection against software-related corruption, recognizing that software can directly influence the safe behavior of machinery.
While these requirements apply to all safety-related software, they become increasingly challenging for Physical AI systems, where AI-based perception, reasoning, and decision-making directly influence machine behavior.
To establish a baseline classification, Annex II provides an indicative list of safety components that includes software ensuring safety functions. This places safety-related software within the scope of regulated safety components, allowing conventional deterministic software to generally follow established conformity assessment routes, including internal production control where applicable. However, Annex I (Part A) introduces a specific high-risk category for safety components and embedded systems that use fully or partially self-evolving machine learning approaches to perform safety functions. Systems falling into this category are subject to enhanced conformity assessment procedures requiring the involvement of a Notified Body, reflecting the additional challenges associated with adaptive and evolving safety behavior.
Where AI and machine learning become integrated into safety-related functions, manufacturers must demonstrate that these functions perform safely and reliably throughout the product lifecycle. While the Machinery Regulation does not prescribe how software development tools such as compilers, linkers, or build systems should be qualified, demonstrating compliance increasingly depends on confidence in the entire software development process.
Established functional safety standards for machinery, including IEC 62061 and ISO 13849 (which are based on principles from the broader IEC 61508 functional safety framework), recognize that the software development process can influence the integrity of safety-related software. These standards emphasize the need for confidence in the methods, processes, and tools used to develop and validate safety functions. As software assumes a more central role in machine safety, the software toolchain becomes an increasingly important element of the overall safety case.
However, demonstrating compliance requires more than proving that software and toolchain is safe at the point of certification. Organizations must maintain assurance as software evolves throughout the product lifecycle. This broader lifecycle perspective is reinforced by two complementary areas of European legislation:
Together, these regulations shift compliance from a point-in-time certification exercise towards continuous lifecycle assurance. Rather than treating safety, AI, and cybersecurity as separate disciplines, the EU is moving towards an integrated engineering approach in which software must remain trustworthy throughout the operational life of the machine.
Although this discussion focuses on the European regulatory framework, the underlying engineering principles are equally applicable to manufacturers operating under other functional safety standards and regulatory frameworks worldwide.
Historically, compliance often followed a familiar sequence:
The emerging regulatory model instead emphasizes:
Demonstrating that a machine behaves safely during testing is only part of the assurance challenge. Organizations must also show how confidence in that behavior is sustained as software evolves.
This expectation naturally extends to the software toolchain. If compilers, standard libraries, and build systems directly influence safety-critical software, confidence in those tools increasingly depends on objective evidence rather than historical experience or engineering assumptions. Organizations therefore need to demonstrate that the development environment remains trustworthy throughout the product lifecycle.
In many respects, regulation is simply catching up with engineering reality. The silent risk posed by toolchains and the update paradox created by continuous software evolution have existed for years. The new regulatory framework acknowledges these realities and requires organizations to demonstrate that they are systematically identified, controlled, and supported by objective evidence.
“Qualification becomes an ongoing engineering process rather than a one-time activity.”
The update paradox makes one thing clear: if software evolves continuously, qualification cannot remain a one-time activity. Modern robotics platforms require an engineering approach that evolves alongside the software itself. Rather than requalifying the development environment from scratch after every update, organizations need a scalable way to assure that their software development toolchain continues to behave as intended.
This is the principle behind continuous qualification. Instead of treating qualification as a one-off activity before certification, continuous qualification makes toolchain assurance an integral part of the software lifecycle. The objective is to ensure that every significant change can be evaluated against objective qualification evidence.
Figure 2. Continuous qualification workflow. Significant changes to the software development environment automatically trigger verification activities that generate objective evidence and determine whether the toolchain remains within the Qualified Operating Envelope.
Defining the Qualified Operating Envelope
Every qualified toolchain operates within a defined set of conditions, including:
Together, these define the environment for which qualification evidence exists. As long as development remains within this qualified operating envelope, engineers can be confident that the assumptions underpinning the qualification evidence remain valid.
The challenge arises when that environment changes:
Even relatively minor changes can move the development environment beyond the conditions originally assessed.
Traditionally, organizations either avoided such changes or relied on lengthy manual reviews. In a continuous qualification workflow, verification can be triggered automatically whenever significant changes occur. Objective testing establishes whether the updated environment remains within the qualified operating envelope or whether further investigation is required.
Continuous Conformance
Shifting the focus to continuous conformance demonstrates that the toolchain continues to perform as expected throughout its operational life. Typical questions to ask include:
These questions can be answered through structured, repeatable verification.
Comprehensive compiler and library test suites provide objective evidence that software development tools continue to operate in accordance with their specifications. Where no significant changes are detected, organizations can maintain confidence in the existing qualification baseline. Where deviations are identified, they can be investigated before they affect safety-critical software.
This approach also improves engineering efficiency. By establishing confidence in the development environment first, application teams can focus on developing and validating higher-level functionality rather than repeatedly questioning the integrity of the underlying toolchain.
Automated Evidence Generation
As software development cycles accelerate, manual qualification becomes increasingly difficult to sustain. Automation is therefore central to continuous qualification.
Within modern continuous integration environments, verification activities can be triggered automatically whenever compiler updates, library revisions, configuration changes, or scheduled regression tests occur. The resulting evidence links software builds directly to the versions of the toolchain used and the verification activities performed.
The results include clear traceability throughout the software lifecycle and auditable records that support both engineering teams and certification authorities. Perhaps more importantly, automation enables assurance to scale alongside software development. Rather than becoming a bottleneck, toolchain qualification becomes an integral part of the engineering workflow, enabling organizations to embrace continuous software evolution without sacrificing confidence, safety, or compliance.
The next generation of autonomous machinery will be judged not only by how intelligently it perceives the world, but by how safely and consistently it behaves within it.
While Physical AI is transforming industrial automation, greater autonomy inevitably increases software dependence. Trust must therefore extend beyond AI models and application software to encompass the entire software development process.
Compilers, standard libraries, and build systems are integral components of the functional safety chain, influencing how software is translated into the physical behavior of real machines.
At the same time, software is evolving continuously. Cybersecurity updates, functional improvements, and advances in AI capabilities have fundamentally changed the assumptions on which traditional qualification practices were built. The latest European regulatory framework reflects this reality by recognizing that compliance must be maintained throughout the product lifecycle rather than demonstrated only before market launch.
Continuous qualification provides a practical framework for making that transition. By combining structured toolchain verification, continuous conformance testing, and automated evidence generation, organizations can maintain confidence in their software development environment while continuing to innovate at the pace demanded by modern robotics.
As manufacturers prepare for the full implementation of the EU Machinery Regulation in January 2027, now is the time to assess whether existing software development and qualification processes are fit for a software-defined future. For organizations developing safety-critical software, this begins with the software toolchain itself. Key questions include:
Answering these questions requires more than regulatory awareness. It demands engineering processes capable of sustaining assurance throughout the lifecycle of increasingly autonomous systems.
Solid Sands helps organizations in establishing confidence in the software toolchains used to develop safety-critical systems. Its compiler validation and library qualification solutions, SuperTest™ for C and C++ compilers, and SuperGuard™ for C and C++ libraries, enable manufacturers to generate the objective evidence needed for continuous qualification while reducing long-term certification risk.
As Physical AI enters mainstream industrial deployment, trust in the software toolchain will become a defining factor in the safe and reliable operation of autonomous machines. Ultimately, the future of autonomous machines depends on trusting the machine that builds the machine.
Home / Knowledge / Whitepaper / Trusting the machine that builds the machine