The Update Paradox in Physical AI

Qualification evidence depends on stable assumptions. Yet, modern robotics platforms are anything but stable. Toolchains used in autonomous systems frequently change: AI frameworks are updated, security vulnerabilities require patching of the software toolchain, and control software constantly evolves. Every update potentially affects the qualification assumptions, the safety envelope, on which safety and compliance depend. At the same time, autonomous systems are increasingly expected to operate safely in physical environments. This creates a fundamental engineering problem.

Humanoid robots, autonomous mobile systems, and collaborative robots increasingly combine AI-driven perception, planning, real-time control, and distributed software architectures. Their software stacks are rarely static. New functionality, cybersecurity updates, middleware integration, and hardware evolution introduce change throughout the lifecycle of the platform.

Traditionally, qualification is treated as a one-time activity performed shortly before product release. The toolchain is then effectively frozen for the remainder of the system lifecycle. For modern robotics platforms, this approach is no longer realistic. If we do not update, systems become insecure and unsupported. If we do update, previous qualification assumptions may no longer hold.  This is the update paradox of Physical AI: autonomous systems must evolve continuously, but every update risks invalidating the evidence that demonstrates their safety and correctness.

Regulators increasingly expect software assurance to remain valid throughout the operational lifecycle of the system. The EU Machinery Regulation, the AI Act, and the Cyber Resilience Act all reinforce this shift toward lifecycle-based compliance. In particular, autonomous systems using AI in safety-related functions increasingly require demonstrable evidence that software and software toolchains remain controlled, verifiable, and trustworthy over time.

Safety throughout the platform’s lifecycle must deal with the risk that evolution of the software and changes to software toolchains move outside of the qualified safety envelope. Continuous Qualification addresses this problem by treating qualification as an ongoing engineering activity.

Instead of assuming that qualification evidence remains valid indefinitely, Continuous Qualification evaluates whether changes to compilers, libraries, build configurations, or system dependencies affect the original qualification baseline. Verification activities can then be repeated automatically where necessary, maintaining confidence that the safety envelope defined by the software toolchain matches the requirements of the platform software.

The objective of Continuous Qualification is to ensure that assurance scales alongside system evolution.

At the Robotics Summit & Expo 2026 in Boston, our Chief Product Officer, Sjoerd van der Zwaan, will present a session entitled “Trusting the Machine That Builds the Machine”. The presentation will scrutinize what we believe is one of the most overlooked risks in modern robotics: the assumption that the software toolchain simply works as intended.

See you on Wednesday 27 May, 10:15 EDT in the Engineering Theater.