Trust and assurance—from consumers, the public, and governments—will be critical issues for the AI and autonomous technology space in the year ahead. Yet, earning that trust will require fundamental innovations in the way autonomous systems are tested and evaluated, according to Shawn Kimmel, EY-Parthenon Quantitative Strategies and Solutions executive director at Ernst & Young LLP. Thankfully, the industry now has access to innovative techniques and emerging methods that promise to transform the field.
The new autonomy environment
Automation has historically been pitched as a replacement for “dull, dirty, and dangerous” jobs, and that continues to be the case, whether it be work in underground mines, offshore infrastructure maintenance or, prompted by the pandemic, in medical facilities. Removing humans from harm’s way in sectors as essential and varied as energy, commodities, and healthcare remains a worthy goal.
But self-directed technologies are now going beyond those applications, finding ways to improve efficiency and convenience in everyday spaces and environments, says Kimmel, thanks to innovations in computer vision, artificial intelligence, robotics, materials, and data. Warehouse robotics have evolved from glorified trams shuttling materials from A to B into intelligent systems that can range freely across space, identify obstacles, alter routes based on stock levels, and handle delicate items. In surgical clinics, robots excel at microsurgical procedures in which the slightest human tremor has negative impacts. Startups in the autonomous vehicle sector are developing applications and services in niches like mapping, data management, and sensors. Robo-taxis are already commercially operating in San Francisco and expanding from Los Angeles to Chongqing.
As autonomous technology steps into more contexts, from public roads to medical clinics, safety and reliability become simultaneously more important to prove and more difficult to assure. Self-driving vehicles and unmanned air systems have already been implicated in crashes and casualties. “Mixed” environments, featuring both human and autonomous agents, have been identified as posing novel safety challenges.
The expansion of autonomous technology into new domains brings with it an expanding cast of stakeholders, from equipment manufacturers to software startups. This “system of systems” environment complicates testing, safety, and validation norms. Longer supply chains, along with more data and connectivity, introduce or accentuate safety and cyber risk.
As the behavior of autonomous systems becomes more complex, and the number of stakeholders grows, safety models with a common framework and terminology and interoperable testing become necessities. “Traditional systems engineering techniques have been stretched to their limits when it comes to autonomous systems,” says Kimmel. “There is a need to test a far larger set of requirements as autonomous systems are performing more complex tasks and safety-critical functions.” This need is, in turn, driving interest in finding efficiencies, to avoid test costs ballooning.
That requires innovations like predictive safety performance measures and preparation for unexpected “black swan” events, Kimmel argues, rather than relying on conventional metrics like mean time between failures. It also requires ways of identifying the most valuable and impactful test cases. The industry needs to increase the sophistication of its testing techniques without making the process unduly complex, costly, or inefficient. To achieve this goal, it may need to manage the set of unknowns in the operating mandate of autonomous systems, reducing the testing and safety “state space” from being semi-infinite to a testable set of conditions.
Testing, testing
The toolkit for autonomous system safety, testing, and assurance continues to evolve. Digital twins have become a development asset in the autonomous vehicles space. Virtual and hybrid “in-the-loop” testing environments are allowing system-of-system testing that includes components developed by multiple organizations across the supply chain, and reducing the cost and complexity of real-world testing through digital augmentation.