This is part of a series summarising guidance and policy around the safe procurement and adoption of AI for military purposes. This article looks at the DSTL Biscuit Books around AI, available here: Assurance of AI and Autonomous Systems: a Dstl biscuit book - GOV.UK (www.gov.uk)

As artificial intelligence (AI) revolutionises industries from healthcare to transport, one critical factor holds back widespread adoption: assurance. Defence Science and Technology Laboratory (Dstl) has released a comprehensive guide, "Assurance of Artificial Intelligence and Autonomous Systems," exploring the steps necessary to ensure AI systems are safe, reliable, and trustworthy.

The Biscuit Book underscores the need for assurance as a structured process that provides confidence in the performance and safety of AI and autonomous systems. Without it, we risk deploying technology either prematurely when it remains unsafe, or too late, missing valuable opportunities.

Why Assurance Matters

AI and autonomous systems increasingly tackle complex tasks, from medical diagnostics to self-driving cars. However, these systems often operate in unpredictable environments, making their behaviour difficult to guarantee. Assurance provides the evidence needed to instil confidence that these systems can function as expected, especially in unforeseen circumstances.

Dstl defines assurance as the collection and analysis of data to demonstrate a system's reliability. This includes verifying that AI algorithms can handle unexpected scenarios and ensuring autonomous systems behave safely.

Navigating Legal and Ethical Challenges

AI introduces new legal and ethical dilemmas, particularly around accountability when things go wrong. The report highlights the difficulty in tracing responsibility for failures when human operators oversee systems but don't control every decision. Consequently, legal frameworks must evolve alongside AI technologies to address issues like data privacy, fairness, and transparency.

Ethical principles such as avoiding harm, ensuring justice, and maintaining transparency are essential in developing AI systems. However, implementing these values in real-world scenarios remains a significant challenge.

From Algorithms to Hardware: A Complex Web of Assurance

The guide covers multiple areas where assurance is necessary:

• Data: Ensuring training data is accurate, unbiased, and relevant is critical, as poor data can lead to unreliable systems.

• Algorithms: Rigorous testing and validation of AI algorithms are essential to ensure they perform correctly in all situations.

• Hardware: AI systems must rely on computing hardware that is secure and operates as expected under all conditions.

Ensuring all these components work seamlessly together is complex, which is one reason we don't yet see fully autonomous cars on the roads.

The Ever-Present Threat of Adversaries

As AI systems become more integrated into society, they become attractive targets for adversaries, including cybercriminals and rogue states. Small changes in data or deliberate attacks on system inputs can cause catastrophic failures. To mitigate these risks, Dstl advocates for rigorous security testing and using trusted data sources.

A Costly but Necessary Process

Assurance comes at a price, but it's necessary to avoid costly failures or missed opportunities. The Dstl Biscuit Book emphasises that the level of assurance required depends on the potential risks involved. For example, systems used in high-risk environments, such as aviation, require far more rigorous testing and validation than lower-risk systems.

Ultimately, assurance isn't a one-time activity. As AI systems evolve and adapt to new environments, ongoing testing and validation are needed to maintain safety and trust.

Looking Ahead

The Dstl Biscuit Book remains a highly relevant reminder of the challenges in ensuring AI systems are safe and reliable. While AI holds incredible potential to transform industries and improve lives, the journey to fully autonomous systems requires a careful balance of technical expertise, ethical responsibility, and robust assurance frameworks.

For now, it's clear that unlocking the full potential of AI and autonomous systems hinges on our ability to assure their safety at every step.