Organisations are increasingly motivated to integrate AI into their services to enhance productivity and reduce costs. However, embedding AI within applications introduces novel security and trust-related challenges. Traditional unit tests and security evaluations are insufficient when AI is a component of an application; AI models must be tested separately for unwanted behaviours. The key to managing these challenges lies in identifying and addressing AI-specific failure modes—scenarios where AI systems may fail, leading to security breaches or unpredictable performance.

To mitigate these risks effectively, organisations must implement targeted strategies and automated processes to test AI models for potential failures. This talk will explain how automating the testing of AI failure modes can ensure that AI-enhanced applications are both secure and reliable. By generating metrics for AI failure modes and using them as automated gates within DevSecOps pipelines, organisations can safeguard their applications from the risks associated with AI, allowing for the seamless integration of AI without exposing the system to unnecessary vulnerabilities.