Best Practice Report

It’s Time To Get Really Serious About Testing Your AI: Part Two

Create A Clear Strategy And A Test Automation Approach Where Possible

October 19th, 2023
Diego Lo Giudice, Vice President, Principal Analyst
Diego Lo Giudice Vice President, Principal Analyst
With contributors:
Chris Gardner , Srividya Sridharan , Mike Gualtieri , Brandon Purcell , Michele Goetz , Martha Bennett , Merve Kandemir , Kara Hartig

Summary

Part one of this research described how to test AI-infused applications (AIIAs) based on what we know about testing automatic software and how data scientists test autonomous software. We then presented the integration and performance of the combined automatic and autonomous software (the AIIAs). In this report, we address testing and behavior of AI embedded or integrated in AIIAs and the technology stack needed for automated and continuous testing in an automated delivery pipeline. Large language models (LLMs) bring an additional level of nondeterminism and complexity to testing as we briefly introduce in this report, knowing that we need to do more research on this topic.

Want to read the full report?

Contact us to become a client

This report is available for individual purchase ($1495).

Forrester helps business and technology leaders use customer obsession to accelerate growth. That means empowering you to put the customer at the center of everything you do: your leadership strategy, and operations. Becoming a customer-obsessed organization requires change — it requires being bold. We give business and technology leaders the confidence to put bold into action, shaping and guiding how to navigate today's unprecedented change in order to succeed.