Continuous automation and testing (CAT) service providers were some of the first to experiment with AI and ML models back in 2019, implementing various testing use cases with key customers. In our last CAT services Forrester Wave™ evaluation in 2021, intimacy, collaboration, and AI were top of mind for CAT services clients. But generative AI (genAI) has completely shifted the conversation. Customers have high expectations of genAI’s ability to augment their provider’s testers to be smarter, faster, and more efficient. Customers still expect CAT service providers to demonstrate client intimacy and collaboration. The Forrester Wave™: Continuous Automation And Testing Services, Q2 2024, just published this week, evaluates a comprehensive list of 27 criteria. We identified the most significant players and researched, analyzed, and scored them.

You’ll Need The Best Partner With The Best Testing Services

Whether you are using agile, DevOps, or TuringBots, to build modern software, you’ll probably need some help to realize the benefits of continuous automation testing. But you’ll have to first select from a diverse set of vendors that vary by size, type of offering, geography, and use case focus. In our continuous automation and testing services vendor landscape report, published in Q4 2023, we scanned the global market and found 42 players that met our first set of criteria for being a CAT services provider. It was not an easy task to identify which vendors deserved the status of Wave participants. We decided to include those with more than $300 million in revenue and used a few more inclusion criteria to consolidate the number to 13. You’ll also find smaller players in the vendor landscape research that might also deserve your attention.

Some Takeaways For You

​Adoption of tester TuringBots is already proving a significant differentiator in firms looking for their testing provider of choice. Tester TuringBots are:
  • ​Being used by providers to not only automate the various testing activities but also to enhance the impact of their software IP accelerators and methods for delivering enhanced time to value for enterprise test transformation programs.
  • ​Driving firms to have to demonstrate not only how they will functionally test AI models themselves but also to test AI-infused applications, as core enterprise software embeds predictive and generative AI capability into common systems and workflows.
  • ​Required to support both firm needs and upcoming AI-related regulation in many jurisdictions (most notably in the EU).
  • Forcing testing providers to be more than a provider of stacks of testing resources to meet demand. Providers have had to make their testing IP more sophisticated in the last 18 months.

There is much more juice in the Wave evaluation for our Forrester clients. Vendors we included in the Wave are: Accenture, Amdocs, Capgemini, Cognizant, EPAM, EY, HCLTech, IBM, Infosys, LTIMindtree, Qualitest, Tata Consultancy Services, and Wipro. If you need any help in putting together the optimal list of vendors to select from for your next continuous automation and testing services RFI/RFP, reach out. I’ll be happy to help.