Part 2: Testing Tools Market Landscape: It’s all about change!
What a strange summer this has been! From Boston to London to Paris to Turin, the weather has offered weekly and even daily reversals, with continuous change from sun to rain, from hot and damp to cool and crisp. I missed a nice spring season. Even today, from 35º-38º Celsius (95º-100º Fahrenheit), we just went to 22º Celsius (71º Fahrenheit) with a perfect storm! A continuous climate and sudden change is quite unusual in some of these countries. Certainly it is where the Azores Anticyclone usually dominates from mid-late June to mid-late August, offering a stable summer. How many times have you had to change plans because you discover weather is about to change!?
You might be thinking, "What does this have to do with this AD&D blog?" It’s about change! I am wondering if, in our daily lives, getting used to unexpected conditions and having to handle continuous change favors a mindset where change is just something we have to deal with and not fight. A new mindset very much needed given the change we see ahead in how we develop, test, and deploy software!
My focus in this blog is testing, although the first change we need to get used to is that we can’t talk any longer about testing in an isolated fashion! Testing is getting more and more interconnected in a continuous feedback loop with development and deployment. (See my colleague Kurt Bittner's report on continuous delivery; I could not agree more with what Kurt says there!)
But as promised in part 1, here is the link to the latest research I just published around Agile testing, “Navigating The Landscape Of Agile Testing Tools.” In the report, you will find more focus on the changes I see happening to Quality and Testing.
The over 20 vendors and clients interviewed for that research add more “crunch” to why the testing center of excellence (TCoE) as we know it can’t work going forward (especially when developing modern applications with mobile, cloud, and analytics). Here are some of the points I make:
1. Remote division of labor between those who develop and those who test slows test cycles: manual handover creates opportunity for mistakes and more fixing; creating artifacts for the only scope of communication among people is a waste.
2. Large volumes of manual test activities slow down delivery of Agile teams: Test professionals develop test cases that cover as much functionality as possible, but that makes the velocity problem worse. There’s no way around the fact that manual testing is slow and time-consuming and eats up a lot of resources. Only more automation can fix that problem!
3. Teams build up too much technical debt. One sure-fire killer of on-time delivery is finding out late in the development cycle that your application has major quality problems late in the development cycle. Late discovery of defects leads to high rates of rework and waste.
4. TCoEs don’t integrate well with modern continuous delivery practices. Segregating testers from developers makes it hard to integrate their work into a continuous delivery pipeline. Fast-moving teams don’t build code and then hand it off to a testing organization; they build code, deploy the application, execute it, and immediately observe the results.
Testing is the crucial spot in the world of “continuous” and of more “automation.” If you’ve promised shorter cycles and faster speed to your business, it cannot come only from some Agile PM introduction alone, you need to make your testing more agile, automate the downstream delivery and deployment, and make sure tools integrate and facilitate the “continuity.” Continuous development, integration, testing, and delivery is going to complete what you’ve started with your Agile journey (Link Holistic change), but your test bed to get there will be how you test!
The huge change of testing has an impact on all of the testing tools categories: from test management tools to functional and nonfunctional automation testing to performance and load testing to security and test data management tools and, finally, to the newcomers' services virtualization for software testing tools. More than ever there is a need for a constant flow of information among BAs, developers, and testers. As testing shifts more in the hands of developers, vendors need to adapt tools that easily plug into the developers' integrated development environments (IDEs), while QA and other software professionals prefer tools that offer a higher level of abstraction and are easy to use. The pressure on tool vendors is to emphasize qualities like collaboration/communication, continuous automation, simplicity, and seamless integration.
What are you doing about your testing strategy? Are you feeling the need to turn your TCoE into a practice center of excellence with fewer manual testers? Do you see a change in the tool strategy as a consequence? Or just a change in the way you’re testing tools?
I am interested to hear more . . .