The story of Agile is more than just one chapter in the history of software development. It's also an extremely valuable case study in innovation, an elusive and often humbling process that doesn't work in quite the way that we instinctively think it should.
The trajectory of Agile points toward increasing the value of the software delivered. However, it started with no metric of value and almost no notion of what happened outside the development team. Instead, the first phase of Agile, as I describe in the recent publication "Navigating The Future Of Agile And Lean," focused primarily on changing the behavior and world view of developers working together in a team. Therefore, you won't find anything in Scrum or XP that says, "This is how you know your software is better." Instead, these methodologies told you how to work more effectively as a team. Presumably, better results would follow.
In fact, the framers of the Agile Manifesto sought to change the whole notion of methodology, according to Martin Fowler:
Engineering methodologies have been around for a long time. They've not been noticeable for being terribly successful. They are even less noted for being popular. The most frequent criticism of these methodologies is that they are bureaucratic. There's so much stuff to do to follow the methodology that the whole pace of development slows down.
Agile methodologies developed as a reaction to these methodologies. For many people the appeal of these agile methodologies is their reaction to the bureaucracy of the engineering methodologies. These new methods attempt a useful compromise between no process and too much process, providing just enough process to gain a reasonable payoff.
Regardless of what sort of innovation we're attempting, we should heed this example. No one is smart enough to figure out every aspect of disruptive innovation from beginning to end. Innovation is a journey, not of a thousand steps, but into an unfamiliar landscape, full of potholes, wrong turns, and dead ends. It's important to start with something that compels you to make the journey at all, and to navigate successfully through the unknown.
You can see that innovation strategy today in the issues Agilists are talking about. We've moved well past the developer-focused topics, such as sprint planning and test-driven development. Now, the Agile community is grappling with other parts of the value chain beyond the development team. How do we better incorporate customer feedback? How do we collaborate better with UX designers? How can we release as quickly as we build? If there are business imperatives beyond just building software more effectively, how do we alter the Agile timeline, creating something like water-Scrum-fall instead of an endless series of sprints? How do we incorporate Lean to ensure that we're all moving in the right direction?
These are the questions that no one could have answered in 2001, or even in 2006. It took a long time to hone core Agile practices, get a lot of teams to adopt Agile, and then watch the ripple effects. Only then could we get into these sorts of discussions about improving software value.