Both Agile and Lean have an ethos that, at least on paper, acknowledges the noble failure. "Fail fast" is part of the Agile credo. Although it sounds as though it contradicts the "fail fast" approach, Lean's admonition to delay decisions for as long as possible is actually very complementary. The first draft of anything, from automotive design to software architecture to student papers, always contains elements that could be improved or that are just flat-out wrong. Practices that sit beneath the banner of Agile or Lean, such as set-based development, provide further ways to make mistakes and overcome them.
To a large extent, these practices deal with the easier varieties of failure. Prototyping a feature quickly, so that you can invite feedback when you actually have enough time to respond to it, is an extremely valuable technique for lowering the risk that you build something the wrong way. You need a different approach to identifying the features that you should not build, period.
At the Silicon Valley Product Camp a couple of weekends ago, Nils Davis of Accept Software gave one of the best presentations of the day, covering his company's experience with innovation jams. (Here's a link to the slides.) It's exactly the sort of exercise that gives people the opportunity to explore ideas that a team is still considering or identify ideas that they might have missed. After collecting ideas from around the company, a panel of Accept's executives selected ideas that looked both cool and profitable. On a Saturday, several teams worked feverishly to build prototypes that could test these concepts and suggest how the final implementation might look. The executive committee then selected the best of these ideas and gave prizes to the winning teams.
That type of innovation jam is a great idea, all around. Even an Agile team like the Accept development organization needs to step outside the assumptions of daily work and give people a chance to participate in ways they don't ordinarily get. The executive review step reminds everyone about the bigger business picture into which everyone's work must fit. (Not every idea that might be attractive to customers will have the same value for the ISV.) The day of fast-paced development gets the blood going. Heck, after hearing Nils' description, I wished I'd been part of their innovation jam.
However, it's important to note that this exercise, as laudable as it was, rewarded success. The awards went to the projects most likely to be added to the product road map. There was no special recognition or consolation prize for the idea that, after the most energetic, creative, and skilled efforts possible, proved to be far less brilliant than it seemed at first.
But that's hardly unusual. In fact, it's extremely difficult to find anyone who, in truth, rewards failure. While we often hear people talk about how they encourage risk-taking, practically no one rewards people for the downside of taking risks.
Rewarding noble failures runs against some of our deepest instincts about application development and delivery. No one prefers to be on the "noble failure" team, given the opportunity to be on the First Prize team instead. While we might have enjoyed the original Rocky movie, in which he loses the big fight, Hollywood felt compelled to release a sequel in which he wins. (And if that didn't drive home the point enough, United Artists produced more Rocky films, in which he wins yet again.)
While our brains might have a hard time embracing noble failures, the organizations in which we work make it even harder. Executives and managers are rewarded for delivering value, not avoiding duds. These same executives and managers are usually uncomfortable with techniques like set-based development, which depends on parallel design and development tracks that seem wasteful, even if the point is to avoid waste further down the road. Bad ideas don't flutter down into the room on their own from some intellectual stratosphere; instead, they're often the brainchildren of influential people within a team, who might not welcome other team members shooting down these ideas.
To overcome the natural tendency of organizations to frown on even the noblest of failures, leadership within a team, or a level or two above in the org chart, is required. Other kinds of changes, like continuous integration, might proceed apace without executive sponsorship. Not so with rewarding noble failures.
Sometimes, team leaders I've met at Intuit, Informatica, and other companies avoid focusing exclusively on the results of investigating ideas, giving conspicuous praise to people who are faithful to the process of investigation. Other teams go out of their way to give out recognition for team members who took a risk on an idea and then proved how bad it was. (Jared Spool is especially good at describing how that technique works in practice.)
Rewarding noble failures is hard, so very, very few teams really do it. It requires leadership that's willing to do more than just show up for the photo op at the end of a development cycle. App dev teams need the kind of leaders that encourage risks, acknowledge noble failures, and help teams learn from their mistakes.