Babies, Bath Water, And Enterprise Architecture Maturity Models
Recently I took a look at an EA-maturity-model-cum-roadmap from Leo de Sousa, Manager of Business Application Services and Enterprise Architecture at the British Columbia Institute of Technology (click to read Leo’s blog on EA CMM). To my surprise, I liked it. Why was I surprised?
I have never liked EA maturity models. Yes, tracking progress is important. And yes, there should be a consensus about what characterizes a mature EA practice. But I don’t like how they would ostensibly be used to compare one enterprise with another, a la benchmarks. Perhaps I was soured on them by seeing them used as a governance technique in US federal agencies.
The Office of Management and Budget (OMB) required agencies to assess themselves against a standardized maturity model, with progressive hurdles in successive years. Federal agencies, accustomed as they are to all sorts of oversight and compliance mandates, know how to pass compliance audits. Did you ever wonder how (then) Popkin System Architect got so popular in the federal government? An EA tool was required to demonstrate a certain level of EA maturity and System Architect was the lowest-cost offering at the time (I’m sure there are other reasons as well). Behavior was around letter-of-the-law compliance, and it seldom catalyzed getting with the spirit. Even when Dick Burk at OMB introduced a clever workaround in a second version of the model — you could leapfrog to a level 4 if you showed actual business benefits, regardless of what other checklist items you missed — agencies simply marched through the maturity level checklists getting the requisite items done. The scores were good, but in my opinion they overstated the degree to which EA was ingrained in the culture of the agencies.
And then there’s the notion of a single number representing an enterprise. In most reasonably large organizations, there is some degree of federation of the EA effort. Some business units might have very advanced practices, while others have virtually none, and the enterprise-wide practice may have an entirely different agenda. What does one do, rate all units and average the number? That would be a pretty meaningless metric.
So, I’m afraid because of all this I threw the proverbial baby out with the bath water regarding EA maturity models. But I like Leo’s. He took some fairly standard criteria and modified them to fit his organizational goals and processes. They still resemble other CMMI-like level criteria, so his 3 would be similar to someone else’s 3, but that’s not the main point. The point is to create a roadmap for where he wants his EA practice to go, and then rate and report actual against plan. This approach is about creating a tailored roadmap to consciously mature your EA practice; it is not about scoring maturity of implemented architectures.
No big deal, you say? I tweeted about this and a couple of wise EA folks who usually respond to my tweets essentially said “of course, you have to make it your own, what else would you do?” making me wonder if I’ve been missing the boat all along. But I don’t think this is a prevalent attitude about EA maturity models. I think the general idea is to rate one’s org against where it ought to be, by some industry-standard scale. I don’t find that worth much. I think an organization should understand where they are and where they want to be, figure out how to get there, and track their progress. If someone else’s model of how EA can improve your org helps you describe your goal or your path, great. But generic doesn’t work — for any generic goal statement, you must edit it into terms specific to your enterprise. And don’t set any goals without including the statement “as measured by ___.” You can use the Goal-Question-Metric technique for filling in the blank, but that’s a topic for another day.
To really manage your EA agenda, I’d recommend taking the maturity model as a roadmap a step further and also populate it with goals that don’t fit the standard CMMI scale language. For example, if you have an incipient information architecture or business architecture practice, place goals and milestones related to expanding and formalizing those efforts on the roadmap. I’ve been interviewing people about their information architecture initiatives and working with Forrester analyst Jeff Scott on approaches to business architecture that can be broken down into a cookbook-like sequence, so watch this space for more on phased approaches to these aspects of architecture initiatives.
How do you all measure EA progress in your shops? Do you use a maturity model? And/or have you created concrete custom goals with associated metrics and measure against them?