Several years ago, I attended a talk by the Chief Architect of State Farm (a US insurer) about how they went about setting up their architecture practice. The morsel of information that really stuck in my mind was the story about getting Buy-in. Essentially, he referenced the set of problems that they faced in delivering on a televised promise from their CEO, and avoiding such panics in the future. I've noticed that a lot of organizations who adopt Enterprise Architecture practices do so for similar reasons – because they've been bitten in the past. But past problems shouldn't just be an impetus for change; they can also act as a test case for whether the controls and practices being implemented are effective.
Summary: When an organization sets up a modeling practice, it’s usually because of having been bitten by past problems. When setting up your practice, it’s worth revisiting the problems that you encountered, and considering whether what you had in place would have caught them. Some Root Cause Analysis may be needed to enable this.
The purpose of implementing a modeling practice is to solve a perceived problem. Unfortunately when setting a modeling practice up, you're predicting the future – in that you're anticipating problems that the model will help or even solve, and you're anticipating improved operations that the model will enable. The issue is, are you right?
It's a fair question, but one that is asked surprisingly seldom. Is the modeling practice that we are putting in place fit for purpose? You can engage in thought experiments, but often the challenge in doing so is that such exercises end up being rather abstract – and often, the devil is in the details. Thankfully, there are plenty of details to be found in past nightmares.
A useful question then to pose when setting up a modeling practice is: if we've been driven to do this by the memory of past issues, can we say with a clear conscience that had the changes we are making been in place, the issues in question would not have happened? It's a good question, and an excellent test.
However, as said before, the devil is in the details. There may well be a few past nightmares that have given the impetus to putting the model in place, but being able to look at them and comfortably say that “No, this wouldn't happen now because our modeling practice would have caught it” means being able to point to specific failings that caused the problems, and compare them to the new state of affairs.
In other words, even if there's a sense that it’s obvious how things fell through, it is worth explicitly breaking these things down to implement this idea; using past nightmares to validate the design of your modeling practice. There are several complementary techniques out there that are probably familiar – Five Whys, Fishbone Diagrams, and so on. Which technique is used will be down to personal preference. What's more important is to identify where the chain of events broke down; and from this, it becomes possible to pose the really important question:
“What would be different now if this happened again?”