My first detailed experience with technical standards was defining the integration of Callminder, British Telecom’s voicemail system, into their telephone signaling system. We used a standard called SS7, and during a preparatory seminar I asked “how often do these standards change?” An attendee at the same seminar came out with the immortal line “They don’t change, they’re standards”. Face palm. As I found later, there were 5 different versions extant, and BT used a mixture of two of these versions.
The other significant quote that I remember about standards was when I did my TOGAF 9 training several years ago. At one point the trainer (who was one of the reviewers of the specification) said “Yes, we know TOGAF 9 contradicts itself in places, it’s a problem, but it’ll never get totally resolved until a few people die off”. I’m fairly certain that he was referring to the passage of time instead of a ruthless dispute resolution procedure, but the point is well made.
Summary: It can often be frustrating how different standards fail to align or even flat out contradict one another. In some cases, one single standard can contradict itself in different places. Unfortunately, this is a natural consequence of how vendors and standards interact, and the egos that highly intelligent people naturally have. Ultimately, the only thing that you can do is identify parts of standards that you find worthwhile and test out the precise use case yourself in a tool.
The first limitation that you run into with any of these standards is that just because a standard is published, this doesn’t mean that it is fully implemented by a vendor. For example, the TOGAF compliance questionnaire includes the line “Tool shall support the Software Engineering Diagram” (to choose one of the 650 lines at random). Leaving aside the fact that TOGAF doesn’t even prescribe a notation, what does ‘support’ mean? That I can draw one? In that case, Visio supports it. PowerPoint supports it. Technically, even MS Paint supports it, but I wouldn’t want to be using Paint as a modeling tool.
This is hard to avoid, given that the standards that exist and especially the compliance questionnaires are the outcome of repeated discussions and, frankly, arguments between vendors (tool vendors and consultants in the space). Every tool vendor, and most especially every consultant, is going to have their own vision of the best way to implement a given task, and so a standard like TOGAF can’t avoid ending up being a middle ground compromise between competing visions – otherwise the standard would never get issued. In a standard as large as TOGAF, the compromise can even be that one section describes one way of approaching a topic, and a different part describes a different part.
So what can an end user organization do in such a situation? Really, the only way to approach this is to identify the topics that are key to your initiative, identify the parts of the standard and examine and rate how each tool vendor has implemented that topic. While it would be nice if you use a standard such as TOGAF or COBIT as a simple test (“Compliant? Yes or No”), in practice most standards that exist in the modeling space act more as guidelines – and this is unavoidable given the different array of organizations involved in setting them.