Welcome to Orcmid's Lair, the playground for family connections, pastimes, and scholarly vocation -- the collected professional and recreational work of Dennis E. Hamilton
Open-Standards Implementation, I: Celebrating Massachusetts
I have strong sympathies for the use of public, open formats for documents in civil administration, governmental records, and in the preservation of our electronic and pre-electronic heritage. I absolutely favor any efforts that remove impedance mismatches from the interactions of citizens with [e]Government in all of its flavors.
I have some concerns about the extent to which offers of solutions are surrounded by too many claims of hypothetical benefits (which always look superior to the practical reality) and by poorly-disguised attempts to distort markets with a seasoning of appealing snake oil: asserted distopian consequences from all protagonists and very untested utopian-salvation promises. I am also pained by the hasty and thoughtless efforts to legislate technology that seems to be the popular substitute for the deliberate department-initiated efforts in Massachusetts. I think legislation of technology, and the double-speak that accompanies it (also from all protagonists) is misguided; most of all, I am concerned that the citizens and their civil agencies and public institutions are going to suffer for it.
Experimentation and trial of alternatives is going to be important in finding successful approaches. Some of the experiments will point to important course correction; most of all, they will add reality to our understanding of what it takes.
The poster child for adoption of open-document formats in the United States is the Commonwealth of Massachusetts Information Technology Division (ITD), part of the Executive Office for Administration and Finance in the Commonwealth. I think it is time to pay homage to the efforts that were undertaken by ITD and the Commonwealth CIO in establishing its Open Initiatives. My goal is to recognize the practical, deliberate efforts in the ITD and the pragmatic realism that is displayed in the public documents on the ITD web site. This is in marked contrast to the way these efforts are often portrayed. I will indicate where I have some concerns; my overall purpose is to recognize the accomplishment and how it is being managed in a professional and highly-responsible manner. It is also a noisy human effort; that is not an indictment.
Open-Initiative Policies: The Stake in the Ground
The current Open Initiatives page has this plain-language statement:
It is important to understand that this is a commitment around the information-technology operations of an executive branch of state government: the policy is focused on technology used primarily in the internal operations of governmental administration. This is consistent with the ITD department’s accountability. The policies can certainly impact inter-governmental operations and coordination with vendors and the public in various collaborative exchanges, despite possibly-disingenuous claims to the contrary . But the focus is on efficiency and delivery of services to agency constituencies. This statement has its origin in the current Open Standards Policy, effective since January 13, 2004:
The policy statement goes on to define what is meant by an open standard:
It is useful to notice a number of features of this statement:
There are great round words in the policy; they speak to the attributes sought in the adoption of technologies for the purpose of the ITD Open Initiative. Still, I am wary. The speculative notion that the mere existence of open standards implies some sort of sure thing need to be regarded critically and with healthy skepticism. I also think it is necessary to consider more carefully how much the context of usage and local practices will impact conditions of “portability” as the term is used here. The notion that affordable substitution becomes automatically available (with affordability including all costs of change-over, not just the acquisition of any products involved) is also something that takes more diligence than the easy inference recognizes.
I am concerned about over-reaching and over-justifying. I don’t want to see unwelcome surprises for those paying the bills and that might besmirch the laudable objectives of the initiative. I also concede that I am nit-picking and there is now actual experience in Massachusetts that is a better gauge on how progress is being made along these not-so-clearly-lit avenues. Going forward, it is constructive to examine how these policies are made actionable and how the benefits are secured in a form that can be measured and compared.
With those caveats in mind, let’s follow along with the foundation that ITD established.
Open-Standards Policy: The Line in the Sand?
These are the teeth in the 2004 Open-Standards Policy:
The enactment is a progressive one that is established in an evolving “Enterprise Technology Reference Model.” It is very revealing to examine that model and its progressive refinements.
Affirming the Focus
There is one more component to the Open-Standards Policy that deserves extra scrutiny [with my bracketed comments]:
Mainly, this is a policy created by an IT organization for how it sees itself as satisfying its mission by reliance on open standards. The statement must satisfy the department’s patrons as well. The position tells us about organizational sensibilities and expresses some advocacy from the IT perspective. Measures and audits would be great for calibrating and validating these positions over time.
Now we can follow along to see how these policies are applied and what the experiences and refinements are.
Risk Management and IT Policies
We have been slowly learning that one of the greatest risks in Information Technology endeavors is failure to manage and mitigate risk properly. We see evidence of it in many ways, including the remarkable failure rate of IT projects that could have been useful if they’d failed much sooner. Fail often and sooner is, in one perspective, the mark of an aggressive but healthy innovative organization. That is sometimes a luxury, especially for undertakings with accountability to the public. Shareholders rarely get much insight into the dazzling IT failures in commercial firms, but the employees can tell you.
I am not suggesting that this policy initiative is any sort of IT train-wreck. On the contrary. What I am suggesting is that the more abstract and speculative a policy-motivated approach is, the more aggressively one must look to risk management practices to see how practical safeguards and corrections are arrived at in the face of increasing experience. In the case of a public agency, it is also a responsible way to account to its constituency.
So when I write about what I’m looking for, or what one would expect to find, it is from the perspective of being able to understand how risk has been managed and how the policy-set course is being navigated in a grounded, measurable way. It is not January 2004, so it is easier to say this from the hindsight of April 2007. In fact, at this point I only have provisional impressions gained as an outsider after cursory examination of public materials. I will look more closely, and I will be careful to qualify my observations.
Coming In Part II: The ODF Fandango
 The OASIS Open Document TC was initiated on 2002–11–04. The OASIS OpenDocument v1.0 Specification was approved on 2005–05–01 with a Second Edition approved on 2006–07–19. ISO/IEC 26300:2006 the Open Document Format for Office Applications (OpenDocument v1.0) was available as an ISO Published Standard on 2006–11–30. The OpenDocument v1.1 Specification was approved on 2007–02–02 and we are all anxious to know what the defined format for spreadsheet formulas will be and whether it appears in the now-in-progress v1.2. We will then get to see how migration is accomplished from existing ODF-based spreadsheet software approaches that were taken in the absence of definitive specification in ODF v1.0–1.1.
 XHTML 1.0, the reformulation of HTML 4 into XML (although HTML 4 is not a subset), was approved on 2000–01–26 and revised on 2002–08–01. The popular HTML 4 predecessor specification, HTML 3.2, was approved on 1997–01–14. The early HTML 2.0 is described in the November, 1995, never-standardized IETF RFC 1866 (obsoleted by the Informational IETF RFC 2854 in June, 2000). HTML 2.0 is described as an application of SGML, ISO 8879:1986. SGML is based on the Generalized Markup Language (GML) devised by Charles Goldfarb at IBM in the preceding decade. For whatever reasons, SGML has not seen the widespread awareness and adoption that its descendants HTML and XML have enjoyed.
 In responding to issues raised by legislators in a review of its OpenDocument Format mandate, ITD created the 2005–09–21 “Final ETRM Version 3.5 Open Document Format Standard: Frequently Asked Questions” material. We will see that this is also a very useful document for tracking the initiative. In spots, the responses seemed a little defensive: less than forthright, including a few ingenuous claims such as “the Final ETRM applies only to documents that agencies create and save, not to documents they receive from third parties” and “The Open Document Format has been in use for several years … .” Recall that ODF v1.0 had been approved for less than five months at that time, and the first and most-complete open-source implementation, OpenOffice.org 2.0 with default ODF support, was still in beta. I see a few other howlers like those, although, overall, this is a very useful and responsive document.
You might find it interesting to do some research on how ITD was doing before they put together this initiative. It certainly was not a case of waking up one morning and saying, "Everything is fine, let's change!". They had some serious pain they were dealing with and the ETRM was their attempt to remove themselves from that pain. This is the type of decisions that private companies make every day. But when you are a public agency, this comes with public scrutiny.
(Also, a side point, didn't ISO HTML already exist at this point in time? So whether ones thinks that the W3C is a traditional standards body or not is moot.)
Rob: I can well believe that the Initiative came out of a lengthy struggle in ITD, and the ETRM is a big deal. (There's something similar at the Federal level, and my initial impression is that the ETRM is superior.) Thanks for emphasizing that.
Concerning ISO HTML, you're right. Who knew? There was ISO 15545:2000 and it is evidently a strict subset of HTML 4 (strict). I don't believe that many knew that or that anybody cared at the US Domestic level, when the W3C specs are what people use just fine and there's a lot more than ISO HTML being used.
I love the way the W3C works and what they've accomplished, second only to IETF. OASIS is a close third for me.
The bigger surprise to me is that there does not seem to be any ISO specification for XML, although there are several ISO Standards related to XML. Did I miss it?
SGML is an ISO standard, and XML is a specialization of SGML. Sending XML to ISO was discussed back around 2000, but nothing came of it.
So, I guess XML was contradictory with SGML? [I know, I couldn't resist. It is funny how one-sided these rationalizations are, though.]
Certainly C++/CLI was defeated in ISO for extending C++ in a way that allegedly caused confusion for users of C++. But no one alleged the same for XML to users of SGML. Part of this may also be due to the widespread vendor acceptance and promotion of XML, versus Microsoft's attempt to "go it alone" with C++/CLI. This clearly can make a difference.
I'm grateful that C++/CLI exists (it makes native implementation of .NET classes and methods much tidier than with JNI for Java).
How it is handled in VC++ 2005 is a nightmare for beginners, and really difficult to explain at that level, when the novice doesn't understand what the platform is, let alone distinguishing from the 3-4 that may be targetted from VC++.
That's not about the standardization effort though, although I assume it was a big lesson for Microsoft.
And, for those playing along at home, lets come back to what I find amusing about all of this. It is (1) that it doesn't matter that W3C is not a conventional standards body because there is an ISO HTML that is at best some meaningless check-off item contrasted with (2) it doesn't matter that there is no ISO XML because of the useless to most humans fact that it has a relationship to ISO SGML. (The relationship of HTML to SGML is even closer, wouldn't you say.)
I think we are reasoning backwards from the desired conclusion here, and the results are bizarre. I will watch myself on this score and I hope you and other keen-eyed folk will point it out when I slip down that slope.
What I find the most interesting in this narrative is that the pain and difficult experiences that led to adopting an open document format policy is basically hidden. It's held in the minds of those who went through it. And, as you point out, the pain and work practice adjustments that need to be made to adopt new document technologies (formats and the applications that let us use them) are never mentioned when talking about the future. But this is the kind of information that would be most helpful to others who will need to take this road (as well as illuminating to historians of technology and innovation). We desperately need to invent and deploy technologies and tools that give us easy document sharing and interchange, as well as permanent access to these document based records.
We're going to have to be honest with ourselves about the costs of technology innovation and stabilization. We know the costs when we undertake other infrastructure projects -- just think roadway or light rail construction. We can do it with technology. So let's hope that Massachusetts can be an example of how to keep notes.
Way to go, orcmid. I can't wait for parts 2 and 3.
|You are navigating Orcmid's Lair.|