Orcmid's Lair status 
privacy 
 
about 
contact 

2010-02-27

Worst Nightmare: OpenDocument Format Embraced-Extended

Once upon a time, there was a celebrated open-standards process and it brought forth OpenDocument Format 1.0, then 1.1, and some day soon, ODF 1.2. There was joy and harmony in all the lands, as dragons were driven off to lonely peaks and uncharted waters.

A great fear and moaning, even gnashing of teeth, arose as $uper $ize $oftware $ystems, Inc. ($$$$ on NASDAQ) joined the fray and raised its own banner for ODF. In early 2009, the OASIS ODF TC took fright over the prospect that $$$$ would exploit provisions in ODF 1.1 that allowed for extensions. This loophole was closed by shrinking the definition of conformance to exclude ODF 1.1-conformant extensions from conformant ODF 1.2 documents. This, along with decrying of any existing work that might be borrowed from the OOXML specifications for fear of patent torpedoes, gave solace and respite to the ODF 1.2 effort. The avaricious $$$$ dragon had been belled and kept away from sullying the open standards movement and all that portends for freedom, motherhood, civil order, and fries with that.

As the devoted mavens of the ODF TC blissfully scratch away at perfecting the careful delivery of long-awaited ODF 1.2 drafts, the foreign-elements Maginot Line is their shield. But wait, heedless of the disrepute and regulatory scrutiny that would befall $$$$ introducing private, co-opting extensions, a major new release makes its way through the low countries, summing all our fears:

Your documents work best if you use our extensions.  Deal with it!

Let me think about this. Today is February 27, 2010. Not only is the ink not dry on ODF 1.2 yet, the ink isn’t even wet on ODF 1.2 yet. I have no idea how one can tell the difference between ODF 1.2 and ODF 1.2 Extended under those conditions. It would all appear to be 1.1 Extended (maybe), and not in a nice way. One is not supposed to reuse ODF namespaces for any extensions. It is just like $$$$, it is said, to claim adherence to a standard based on their own premature implementation, forcing everyone else to deal with their quirks, unapproved extensions, and other deviations.

Will any of the eagle-eyed monitors and protectors of truth-through-openness notice? Will eyebrows be raised? Will the phones in Massachusetts and European Commission executive departments ring off the hook? Will voice mailboxes overflow? Will Boycott Novell churn out another 100-or-so posts linking to this smoking gun?

Maybe not:

All your ODF documents are belong to us

I wonder which community member we can thank for this particular bright idea. I figure that Boycott Novell will spin this, as spin there will be, if the Novell distribution commits this same abomination by relying on the same code base. Or if the Novell distribution doesn’t. Either way, there will be much ink spilt. Or maybe not.


I knew that OO.o 3.0.1 claimed to support ODF 1.2 format. I figured that was simple competitive cluelessness. Just the same, I clung to my OO.o 2.4.1. I would not have considered installing ODF 3.2 except others on OASIS TCs are using OO.o 3.x versions and we were having trouble collaborating on documents because of version differences, apparently-known bugs in pre-3.2 releases, and peculiarities of suddenly-ancient OASIS document templates. I only kept 3.0.1 around for forensic work and was not eager to upgrade at all. Still, forensic work, and collaborating with OASIS TC peers, led me to install 3.2 on Friday, 2010-02-26. (Haven’t we heard this need-to-upgrade to get-along story before?)

I routinely set OO.o 3.x installations to load/save in ODF 1.0/1.1 format. I must have seen the 1.2 Extended option, but I was busy and I didn’t think about it much. I was doing some forensic work and making protected and signed documents with both OO.o 2.4.1 and the freshly-installed OO.o 3.2.0. Even though 2.4.1 allows digital signatures, and 3.2.0 will recognize them, 3.2.0 will not itself allow signing of a document unless it is saved in what is called OO.o 3.2.0 ODF 1.2 format. So I did it that way for the protected spreadsheet I wanted to save out of 3.2.0. I did not and will not swallow the ODF 1.2 Extended pill. My 3.2.0 is now reset to Load/Save in ODF 1.0/1.1 format and it will stay that way. And I will only use it for document forensics work (and for shared-document authoring under duress).

Continuing these 1.2 claims is now at the level of travesty at worst, self-satire at best. I suppose it demonstrates that cluelessness is alive and well and thriving. I’m not putting up with it. If this mockery is in plain site, what is there that is not so easy to see?

It’s not clear what I should install on the machines of family members who only need a basic office-document software suite that will import and export Microsoft Office compatible documents. If KOffice provides an honest and good-enough independent implementation, I may start recommending and supporting that. I suppose $$$$ has no interest in putting a stake through the heart of these shenanigans, letting us and the various regulatory authorities scratch our heads and ponder just exactly what we have become willing parties to.

It has been observed that human institutions have a tendency, over time, to turn into the problem they were created to solve.

It is said that warfare teaches the victor that violence works. I say that warfare teaches the vanquished that violence works.

Have you ever noticed that our fear and distrust of others gives us permission to be what we are afraid they are?

Maybe it was ourselves we didn’t trust all along?

 
Comments:
 
Dennis, I'm not sure I can see clearly through your soaring rhetoric, but clearly you are upset. What exactly is your objection?

Surely, "extended" is a conformance class in ODF 1.2, right? If we didn't believe that such a conformance class would be implemented and used, we would not have voted to put it into the spec. This isn't like OOXML where they put in conformance classes like "strict" that no one actually implements.

From your screen capture, it looks like OpenOffice 3.2 also implements the core ODF 1.2 conformance class, the unextended version. Did you try that? Isn't a good thing that they offer the user the choice of extended or non-extended ODF?

There are reasons why a user might want extended functionality and there are reasons why they might want to limit themselves to the unextended schema. Ditto for a vendor who needs to support customers with diverse needs.
 
 
I think the first image stands on its own. What is this 1.2 and 1.2 Extended (recommended!) whereof it speaks?

I focus on ODF because that is where I put my standards-development energies.

I'm not a participant in the ECMA TC, SC34 WG4, or any National Body mirror subcommittee. I was not at the BRM either, but my impression is that the strict-transitional separation was a creature of the BRM in early 2008. The current released OOXML products that I run across all appear to be honestly-transitional. I haven't reviewed the implementer notes for whatever the deviations are.

I think the namespace change between strict and transitional is rather more recent. Seems like a good idea to me. I have no idea what sort of spanner that puts in the effort to migrate products and deal with transitional (and before-OOXML) legacy documents and products. It's not the game I'm playing in. I didn't think you were either.
 
 
What does the picture mean? I'm taking it literally, in which case the simplest interpretation would be that they are referring to what the latest draft of ODF 1.2 Part 1, section 22.2.2 calls a "Conforming OpenDocument Extended Document".

Unless you're seeing something else here that has escaped my casual inspection, I don't see the source of excitement. As I said before, we added that conformance class, as well as its non-extended versions, with the intent that either could be implemented. It appears that OpenOffice 3.2 implements both. Is that so bad?

Or is your concern that they are saying "ODF 1.2" rather than "ODF 1.2 (draft)" or something like that? If so, I agree that it wold be better to indicate that it is draft and could still change, like the way wireless "N" routers were sold with "draft" support.
 
 
@ Rob Weir: "What exactly is your objection?"

Surely you are not as thick-headed as the impression you leave. Look at the first screen grab and note the line that says, "Not using ODF 1.2 Extended may cause information to be lost."

In other words, no compatibility mode implemented for ODF 1.2 Strict. If users choose ODF 1.2 Strict, users will have features available in the GUI that do not write the needed markup to the file. And when they save ODF 1.2 Extended files received from others on their own instances of OOo/StarOffice/Symphony/Notes/Domino, set to save documents ODF 1.2 Strict, data will be lost without warning.

And of course earlier versions of OOo carried the same warning about selecting ODF 1.0/1.1 as the default file save format.

I would have more patience for your expressed mystification had you and I not discussed this topic at some length previously, had I not submitted a comment on this precise topic on the OASIS office comment list, and had Microsoft not been so roundly and loudly criticized recently for a similar message in the choice menu for its MS Office 2010 ODF 1.1 vs. OOXML support.

Dennis accurately characterizes the problem as an embrace and extend play. That's accurate, but it is also accurate to characterize it as a vendor lock-in, forced upgrade play.

One need look no farther than the menus in OOWriter after selecting File > New > HTML Document to see that IBM and Sun know darn well how to implement a compatibility mode in the GUI to produce documents in different formats without inflicting fear and the reality of data loss.

What IBM and Sun have done is to produce a new OOo version that has only non-lossy read support for ODF 1.2 Strict. The write support for that format is lossy.

IBM and Sun have just killed ODF 1.1 Strict via their market share with an embrace and extend maneuver. So much for the claimed benefits of your claimed "incremental improvement" approach to ODF interoperability issues used an excuse to maintain ODF interoperability barriers.

Just more proof positive of Sun and IBM's anti-ODF interop agenda to keep their code base at the center of the ODF universe.
 
 
@marbux,

Certainly if you receive extended document and save as ODF 1.2, the extensions will be lost. I know this must cause you continued frustration since your over-hyped and vaporous 'da Vinci plugin" at the ill-fated and long defunct OpenDocument Foundation was predicated on every ODF application in existence preserving every extension in existence. But that really didn't work, did it?

In any case, I don't read this dialog as saying information will be thrown out without warning. I don't see anything here that would be incompatible with a "compatibility ode". Where do you see otherwise?

I think you would need to do some testing to show that. But just spilling gallons of digital ink over a dozen words in a settings dialog is futile. If you want to show a problem, show what the problem is. Post some files or give a walk-through to show where a specific problem is. If there is one, I'm sure the OpenOffice.org team would love to hear about it.
 
 
@ Rob: "Certainly if you receive extended document and save as ODF 1.2 [Strict], the extensions will be lost."

Thanks for admitting that much, Rob. Now please explain how ODF 1.2 Strict users will know when they are opening an ODF 1.2 Extended document and the contortions they will need to go through to save it as Extended.

But the problem runs far deeper. A few days ago IBM VP Bob Sutor tweeted about an article discussing the MS Office 2010 ballot screen. http://twitter.com/bob_sutor/status/9486748637

From the article Sutor linked:

===

In May 2009, the ODF alliance announced that Microsoft's support for OpenDocument (ODF) needed substantial improvements for real-world interoperability. "Putting potentially millions of ODF files into circulation that are non-interoperable and incompatible with the ODF support provided by other vendors is a recipe for fragmentation," said ODF alliance spokespeople.

In an email to Neowin, Marino Marcich, executive director of the ODF Alliance said he believed the ballot screen falls short in several areas. Comparing it to the browser ballot screen for Windows 7, Marcich said: "Microsoft offered the EU a ballot screen that gave the browser choices in randomized order, with an unbiased message, including a link for further information provided by the vendor. But the file format ballot screen gives OOXML the first position. It gives a biased description of ODF, listing the liabilities of Microsoft's ODF implementation while failing to state any of ODF's advantages."

===

We are now discussing the OpenOffice.org ballot screen for, inter alia, ODF 1.2 Strict and ODF 1.2 Extended. But in the quotes from the article above, it seems we could, without introducing any error, substitute "Sun/IBM" for "Microsoft," "ODF 1.2 Extended" for "OOXML," and "ODF 1.2 Strict" for "ODF."

Flooding the market with documents that provide only lossy compatibility with other apps is bad when Microsoft does it, yes? Why then is it any less evil when IBM and Sun do it?

Simply warning users that data may be lost if they elect to save ODF 1.2 Strict biases the selection process against it and provides no information about the advantages of ODF 1.2 Strict, yes?

IBM and Sun are trying to make ODF 1.2 Strict sound like a bad choice, so that few users will really use ODF 1.2 Strict. We should conclude that IBM/Sun want to be able to say they offer ODF 1.2 Strict support, while having few users for it, yes?

Or were RMS' and the ODF Alliance's quoted statements somehow irrational?

The problem lies in the ODF specification that allows this kind of anti-competitive manuevering, Rob, not in the OOo implementation. As you correctly argued yourself in regard to the OOXML standard:

"Thanks, Doug. But none of that is really 100% compatibility with legacy anything. That is really just saying that OOXML is compatible with code that Microsoft is writing months after OOXML was standardized by Ecma. But the qualities of the format were set the day the standard was approved by Ecma. The standard does not gain capabilities by Microsoft writing code. Microsoft applications may gain capabilities, but the standard is what it is, and is as compatible as it is going to get the day it was standardized. If OOXML was really compatible with legacy binary formats then they would work without requiring code changes or customer upgrades."

http://www.robweir.com/blog/2007/03/compatibility-according-to-humpty.html

Same for OOo and ODF 1.12 Strict, yes? Embracing and extending specifications is just as evil whether done by Microsoft, IBM, or Sun, yes?

File bug reports with the OOo developers? "The standard does not gain capabilities by [IBM/Sun] writing code. [IBM/Sun] applications may gain capabilities, but the standard is what it is, and is as compatible as it is going to get the day it was standardized."
 
 
Marbux, this is a settings dialog, not a ballot screen created as part of an anti-trust settlement with the EU.

In any case, this is no different than what we've had since ODF 1.0, 5 years ago. That allowed extensions in ODF just as ODF 1.2 does. The only difference is that in ODF 1.2 we define two conformance classes, extended, and unextended documents, whereas before they were all lumped together into the same conformance clause. I think this is an improvement, since it allows an adopter to mandate a particular conformance class according to their needs.
 
 
@ Rob: You have addressed only a single point I made in my last comment and ignore your own contrary prior statement.

The Office 2010 ballot screen is in fact a settings dialog for the selection of the default file save format, yes? So is the OOo dialog under discussion. That distinction you draw between them is irrelevant from the user's perspective except that OOo makes the user dig deep in the user interface for the setting whereas Office 2010 presents the choice the first time the user launches the app.

You also attempt to distinguish the situation by branding the Microsoft's ballot screen as a settlement condition in E.U. antitrust proceedings, as though the ODF TC were immune from antitrust law. Antitrust law is not restricted to those with a dominant market share. I will remind you once again that the ODF TC is an "agreement among undertakings" subject to antitrust regulation under Article 81 of the treaty establishing the European Union. See e.g., X/Open Group -- 87/69/EEC: IV/31.458 (15 December 1986), http://tinyurl.com/yfuejuw (crucial nature of interoperabiility specifications in lawful standards work).

The ODF TC's conduct is also subject to potential antitrust regulation in the U.S. as a "conspiracy in restraint of trade." Allied Tube & Conduit v. Indian Head, Inc., 486 U.S. 492 (1988), http://laws.findlaw.com/us/486/492.html ("trade and standard-setting associations routinely treated as continuing conspiracies of their members") (internal citation omitted).

The ODF specification itself is subject to mirroring competition regulation under international law, the Agreement on Technical Barriers to Trade, http://www.wto.org/english/docs_e/legal_e/17-tbt_e.htm (comprehensively prohibiting technical standards with "the effect of creating unnecessary obstacles to international trade." Enforcement proceedings may be initiated either at the national level or via the World Trade Organization ("WTO") dispute resolution process.

In that regard, the WTO Appellate Body has unambiguously ruled that a technical regulation (and by unavoidable implication an international standard) must specify: [i] "any objectively definable 'features', 'qualities', 'attributes', or other 'distinguishing mark'" [ii] of an identifiable product or group of products [iii] only in mandatory "must" or "must not" terms. WTDS 135 EC - Asbestos, (World Trade Organization Appellate Body, http://tinyurl.com/ylrpgtm (12 March 2001; HTML version; para. 66-70). There is no principled argument that vendor defined extensions to an international IT standard may be lawfully be classified as conformant because the extensions by definition are not defined by the standard.

Or, as that principle was translated into the IT context by ISO/IEC JTC 1 Directives, http://www.wto.org/english/docs_e/legal_e/17-tbt_e.htm (,) an international standard is to "specify clearly and unambiguously the conformity requirements that are essential to achieve the interoperability."

As to interoperability's vital importance to competition, you have no lack of practical understanding. See e.g., Rob Weir, Those Who Forget Santayana, An Antic Disposition (20 December 2007), http://tinyurl.com/2ym2q8  ("Those who control the exchange format, can control interoperability and turn it on or off like a water faucet to meet their business objectives").

Embrace, extend, extinguish. Bad when Microsoft does it; bad when IBM and Sun do it. Bad when the ODF TC purports to authorize it. But interoperability isn't an optional feature of an IT standard. It's a minimum legal requirement to maximize competition.

So your deflecting excuse that the Microsoft ballot screen was part of an antitrust settlement doesn't hold water. The bug is in the ODF specification, not in OOo 3.2, which only manifests the bug in the specification.

Fix the bug, please.
 
Post a Comment
 
Construction Zone (Hard Hat Area) You are navigating Orcmid's Lair.

template created 2002-10-28-07:25 -0800 (pst) by orcmid
$$Author: Orcmid $
$$Date: 10-04-05 13:03 $
$$Revision: 1 $