Orcmid's Lair status 
privacy 
 
about 
contact 

2008-06-26

Interoperability by Regulation: The Glass Half Full

[update 2008-07-01T03:16Z Some additional updates were announced June 30.  I account for those and the trade-press activity around them already.  There is renaming to Open Specifications generally, not to be confused with the Open Specification Promise (although there is some overlap).
 update 2008-06-28T22:34Z I added additional resources noticed while reviewing material on the Microsoft Interoperability Forums threads.]

This is a very long post.  I work through the history of Microsoft compliance with the consent decree and November 2002 final judgment in the civil anti-trust suit.  I wanted to confirm how this is a continuous improvement activity with the provision of protocol specifications and licenses at center stage now that other major compliance actions have been carried out and are being monitored for continued satisfaction. 

Because there is a perception, promulgated in what passes as the trade press, that Microsoft has not delivered and is perpetually promising delivery some day, I wanted to confirm the history and progress that has been made and that is easy to confirm using official documents and the online, public downloads of the Microsoft protocol documentation.

I also wanted to affirm how much this is a journey, it may not have a decisive end point, and the work to refine and correct and respond to difficulties in use of the documentation is an ever-expanding effort for good cause.

For simplicity, I do not track the parallel events that arose in conjunction with the European Commission judgment against Microsoft.   This progression also maps to the availability of protocol documentation in compliance there also.

1. The Story So Far
    1.1 The development of protocol specifications and licenses
    1.2 The Technical Committee assessments
    1.3 Useful review
    1.4 The progressive expansion of technical documentation (long)
2. The Current Situation
3. Public Perceptions and Debate
    3.1 My appraisal
    3.2 The eternal spotlessness of lazy minds

1. The Story So Far

1.1 The Development of Protocol Specifications and Licenses

The carrying out of the final judgment and consent decree in the civil anti-trust case between Microsoft and the U.S. Department of Justice (and other plaintiffs) is over-seen by a Federal Court judge.  Part of the on-going review of the settlement involves a requirement that Microsoft document and license, under agreeable non-discriminatory terms, a variety of protocols used between separate Microsoft products.  There is overlap with similar judgments that arose in Europe.  The idea is to encourage competition by those who would employ the same protocols to also interoperate with Microsoft products and even offer competing interoperable products as substitutes and complements.

Until this year, if we were not someone involved in their development, licensed use, or review, we only knew anything of the process of delivering and improving the necessary specifications through progress reports and other communications from Microsoft, over-sight bodies, and other authorities.  It's a bit like watching a football game during a heavy, pea-soup fog without being able to see the players and even know how the game is played.  To top it off, the various announcers and commentators don't seem to be at the same game.

This year, the more-technical members of the general public are able to observe the delivery of such specifications in a direct way.  As part of the February 2008 declaration of Interoperability Principles Microsoft provided the already-delivered documentation for on-line public access.  This documentation was developed as part of the compliance effort.  The initial materials consisted of 30,000 pages of documentation.  In April, over 14,000 additional pages of drafts for additional protocols involving more-recent products were added.  There are also on-line forums for discussion of the protocols, questions about details of the specification, and so on.  [As a point of calibration, I am not an insider in this activity nor have I taken advantage of the available protocol documentation.  I do have a personal curiosity about the FrontPage extensions and the patented Microsoft extensions to WebDAV.]

Licensing for all of these protocols and their formats where Microsoft has applicable patents is also provided under reasonable and non-discriminatory (RAND) terms, but you no longer need a license to access the specifications.   Final specifications are promised as future updates in this ongoing Open Protocol Specification activity.

These are all in addition to the specifications that Microsoft has made available under the Open Specification Promise and which have been steadily growing since 2006.

    • additional resources:

      MCPP - Microsoft Communications Protocol Program (via Interoperability Forum).   This is the program for licensing and specifications that was originated in response to the U.S. consent decree and final judgment of 2002.  In addition to covering the current support for licensees under the program, there is now a map of individual protocols against applicable patents (June 9, 2008 PDF download).
        
      WSPP - Microsoft Work Group Server Protocol Program (via Jason Perlow and Live Search).  This is the program for licensing and specifications that was originated in response to the European Commission decision of 2004.  These specifications are also available under the Open Protocol Specification activity.  As part of the available documentation, there is a map of individual protocols against US and European patents (April 29, 2008 PDF download).  There is overlap between WSPP and MCPP.
         
      Microsoft Protocol Programs (via MSDN).  [2008-06-30 update] This covers all protocol specifications that are available, including those involving the Microsoft Office System, Sharepoint Server, and Exchange Server.  There are also additional patent licenses and patent maps for the protocols of these these products and protocols of Windows Vista (including the .NET Framework), and Windows Server 2008.  The patent maps provide check marks indicating whether there are patents or patent applications applicable to each itemized protocol.  The MCPP and WSPP patent maps provide identifying numbers for patents and patent applications of those protocols on these lists that are also covered under MCPP or WSPP.
        
      Open Specifications (via Doug Mahugh).  [2008-06-30 update] This is the location for downloading of all material provided as Microsoft open specifications, including additional formats (office binary and XAML) and computer languages (VBA) not covered by industry standards.
         
      Open Specifications, MSDN Forums.  [2008-06-30 update] Forums (new location) for discussion the open specifications, using and interfacing with the related technologies, including use and implementation of the Microsoft Office binary file formats.

1.2 The Technical Committee Assessments

Meanwhile, there is a three-person Technical Committee (TC), plus one expert assigned by the "California Group" of plaintiffs, that reviews the specifications as they are released and provides feedback to Microsoft and the Department of Justice.  The TC has its own staff.  The TC has broad access to Microsoft and its technical materials.  The TC was appointed in 2003 and the TC term has been extended until November 12, 2009, the same date that already-extended portions of the final judgment will expire.  (Some elements of the final judgment expired in 2007.)  There is provision for further extension of the judgment and the work of the TC until as late as November 12, 2012.  [My intuition is that such incremental extensions will occur.]  Microsoft has declared that it will make no objection to such extensions.

The purpose of the TC includes assessing the technical availability, completeness and usability of the specifications for successful use by third parties.  These reviews lead to requests for rewrites and other improvements in the materials and their coverage.  The TC also develops tests that are usable in some degree of interoperability verification and are being made available to Microsoft for its use as well.  Findings of the TC inform the court, Microsoft, and the Department of Justice (and other plaintiffs) of the progress and status of rewriting specifications and other activities to further the support for interoperability.  Operation of the TC is part of the oversight process agreed among Microsoft, the Department of Justice, other plaintiffs, and incorporated in the final judgment. 

On June 17, 2008, the latest Joint Status Report was filed with the court.  This is the 17th such report (including interim reports as well as the twice-yearly ones required specifically by the court).  The previous one was on February 29, 2008.  The first one was on July 3, 2003.  The complete list and texts of status reports and related documents are available on the U.S. Department of Justice web page for the case.

1.3 Useful Review

To understand the scope of all effort involved in carrying out terms of the consent decree and the final judgment, it is very useful to review the first Joint Status Report of July 3, 2003.  There are far-reaching provisions that are implemented and constantly being reviewed.  With regard to the provision of communication protocol documentation, the initial materials consisted of 5,000 pages, representing 90 man-months of technical writing effort, made available in August, 2002.  At that point having the technical documentation was but a small part of the compliance activity being carried out.

It is clear from the outset that there is a valuable iterative process by which proposed compliance approaches, license structures, and resolution of received complaints are worked out.  The progression of effort is quite revealing.

1.4 The Progressive Expansion of Technical Documentation

Although there were a number of protocol licensees by the time of the third report, January 16, 2004, there was also a complaint regarding "the sufficiency and completeness of the Technical Documentation that Microsoft provides." 

By the April 14, 2004 interim status, the plaintiffs concluded that "the technical documentation needs substantial revision in order to ensure that it is usable by licensees across a broad range of implementations ... ."  In its response, Microsoft recognized "that there is (and likely always will be) room for improvement given the scale and complexity of the undertaking. Microsoft anticipates that modifications and enhancements to the technical documentation will be an on-going process as Microsoft and the licensees gain more experience in working with the documentation, new versions of protocols are released, related technical information resources that Microsoft makes available are modified, and so forth."

According to the July 9, 2004 status, development of measures for completeness and a plan to revise the documentation by Fall, 2004 were underway.   By the October 10, 2004 status, revised documents were being placed in beta release for review and concerns over the format of the documents were being addressed. 

On January 25, 2005, it was reported that revised documentation had been made available in December, 2004. "Plaintiffs believed that the revised documentation represented a significant improvement over the original documentation. Plaintiffs nevertheless remained concerned that further work on the documentation was needed to provide confidence that licensees would be able to implement the relevant protocols."  At this time, there was a significant expansion of technical developments to be used in verifying the protocol documentation:

"The TC's prototype implementation project and Microsoft's parser development project are designed to capture different types of potential issues with the technical documentation. The two projects will therefore proceed on parallel tracks. Plaintiffs and Microsoft believe that the two projects, combined, provide a realistic mechanism to ensure the overall completeness and accuracy of the documentation. Given the substantial scope of these projects, we propose to complete them in approximately one year. Licensees, however, should realize benefits from the projects well before their completion. As issues with the documentation are discovered during the projects, Microsoft will make the appropriate changes to the documentation and release updated versions of the documentation to MCPP licensees on a regular basis. Accordingly, the documentation available to licensees will improve in quality and accuracy throughout the course of the project work. Plaintiffs believe that this plan will provide an expeditious and effective way to improve the technical documentation."

According to the October 19, 2005 status, the TC projected that the prototype implementation would not be complete until July 2007.  The extension was required by ramp-up delays, recognition of greater effort required, and finding more difficulties than expected with the available specifications.   Microsoft offered these supplementary comments to what it agreed was a successful process:

"As Microsoft indicated at the outset of the process, creating from scratch technical documentation of the magnitude, scope, and complexity that [the final judgment] requires would be a substantial—and ongoing—undertaking. At the time the technical documentation was created, and still today, Microsoft has been confident in its accuracy and usability. To date, Microsoft is unaware of any existing or potential licensee that has been unable to use any Communication Protocol because of flaws in the technical documentation. Nonetheless, as Microsoft has consistently advised the Court, Microsoft also expected and understood that the documentation would not be perfect, which is why the license agreements include the Correction Assistance provisions to address just such imperfections.

"Additionally, in every case of which Microsoft is aware, use of new documentation will inevitably uncover 'bugs' and only through experience can those bugs be corrected and the documentation improved. The Prototype Implementation Project undertaken by the Technical Committee, with the participation of Mr. Hunt, is designed to expedite the process of uncovering and correcting bugs. As indicated by Plaintiffs, the project seems to be a success in that regard. Moreover, by establishing that software developers using the documentation can develop implementations of the Communications Protocols, the project has served to verify the usefulness of the documentation."

The Microsoft protocol parser project was having serious difficulties and was being re-planned and redirected to provide useful results using alternative approaches, leading to a November 18, 2005 Microsoft Supplemental Report to the court.  A January 23, 2006 Plaintiff Report added that there was a serious situation in the drop-off of responsiveness in handling of technical documentation issues.  There were also difficulties around the setup and configuration of a test center in India.

By the February 8, 2006 interim status, Microsoft has offered to provide Windows server source-code access at no extra charge to licensees.  This is proposed as a way to overcome the difficulties in maintaining the desired rate of responses to the technical documentation issues while the documentation effort is expanded.  [Ed. note: I am amazed that this was considered.  There are many reasons why reversion to the code as the authority is a bad idea, although it is difficult to argue with the desire to provide schedule relief.  It is important to understand that the Court's measure of whether the specifications are successful and the remedies of the final judgment are effective has to do with the arrival of substantial competitive products that employ the protocols; the judicial clock is ticking.] 

According to the May 12, 2006 status report, the availability of source code and a review of outstanding issues shows improvement, but not the high-level of resolution experienced previously.  Microsoft proposes that resolving the technical documentation issues and making revisions one-by-one is ineffective.  In considering a "reset" recommendation from Microsoft Senior Vice President Bob Muglia, the plaintiff's agree that a project to completely rewrite the documentation is preferable to the current revision approach under the conditions that "(1) the parties must agree on a specification for the new documentation; (2) the parties must agree on a project schedule for all the protocols; (3) Microsoft must rewrite the documentation for each protocol according to this schedule; and (4) the resulting new documentation must be tested and validated to ensure its completeness and accuracy. As it is clear that this project to rewrite the technical documentation will require a significant amount of time to complete, ... Plaintiffs have informed Microsoft that an extension of portions of the Final Judgments will be necessary, and Microsoft has consented to such an extension."  The extended portions were originally scheduled to expire in November, 2007. 

There is also rewriting required to satisfy the European Commission, so one could see how appealing a single, comprehensive rewrite could be.  The US plaintiff's are impressed by the samples they are shown, but the current approach must continue until it is clear that the rewrite is producing what is needed.   At this time, there is also new initiative to promote adoption of the protocols in third-party products:

"Microsoft will also move forward this year to create a new interoperability lab in which licensees can test and debug their protocols and obtain easy access to on-site Microsoft engineering assistance. The lab will provide a testing facility, training, best practices, trouble-shooting and technical support for licensees working on the implementation of protocols using the MCPP documentation. Microsoft will also sponsor 'plug-fest' events, which take place regularly in the information technology sector and at which a variety of companies can test and debug their protocol implementations with the goal of improving interoperability. Plaintiffs and Microsoft agree that these steps can promote interoperability. "

According to the August 30, 2006 interim report, the plaintiffs and the European Commission are coordinating and Microsoft is estimating that the rewriting of the protocol documentation will occur in stages with completion in mid-2007.  The plaintiffs are concerned with the estimate being so long.  There is also to be addition of protocols and APIs that will be arriving with "Longhorn Server," the successor to Windows Server 2003.  The TC and staff have grown to 40 people.  The Microsoft team has grown to 245 Microsoft employees and contingent staff.  The Microsoft Parser project has recovered and is making staged deliveries.  The TC prototype work is now dependent on availability of the new, rewritten technical documentation.

According to the November 21, 2006 status, the first rewritten documents delivered in the first of five initial-availability milestones are recognized as improvements, although there is some disagreement on the template that was agreed to.  This is worked out among the parties, along with an agreed review and correction process.  Microsoft warns that the current schedule is aggressive and contains no buffer.  The fifth milestone is targeted for May 29, 2007, with an online build of that milestone's reviewed/revised documents 90 days later.  The Microsoft staffing has grown to 260.

According to the March 6, 2007 interim status, the first three initial-availability milestones were met on schedule.  Microsoft has requested insertion of an additional "Longhorn milestone" before the original 4th and 5th milestones.  This moves the final initial-availability to July 20 2007 with online build later.  The Microsoft staffing has grown to 313.  Microsoft notes that "the number of pages of documentation that Microsoft now expects to deliver as part of the rewrite project is significantly larger -- likely thousands of pages larger -- than Microsoft originally anticipated in Fall 2006." 

"The increase in the volume of the documentation is due to two primary factors: 1) the need to document new protocols that Microsoft identified as part of an internal audit, and 2) a conclusion by Microsoft that overall quality of the documentation could be improved by providing more comprehensive and detailed descriptions for a certain number of existing protocols."

On June 19, 2007 work is continuing.  The TC decides it needs to hire some consultants for auditing Microsoft's identification of applicable protocols, following the discovery of some omitted ones.  The plaintiffs repeat, at each status report since the rewrite started, that they will not be able to determine that the rewritten documents are acceptable until the full set is available.   [Actual take-up in competing use appears to remain a shadow over the effort.]

According to the August 31, 2007 interim status, "the technical documentation project has proceeded according to schedule and Plaintiffs are encouraged by the quality of the new documents."  The TC noticed that a number of protocol elements were removed from the new documents.  The TC had expected that they would have been consulted before that was done.  Microsoft agrees to locate all removals and review them with the TC.  The online build of Milestone 5 is scheduled for September 28, 2007 [on schedule].  "Microsoft has committed to producing, at the same time, a final set of overview documents that explain how all of the protocols work together. As these documents are essential to ensure that licensees can make full use of the documentation to effectively implement all of the communications protocols, the TC will be reviewing them closely when they are produced."  The Microsoft staffing has grown to 630.

By the February 29, 2008 status, additional protocols are being documented to cover an update to Windows Server 2008 (the "Longhorn" server) and other documents are to be updated by March 14, 2008.  The protocol documents are now provided as part of Microsoft's Open Protocol Specifications under the Interoperability Principles.

"Plaintiffs are satisfied that Microsoft has maintained the overall quality of these documents through the final milestone. Microsoft will continue to improve the documents over time, based on feedback from the TC's implementation and validation projects and from Microsoft's own test suite project. This process is proceeding without major problems and is important in ensuring the overall quality of the documentation."

At this point there is disagreement on the depth and content needed in the overview documents.  The Plaintiffs and the TC assert that the quality of the overviews is critical and the scenarios need to be expanded along with other improvements.  A template and revised contents are being negotiated with Microsoft.   The Microsoft staffing has grown to 650.

2. The Current Situation

It is important to appreciate that major, extensive measures were required for compliance with the final judgment and the consent decree.  In this broader context, the development of documentation and licensing of protocols were defined in but a few paragraphs of the final judgment.

Although I have mainly traced the progression of the protocol documentation effort, its growing significance is a consequence of all of the other actions having been successfully carried out, with their ongoing review and confirmation taking on a routine quality.  There are still complaints that end up needing to be resolved and there are also many technical documentation issues being tracked.  This should be understood in the same way as bugs are identified and tracked in software:  there is no end to it, and some issues seem to persist far longer than others.  Nevertheless, a systematic process for identifying issues, agreeing on resolution, and revising the documentation is in place.

According to the plaintiffs in the latest, June 17, 2008, Joint Status Report on Microsoft's Compliance with the Final Judgment, an interim report,

"Plaintiffs' work ... and the Microsoft Communications Protocol Program ("MCPP") continues to center on efforts to improve the technical documentation provided to licensees. In particular, Plaintiffs, in conjunction with the Technical Committee ("TC") and Craig Hunt, the California Group's technical expert, are reviewing the results of Microsoft's project to rewrite the technical documentation, described in detail in previous status reports, and are providing feedback to Microsoft on what additional work is still needed.

"Since the prior Joint Status Report, Microsoft has completed producing all of the documents in the Milestone schedule, including the last group of documents that were added to cover an update to the Windows Server 2008 ("Longhorn") product. Microsoft continues to address issues in the documents that have been identified by the TC and by Microsoft itself; this work will continue over time in order to ensure the overall quality of the documentation."

The plaintiffs identify three areas of current concern:

  1. "Microsoft removed a number of protocol elements that were included in previous versions of the documentation. When this same issue arose last year, Microsoft and the TC discussed that Microsoft would not remove protocol elements from the documentation without first discussing it with the TC in order to ensure that there was no substantive disagreement. ... It is important for the stability of the documentation that the TC review the proposed deletions before they occur, as Microsoft and the TC previously agreed."  [The original discrepancies and remedy were reported in the August 31, 2007, interim status.]
      
  2. "TC has suggested to Microsoft that it would be extremely beneficial to the TC and licensees to create a mechanism for detailing changes between versions of the documentation. Currently, it is difficult to tell exactly what has changed when Microsoft releases a new version of the documentation. This slows down the TC in its work by making it difficult to evaluate revisions to the documentation and causes issues such as the one discussed in the previous paragraph, where it is difficult for the TC (and Microsoft itself) to determine whether protocol elements have been removed from the documentation. Licensees have also informed the TC that the absence of version-to-version change information complicates product development. Microsoft was receptive to the TC's suggestion and will work with the TC to develop an effective mechanism to track changes to the documentation."
     
  3. Document revisions were moved to a quarterly release cycle.  This is found to be too long and the parties are working to provide revision updates more frequently.

These seem to be natural occurrences as part of growing experience.  The "regression" in regard to removal of protocol elements from the documents, and difficulty of identifying changes, is an important area for process improvement and better accountability.   In the context of the overall progression from the 5,000 pages of documentation provided in 2002, I see this as a sign of maturation of an ongoing, iterative process for producing something that has no ideal measure of perfection.  The practical measure is whether or not licensees and others are able to use the documentation successfully.  This is also a matter for debate, with Microsoft agreeing to produce more while asserting that the available material is sufficient.  This depends, I suppose, on the determination of licensees.  The greatest practical concern appears to be the desire for determined licensees to appear on the scene and exploration of how that can be encouraged.

In this regard, the greatest outstanding technical-documentation activity is with regard to the additional overview documents which were first introduced in September, 2007.  Following the concerns expressed in the February, 2008, status report, the plaintiffs report the planned resolution:

"Microsoft agreed to create a set of additional "system" documents which would provide more detailed information on the interaction between the protocols in a number of complex scenarios. At the end of March, Microsoft provided the TC with three pilot system documents to evaluate Microsoft's proposed template for the new system documents. The TC recently gave Microsoft feedback on the pilot documents by identifying a number of technical documentation issues; as a general matter, the TC was concerned with the overall quality of the pilot documents.  ... The TC and Microsoft will discuss over the next few weeks what ... steps should be taken to ensure that subsequent documents meet everyone's expectations in terms of quality.

"Microsoft has developed a list of nineteen system documents it plans to create and developed a rough, high-level schedule for producing these documents. As Microsoft describes in its section of this report, Microsoft plans to publish drafts of all nineteen documents by the end of March 2009 and to publish the final version of all system documents by the end of June 2009. Once Microsoft and the TC finalize the template (or templates) for the system documents, Microsoft will develop a more detailed schedule with a number of identifiable deliverables over time. As with the earlier phases of the reset project, establishing individual milestones for particular groups of system documents will allow Plaintiffs and the TC to assess the quality of the newly-created documents in a more orderly way, to provide Microsoft with timely feedback, and to monitor Microsoft's progress in meeting its schedule commitments. Plaintiffs and Microsoft will provide the Court with this detailed plan when it is available. As with the reset project itself, experience in preparing the system documents over time may result in changes to the current list of nineteen documents such as adding additional documents, combining documents, or shifting subject matter between documents."

It needs to be understood that this is new, additional work, that the need for it was identified by Microsoft in 2007, and that the TC sees its quality as being critical.  The TC also expects to see issues dealt with here that Microsoft had earlier declared would best be handled in what were then called overview (now "system") documents.  In that context, the original rewrite is considered to be complete and now in what I would call continuation and maintenance status.  It would not be unusual if the creation of the system documents led to some revision back into the protocol documents.

For its part, Microsoft reports that, as of June 1, there have been 146,000 downloads of the on-line, public documents.  Microsoft staff for development of the documentation and testing of the protocols is now at 750.

Finally, Microsoft's tracking of Technical Documentation Issues (TDIs) is reported on a per-period basis in the status reports.  In reviewing these numbers, it is important to notice that previously-closed TDIs are not accumulated.  It would be very useful to review all of those reports and see how this works as a progressive activity.  Although new TDIs are always coming in (including ones identified by Microsoft itself and by Microsoft in addressing queries from licensees), it is important to understand that this is very much like a bug tracking history.  Showing only per-period activity tends to remove important perspective on how well this is working as a progression over time.  I am curious to see if the increasing availability and visibility of the documents is also resulting in an increase in identification of TDIs.  I'm not ready to fire up Excel just yet, though.

3. Public Perceptions and Debate

3.1 My Appraisal

When I first saw the popularly-cited June 21 commentary by Matt Assay, I had not reviewed anything but the current, June 17, interim status.  That was enough for me to assert the following in a comment that same day:

This is an iterative process, including review for acceptability, requests for changes, and tightening of the process. ...

One problem with specs developed (often after the fact) by the implementers is that there is a great deal of tacit understanding that can only be caught by outside parties.

My take is that this is a mammoth undertaking, it takes great effort to shift to an interoperability-focused approach, and this is a journey, not an end point.  There may never be a time when there is a nice bow tied around everything.

What I find encouraging is that all parties seem to acknowledge the progress that is made and how course-correction is being put in at each iteration.

It is magical [thinking] to expect that this could be any different. Go look for specifications anywhere in the industry that satisfy the implicit assumption ... that there is some perfect state available. The stable ones that I know of (TCP/IP, say) all had birth and growing pains before settling down. It is natural to expect, even for Microsoft protocols that are a bit long in the tooth (but still evolving).

I should add that attempting to develop comprehensive specifications without testing by outsiders is a guarantee of wasted work.  The risk is over-explaining something that is obvious and/or irrelevant to others and overlooking those tacitly understood matters that outsiders will stub their toes on.  Finally, it is not possible to write perfect specifications, although they can be evolved to be good enough.  In this case, the "good enough" test is ensnarled with the Court's desire to see competition fostered, something that the documents can facilitate, but not cause.

Having gone deeper here, tracing through the entire history of this aspect of Microsoft compliance with the final judgment, it is clear to me how much this has been a learning experience, with several course corrections and expansions along the way. 

  • The first documentation, available to licensees in 2002, consisted of 5,000 pages produced by a team of 10 technical writers working for 9 months.  This documentation was apparently written from scratch.  This is incredulous for some.  I find it unsurprising since interoperability with oneself can be accomplished by easier means, especially for process-adverse software developers (although such means do not scale and are perilous for preserving interoperability-over-time).
      
  • Attempting to improve the original documentation by revision updates, initiated in 2004, broke down as unsustainable by 2006.  A parallel "reset," proposed by Microsoft, was initiated to completely rewrite the documentation and also add additional protocols and updates for newer Microsoft products (such as "Longhorn" server and updates to its release as Microsoft Server 2008).  Important efforts to provide a prototype implementation of the protocols and to introduce a protocol parser, already recognized in 2005, were continued as indispensable.
      
  • The set of rewritten technical documents was completed at the end of  September 2007.  Those documents, with editorial updates, were made available as online, public downloads, in February, 2008.   The documents, now consisting of more than 41,000 pages, are now under maintenance.  These are the documents that current licenses apply to and that are the subject of discussion on on-line forums created for that purpose.
      
  • In August 2007, the need to provide overview documents was also identified.   In February 2008, review of the overview documents led the TC to insist on greater quality and comprehensiveness in the overviews, leading to the current (June 2008) schedule for the expanded set of additional system documents to be completed by the end of June 2009. 
      
  • [Updated 2008-06-30] Microsoft published, as promised for June, version 1.0 specifications replacing a variety of drafts first published in April 2008, involving Microsoft Office, Sharepoint Server, Exchange Server, Vista (with .NET Framework), and Windows Server 2008.  The drafts on the Microsoft Office binary formats were also updated.  This is evidence of the ongoing progressive refinement and maintenance of protocol specifications. This extends to availability of other specifications that are not available as published standards: the Office binary formats, XAML, and Visual Basic for Applications (VBA).

I wouldn't be surprised to see wider availability of tests, examples, and even tutorials before this work is considered complete.  The ability to build successful prototypes of the protocols using the specifications is a critical test.  An even more important test will be the introduction of substantial third-party products that rely on the protocols and are able to fully demonstrate achievement of interoperability.

Everything we have before us is evidence of laborious progress toward that achievement.  The glass is more than half full.

3.2 The Eternal Spotlessness of Lazy Minds

It is amazing to see the ways that the latest interim report is used as a launching pad for un-researched criticisms and complaints that have no foundation in reality.   The progression of exaggeration and taking leave from the facts is quite impressive.  I recount this particular drama to show how much even the simplest fact checking is apparently considered irrelevant to reporting on this effort and its status. 

It is indeed fortunate that the source documents produced by the U.S. Department of Justice are online for all to see and verify.  They are actually quite readable by non-lawyers, even the ones at Groklaw.

3.2.1 More-or-Less Balanced.  There are relatively straightforward accounts of the June 17 interim status, with some degree of spin in the headlines, selection of emphasis, and presumed causality.  Ars Technica led off with Jacqui Cheng's DoJ: Room for Improvement on Microsoft Antitrust Compliance, an advance over Jeremy Reimer's earlier US DOJ: Microsoft Dragging Feed on Documentation.   InternetNews.com carried the Stuart J. Johnson report, DOJ Says Microsoft Cooperating but Slow, including an unsubstantiated (and already repudiated) claim of investigations in China.  Some of these accounts seem to have the dates of various events (such as the final judgment) seriously muddled.  There is also some confusion over the process of the European Commission versus those in the United States.  [I have focused on the US-only aspects just for clarity of the timeline and the progressive expansion of the documentation activity.]  The most balanced early account is the straightforward reporting in Dawn Kawamoto's CNet News.com blog entry, Microsoft, DOJ Issue Status Report on Interoperability Compliance.
   [update 2008-06-30] With the announcement of updated documents today, delivered as promised, there is improving balance in Ars Technica Jacqui Cheng's We love interoperability: 50K-page Microsoft document dump (even though 50k is the running total, not the total delivered as new today).  I saw 2 Twitter tweets on this one.  Marshall Kirkpatrick reported Microsoft Releases Interop Docs: Is This What Data Portability Looks Like? on ReadWriteWeb.  This is a speculative piece using the announcement to leap into a discussion of Bill Gates' absence and how this might be to Microsoft's advantage., earning 9 tweets.  Elizabeth Montalbano's piece in InfoWorld, OOXML projects bolster Microsoft's interoperability efforts (which has only tangential relevance to OOXML) provides accurate coverage with some context along with speculation about what provoked the Interoperability Principles in February 2008 (good for one tweet).  Ina Fried of CNet News.com has a simple and direct Microsoft tries to live up to interop pledge (earning 7 tweets).  What's striking about this event is the greater balance and the number of Twitter tweets from various sources that announced the different articles.

3.2.2 The great Leap Out of Compliance.  When the Groklaw reporting on the June 17 interim status is cited as more comprehensive than the status report itself, it is time to wonder about veracity.  You'll notice that the Groklaw analysis, based on the same progression analyzed in section 1.4, here, completely overlooks that the 19 system documents reported in June are new work in addition to the 41,000 pages of technical documentation already delivered.  The system documents replace a smaller selection of overview documents that were produced for the first time in September, 2007. 
   This oversight is apparently seized on by Matt Assay in his June 21 CNet News.com article, Microsoft still not in compliance with DOJ interoperability order.  This is where the glass is taken from half empty to having a hole in the bottom.  Both Assay and Groklaw have the entire timeline of Microsoft's compliance efforts screwed up.  It is apparently a well-guarded secret that there is substantial documentation and related activity and you can find it online. 
   By June 23, the message is headlined as Microsoft Deleted Windows Interoperability Documents, Feds Say over Paul McDougall's InformationWeek article.  In the body of the article, the "so-called Technical Committee" is quoted accurately and then interpreted incorrectly.  I love it that Microsoft is said to have been "blasted" over the change in its publication release schedule when the biggest worry of all is the production of the system documents, not even mentioned in this breathless account.  Even so, most of the observations here are accurate.  The headline is awful, and suggesting that a pact has been broken is also a little strong.  The headline was all that the Microsoft Subnet blog required in its NetworkWorld version, Microsoft Removed Interoperability Documents, Feds Accuse.  This article, fixated on the incorrect statement in InformationWeek, reports on only the concern for removed protocol elements (not documents) and goes off on evidence for "more guile on the part of Microsoft."

3.2.3 Here Comes the Judge.  On Tuesday, June 24, the Federal Judge overseeing the Microsoft final judgment and consent decree held a meeting to review the June 17 interim status.  Ina Fried provides a balanced report in her June 24 CNet News.com article, Judge presses for more from Microsoft.  With all of the background here, it is straightforward to tie the reported remarks to the interim status and the history leading up to the system ("overview") documents that are under development.  It is not clear, here, whether on-platform integration is also meant (requiring more documentation than that already covered by published APIs), or whether attention remains strictly on the communication protocols that work between services.  This could be a confusion on the part of Judge Colleen Kollar-Kotelly or just in the coverage of the meeting.  I expect it will be resolved over time as the system-document work proceeds.

3.2.4 The Empty Glass.  Having not caught on to the fact that Microsoft's 19 system documents are new work that was found necessary beyond the current 41,000 pages of protocol technical documents, Matt Assay continues the great leap with his June 25 CNet News.com article, Microsoft promises to deliver interoperability documents by March 2009.  The Assay perspective is that no documents have been released and Microsoft is withholding material that it must have available internally.  The offered example seems to confuse API documentation (which is all fully available already) and the system documents that apply to the licensed communication protocols.  It appears that Assay has not noticed that the June 2009 documents, with drafts all completed in March 2009,  are the recently-defined additional ones. 

 
Comments:
 
Dennis,

Outstanding post. I had no knowledge about this, other than the very barebones facts.

This is an excellent summary indeed.

It's the same old, same old we saw in the Open XML bunfight.

It looks like Microsoft are making as much of an effort as anyone might be expected to do under the circumstances.

It must get to something when you essentially say "what do you want, blood? Have the source code, we're past caring"

I can see the next scoop from these ill-informed characters

"Nobel peace prize winner Mandela revealed as convicted terrorist - raised funds for paramilitary operations"

The National Enquirer beckons...

Gareth
 
Post a Comment
 
Construction Zone (Hard Hat Area) You are navigating Orcmid's Lair.

template created 2002-10-28-07:25 -0800 (pst) by orcmid
$$Author: Orcmid $
$$Date: 07-12-26 16:37 $
$$Revision: 27 $