Several excellent comments as well as responses to Microsoft General Manager Alan Yates' letter in opposition [PDF] have surfaced:

An Open Letter to Alan Yates of Microsoft from the KOffice team:
KOffice is in fact not related to StarOffice or OpenOffice. It is a completely separate product, and a very fine one at that. One of our team members, David Faure, was an active party in the creation of the OASIS OpenDocument standard, and KOffice was the first office suite that publicly announced support for it. . . . In case you think that even two competing products will not be enough to satisfy the "well-accepted public procurement norms", I can assure you that they will soon not be alone. The fine word processor AbiWord and the spreadsheet program Gnumeric, will also soon support Open Document due to an independent effort by a Nokia research lab.

Sam Hiser, whose letter regarding the ETRM draft document is also highlighted on the Massachusetts site:
Yates highlights the "technical challenges" of changing the State's standard document file format. As a migration expert, I can state from personal and client experience that it is not actually that hard. It is certainly no more difficult than managing an organization of Massachusetts' size (approximately 60,000 desktops) with as many as four different versions of Microsoft's Windows operating system and possibly four different versions of Microsoft's Office suite -- each of which has a different file format that is not backward-compatible. Additionally, any possible difficulties changing away from the Microsoft file formats make the case itself for the immediate change, for this is certainly a 'pay-me-now...' situation.

Nicholas Carr:
Although I'm a resident of the Commonwealth of Massachusetts, I have to admit that I haven't been paying much attention to this situation up until now. I assumed it was just another case of anti-Microsoftism, an assumption backed up by the press's "Massachusetts Dumps Office" headlines. I was wrong. This isn't about Microsoft. It's about a state government launching a serious and comprehensive initiative to replace its fragmented, inefficient set of traditional information systems with a modern, coherent, and flexible IT architecture that allows data to be shared and reused easily. The adoption of OpenDocument as a standard is just one element of the state's ambitious plan. As described in a comprehensive technical paper, called the Enterprise Technical Reference Model (ETRM), the state aims to make a transition "from siloed, application-centric and agency-centric information technology investments to an enterprise approach where applications are designed to be flexible, to take advantage of shared and reusable components, to facilitate the sharing and reuse of data where appropriate and to make the best use of the technology infrastructure that is available."

Steven Walli:
Another categorization attempts to discuss the difference between de jure standards developed in a consensus-based process and de facto standards. A more accurate statement might be de facto technology that describes a market dominant product, rather than a specification for interoperability open to all implementers. . . . Proprietary specifications of de facto technologies are NOT standards. They don't exist to encourage multiple implementations of the technology for the consumer, but rather to encourage multiple complements to the technology, therefore increasing the value proposition of the de facto technology for the vendor. They serve a very different economic purpose. All of Microsoft's claims about being "the standard" are the antithesis of the standardization efforts the Commonwealth want to support.

Simon Phipps:
All these (and more, watch for it now I've mentioned it) want you to approach the discussion from the perspective this is Microsoft vs OpenOffice.org, Microsoft vs Sun, Microsoft vs Free Software - in other words, they want to frame the conversation as company competitive when it's nothing of the sort. Massachusetts are not mandating OpenOffice.org or any other specific product. If Microsoft would stop and listen for a split second, they would see they could trivially comply with what Bob Sutor explains is their customer's reasonable and necessary requirement for an open document format and add support to Office like they have for countless other file formats. The argument is not company competitive, it's about end-user freedom. Claiming OpenDocument is open-source-only is an act of framing.

Microsoft's Brian Jones continues to chirpily blog (see also here) about how wonderful Microsoft's XML is, and the comments on that page are well worth taking the time to read, because some very thoughtful comments were left. I enjoyed this one:

MS can stop granting the license when they want. At that point, anyone who already has a copy of the software I wrote that infringes the patents in question can continue to use it. Perhaps new versions could be distributed to those same people (since they already have a license). I cannot, however, continue to distribute my open source project, because only MS has the right to grant my potential user a license. That is, MS can effectively kill (or at least place in stasis) any project that gets big enough to pose a threat. This is at the core of every OSS license - the right to grant the same rights I have to the recipients of my software.

I didn't see a chirpy answer to that. I found this pertinent question there as well:

"I think a good test of the license is this: if Microsoft had to license someone else's file format as a mandatory format in Office, and that format was only available under the same terms Microsoft is offering the Office schemas under, would Microsoft's legal department accept the terms as-is or not?"

To read, or not to read, that is the question,
Whether tis nobler of the IT departmente to suffer
The lost documents and man-houres of outragious proprietary formats,
Or to take Armes against a monopoleye of FUDsteres,
And by opposing, restrain them, to open to compete

No more, and by a standarde, to say we end
The conversiones, and the thousand read/write errors
That archiving is heire to; tis a consumation
Devoutly to be wisht to open to compete,

To compete, perchance to innovate, Aye there's the rub,
For in that competition of open standardes what innovation may come
When we have upgraded off this end-of-lyfed formate
Must give us pause, there's the respect
That makes calamitie of so long archival requirementes:

For who would beare the failed hard-disques and office suites of time,
The monopolistes wrong, the proude CEO's contumely,
The pangs of despiz'd greed, the lawes delay,
The insolence of patente office, and the spurnes
That patient merrit of the unencumbered takes,

When he himselfe might his open filtre make
With a bare XML specification; who would programmeres beare,
To toil and type under a wearie employmente,
But that the dread of something after release dayte,
The undiscover'd patent, from whose infringemente suit
No developer returnes, puzzles the will,

And makes us rather beare those ills we have,
Then flie to others that we know not of.
Thus patent lawe doth make cowards,
And thus the native hue of innovatione
Is sickled o'er with the pale cast of litigatione,

And enterprises of great pitch and moment,
With this regard their fortunes turne awry,
And loose the name of freedome...

Hark you now,
The faire OpenOffice, openness in thy 'orizons
Be all my documents remembred...

There was also a useful and thorough comment from Gary Edwards, who responded, as a ZDNET Talkback, to a comment that said this:

The only reason why Massachusetts is implementing this is because they are sore losers and they could not get Microsoft broken up. I believe the pretty much are acting against their best interests in this matter. I think that the person that is pushing this should be fired ...... bigjim01

I asked Gary if I could republish his comment on Groklaw to make it part of our history of this event, one, because it's a very good overview of the subject matter, and two because it's a good example of how to respond to such comments. Here's the Edwards response:

******************************************

*You're Kidding, Right?*

I don't see how Massachusetts could have made any other decision except OASIS OpenDocument. Let me break it down for you as simply as possible.

MASS has a sprawl of disparate, disconnected legacy information systems that have accumulated over the past 30 years or more. Many of these systems continue to perform their departmental and business process specific functions quite well, even though they are outdated and sometimes even abandoned by vendors or otherwise orphaned by the ravages of the technology marketplace. But they still work, and they're still on line.

Rather than replace these disparate systems (at great cost), the Commonwealth is going to connect them using Open Internet technologies and methods. The blueprint or collection of best practices for doing this is called SOA, or Services Oriented Architecture. With an SOA approach to solve the dilemma of their information systems sprawl, the state will save a bundle of money, yet also make the move into the next generation of collaborative computing.

The promise of SOA is that disparate systems, platforms, and applications can can connect, and once the connections are written, effortlessly exchange information. Writing these connections is a one time affair if done properly. The trick is to focus on ?loosely coupled? connections instead of hard wiring. This is where the magic of Open XML Technologies becomes important. In particular, at the messaging layer of an SOA XML methods such as Web Services, HttpXMLrequest, and Jabber become important.

Open Internet technologies began with a "connectivity and exchange" layer based on such things as TCP/IP, HTTP, and HTML. But to truly become a universal platform of connectivity and collaborative computing, the Open Internet had to wait for Open XML technologies. There is simply no way to transform the Open Internet into a computational platform without also standardizing how disparate systems format information and package it for digital exchange.

With XML the problems of deploying portable logic - accessible logic and portable content - data - media across the global infogrid are largely solved. At least the foundation for collaborative computing can now be said to be in place. The Open Internet is not just "everything", it's also a way of solving some of the most stubborn problems of integrating information systems into a manageable framework of massive connectivity and exchange, without having to rip out and replace any legacy systems. Originally devised as a "network of networks", the Open Internet is well on it's way to becoming the network of everything.

If you want to say that Peter Quinn and Commonwealth are making a mistake in their decision to adopt an SOA model for solving the most intractable systems integration problems known, problems that no amount of client/server dollars could ever solve, then go ahead and make that SOA - ESB argument. But keep in mind that even the mighty Microsoft knows full well that XML is the future of both the Open Internet and the Bill Gates Internet. At an average of 300 patent filings per day, the good Chairman takes XML very seriously.

And so does the Commonwealth of Massachusetts. No XML, no SOA, no way to effectively integrate disparate information systems, no way to move forward into the next generation of collaborative computing.

So here we have it. It's not about MS Office, Windows, or Linux. It's not about proprietary or open source. It's all about what makes an SOA model work; the Open Internet , Open Standards, and Open XML technologies.

Open Standards are important in that you can't have a loosely coupled approach to systems integration unless you have Open Interfaces, Open Messaging and Communications protocols, Open run time engines and libraries (portable logic), and Open XML technologies, including file formats and shared business process schemas.

That the Open Internet is "owned by none, used by all", is also the enabling factor behind Open Standards. Any attempt to assert ownership over core Open Internet protocols and methods through patents, permission based licenses, royalty schemes, or other encumbrances, is both a threat to the Open Internet and, a blow to global participation. Both of which represent exactly those elements that would be SOA killers.

The "participation" demands that every government must answer to are extreme. There are state level workers, department level information systems, local, federal, and intrastate level information systems, trading partners, supplicants, and on and on. Hey, there are even tax payers to consider. At some level or another, everyone having business with the state needs clean, secure digital access and transparent participation.

So where SOA is a solution to systems integration, strict adherence to Open Standards and Open XML technologies are also a means of lowering the "participation" threshold. The Open Internet is a double edged sword.

If you're not going to argue SOA, then were down to some very simple Open XML decisions. Let me give you a hint on how to make SOA work, "first, get everything into XML, and then get it back again". The ability to connect and transform into a common XML layer is critical to an SOA. It is this XML transformation layer that enables all the different content, data, media, and transaction processing silos to interoperate.

Once you get the XML transformation layer connected to the silos and other disparate information systems, you can start building on the other side of the layer your next generation collaborative computing solutions - many of which will no doubt be XML enabled rich client based efforts.

An important decision in all this is the Open XML file format you choose to standardize on. Remember, SOA is an Open Internet approach, and the Open Internet file formats, while many, are basically limited to HTML and XHTML frames when the issue is that of "compound documents?". XHTML was written as the successor to HTML. OASIS OpenDocument XML was written as the successor to XHTML. ODF today is where XHTML will be in two to three years, but what's more important is that the two XML file format proposals come from very different directions, destined to meet at some point. That "some point" is exactly where browsers, productivity applications, and application service providers can read and interoperate via OpenDoc.

Think of it this way; XHTML comes at the interactive-portable-compound document issue from the browser. OpenDocument comes at the same problem from a two pronged perspective; that of the desktop productivity environment, and that of the enterprise content management - legacy systems integration point of view.

Considering how important the desktop productivity environment is in the grand scheme of information, there are only two XML file format candidates the Commonwealth can choose from; OpenDoc XML and MSXML.

To perfect an SOA, you need to have your information in an XML format that enables clean, high fidelity transformations. It has to also be application and platform independent; otherwise it's going to clog up the common transformation layer that universally connects the disparate information systems - which is the whole point of an SOA! In an SOA, all file formats are welcome as long as there is a way of perfecting a transformation in to and out of the common XML layer. But the layer itself has to be open and perfect XML. Otherwise arbitrating the traffic becomes a nightmare. You do SOA right and it's a dream come true.

The XML file format also has to be an Open Standard with an enduring guarantee that it will always be open. Having a clean transformation layer is one thing, but it's a serious matter if your transformation and connectivity process is hostage to the whims of a single vendor. That kind of a mistake places every other systems vendor or solution provider in a permission based or royalty based dependency with that single vendor. You do that and you can kiss your SOA goodbye. So long to any hope of ever having an orchestrated symphony of information. It's back to a single platform integrated stack approach where nothing works together unless the single vendor with control of the desktop productivity environment says it can interoperate.

So here's the problem; MSXML (and the governing MS XML Reference License) is crippled XML. It's not interoperable. And it breaks the most basic promise of XML, the promise of easy high fidelity transformations.

I am well aware that Microsoft shills are out in force, trashing OpenDocument in every way imaginable. Even in ways that result in their also trashing MSXML in the process. Most of the stuff i see is nonsense. Besides, if Microsoft had any problems with OpenDocument they could have said so at any time during the past there years of their membership on the OASIS OpenDocument Technical Committee. As the OpenOffice.org representative on the OpenDoc TC, i've only missed one meeting in the near three year effort. Never once has Rahib complained or objected. So their complaints today about OpenDoc are not based on XML expertise, interoperability, legacy compatibility, or the implementation of all those Open XML technologies coming out of the W3C. No, their complaints are entirely conceived to protect their monopoly control over the desktop productivity environment, and how that environment transitions into the next generation of Open Internet (or not) collaborative computing.

Note that there is a big difference between what MS means by "legacy compatibility" and what the OpenDocument TC had to address. Microsoft is only talking about MS Office legacy information. The first 18 months of OpenDoc TC work was focused on thirty year legacy needs of all information platforms and systems. By comparison, the MS Office stuff is a drop in the bucket compared to the enterprise publication and content management concerns of OpenDoc TC participants like Boeing, ArborText, Stellent, Documentum, Corel, The Society of Biblical Literature, the Australian National Archives, IBM - Lotus Notes, Adobe and SpeedLegal. Having said that though, let me also say that the OpenDocument TC had some of the world's foremost experts in reverse engineering that legacy of MS Office documents.

Anyway, if Microsoft had any complaints about the volumes of effort that went into perfecting OpenDoc fidelity with the MS Office legacy documents, they had every opportunity to contribute.

From an interoperability - transformation perspective, MSXML fails miserably. Every MSXML file has a binary key in the header. The binary key holds the critical layout definition for styles, or what would otherwise be called the "presentation" aspects of the file. Sometimes this is also called "contextual information". With the "content" aspects of a file, MSXML provides an excellent text book XML transformation opportunity. But this low level of transformation is only useful to publication and content management systems that would prefer to provide a higher level systems level formatting style anyway.

If your purpose with XML is to have all your information in a structured file format that enables the aggregation, reuse, and re purposing of massive stores of information, the binary in MSXML is a deal breaker. It's an SOA killer.

Now don't get me wrong, XML is intended to be eXtensible. So it is kosher to put a binary key in the header. What happens though is that you break transparent interoperability wherever these eXtensions occur. In some cases, the transformation process only needs to be tweaked to perfect the one to one relationship between an XML file and an XML ready information system. If the eXtensions are openly declared, this isn't a big deal. XSLT, XSL-FO and XPath are incredibly effective tools if you have access to "all" the XML information.

If on the other hand, the eXtensions are not transparently declared, or worse, critically important aspects like the layout definition are locked in a binary key - and comes with a governing license that threatens and looms over any reverse engineering attempts, then this simply isn't XML. It's a scam using "XML" to control information and information processes far into the future.

In the case of MSXML, interoperability and transformation is broken right from the git go. Because of the binary key in MSXML, you can not transform a MSXML file to OpenDocument. And those transformation "filters" Microsoft promised the European Union in November of 2004? Forget it. They don't exist.

Since OpenDoc is Open XML, and an Open Standard, it would be easy for Microsoft to perfect a transformation from OpenDoc to MSXML. But until and unless the reverse engineer experts break that binary key, it is not possible to perfect in any useful way a transformation from MSXML to OpenDocument.

Which leaves us starring at another fine example of a Kerberos type corruption of an Open Standard. Information can go in. But it can't go out. Least ways not without Microsoft's permission. This is one way arbitrarily limited interoperability.

Microsoft argues that the standards requirement threshold should be lowered by Massachusetts to the level of basic XML. They argue that requiring such a high level, well defined, and robust file format specification as OpenDoc is anti competitive. (This coming from a convicted anti trust monopolists). Well of course they want to keep that binary key. With the key in place they can leverage their monopoly on the desktop deep into the realms of Open Internet servers and back end data - transaction processing systems. The stockholders are going to be happy if they pull off this coup. But how this works for the state of Massachusetts, or anyone else for that matter is beyond me.

If Massachusetts were to accept this low level of non transformational, non interoperable XML, they would have to kiss their SOA goodbye, and accept the cost of an all Windows, all the time from now on far into the future. And oh what a cost this "acceptance" entails. I'll get to that in a moment. But one thing that needs to be pointed out here is that when your dealing with systems architectures, if you set the standard of XML to it's lowest level, as Microsoft is proposing, than you also reduce your interoperability to that same low level. (pssst SOA killer!)

One of the things we have tried to do with the OpenDocument XML specification is to reduce the areas of eXtensibility. The higher the threshold for having to create eXtensions beyond the standard, the greater the interoperability and ease of transformation. Think of it this way, wherever there is a need to extend, there is a "fork". Some forks are soft, and can be folded into a highly interoperable system. Other forks like the MSXML binary key are hard, and break interoperability before there's even a chance to attempt a transformation. So one of the things the OpenDoc TC tries to do is to embrace W3C Open XML technologies as soon as they can reach "baked enough" consideration. In that way we can push the eXtensibility threshold higher, broadening our interoperability without closing off the importance of being both eXtensible and scalable.

Two areas of this OpenDocument approach to eXtensibility are of immediate concern; XForms and Metadata.

OpenDoc fully supports XForms, But what's really important is that Adobe Acrobat Interactive Forms are an eXtension of XForms, and, IBM has committed Lotus Notes to some sort of coexistence with XForms. The trick for the OpenDocument TC will be to push the XForms eXtensibility threshold to include and accommodate Adobe and IBM needs at the highest level of convergence possible. Which we will do. The result will enable governments, enterprises, organizations, and collaborative associations of all sorts to work the same information with applications and tools of their choice, rather than being bound by platforms and applications of someone else's choosing.

Microsoft of course does not support XForms. They have their own digital forms application; InfoPath. While i don't know whether or not InfoPath files are bound by a non interoperable binary key, it is clear that there are serious platform and cost requirements.

The other area that is very important is that of Metadata. There is currently an all out effort on the OpenDoc TC to merge the Dublin Core metadata model with RDF. An effort that will no doubt bridge XML:RDF to the entire OpenDoc specification. But where the metadata effort is of immediate significance is that through this work, the Adobe XMP Metadata model will effectively be merged. At the end of the day, we seek to enable Massachusetts to manage, aggregate, and re purpose at the metadata level, with the same set of tools, stores of information in OpenDoc, PDF, and Lotus Notes. And that's just for starters.

All Adobe files, even Photoshop images, implement the XMP XML:RDF metadata model.

Okay. What is the cost of MSXML vs OpenDoc XML?

OpenDocument is a purely text based XML specification; a human and machine readable structured file format for the most advanced kind of intelligent and compound documents required by next generation Open Internet collaborative computing systems. Today OpenDoc is available across all generations of Windows (3.0, 3.11, 95, 98, 98SE, 2000, and XP), Linux, Solaris, and Mac OSX. There are even USB keys fully OpenDoc enabled. Because it's open and text based, there is no reason why DOS, DRDos, Atari, BE OS, OS2, all UNiX, or any device specific applications couldn't work with OpenDocument. It's completely application and platform independent.

As an XML file format, OpenDocument has been in production for five years, meeting the rigorous demands of a real world productivity environments. The cost OpenDocument ready applications range from zero to whatever an organizations wants pay for support, enhanced implementations, and enterprise publication and content management. There are choices at all levels, but at the zero cost level, OpenOffice.org has reached parity with MS Office. And although OpenOffice.org 2.0 and the yet to be released MS Office 12 take near opposite directions regarding collaborative computing, the race is on. And with OOo ecosystem participants like IBM's WorkPlace, Microsoft is going to have their hands full.

MSXML is a comparatively recent release, with so little real world implementation that there isn't enough workflow involving MSXML to prime the reverse engineering pump needed to crack that binary key. Microsoft does provide an XSL-FO transformation file called Word2FO.xsl. But the styles are so crippled that the transformation is useless. This is inexcusable. The promise of XML is that we should have perfect transformations without any kind of reverse engineering!

Unless someone shows otherwise, the platform requirements for collaborative computing with MSXML are Windows XP, MS Office XP Professional 2003, IE 6, and at least a few of the many offerings in the MS XP suite of Servers (Exchange, SharePoint, Collaboration Server, Server 2003 - Active Directory, and MS Office Server).

IMHO these requirements are extraordinary. The taxpayers of Massachusetts should not have to pay this incredible price for moving into the next generation of the Open Internet. The problem is that Microsoft stubbornly insists on tying their collaborative computing efforts to a tightly integrated XP stack where a cascading entanglement of interfaces and dependencies bolt together the operating systems, application layer, development tools, framework and server suites.

Open XML technologies should provide a means for other systems to establish effective interoperability with the XP integrated stack, but there is a world of difference between the promise of XML, and the proprietary and controlled way Microsoft has chosen to implement (cough cough corrupt) Open XML technologies.

I doubt if even 5% of the entire state of Massachusetts is MSXML ready. The upgrade requirements in both hardware and software at both the desktop and server side has to be beyond costly. Microsoft tries to hide this cost behind the magnitude of the monopoly base, posturing that MSXML is available today for the great herd of 450 million Windows users. But nothing could be further from the truth. MSXML is as bolted into the XP 2003 stack as everything else in Microsoft's collaborative computing proposal. I've even seen numbers posturing that OpenOffice.org has a 12% desktop penetration, and MS Office XP Professional is at 8%. But even if the XP Office Professional market penetration was at 33% (which it isn't), the cost of bringing everyone up to the participation threshold is beyond belief.

If MS Office embraces OpenDocument, instead of trying to shove another bolted, interdependent, proprietary platform - application version bound file format down our throats, the whole world wins. The cost of moving forward and truly taking advantage of the Open Internet becomes minimal. Global participation in the collaborative computing age will explode. But as we all know, that's an impossible "if".

The State of Massachusetts has chosen to pursue an SOA going forward and into the next generation of collaborative computing. They did not choose the Win XP integrated stack model. The OpenDocument ecosystem on the other hand is SOA ready, and growing like crazy with IBM, Adobe, Novell, Red Hat, and Sun leading the way. Apache servers command over 70% of the Open Internet. Which XML file format do you think they will be able to integrate and interoperate with? Which XML file format do you think Google and Yahoo will consider?

The bottom line is that OpenDocument is today able to be used by almost the entire state of Massachusetts.

If you use the Open Internet, you can use the OpenDocument. Today. No upgrades. No strategy sessions about whether or not to pay Microsoft forever or get off the Redmond treadmill now. It's an easy decision. Open Internet, Open Standards, Open XML. The world is just a download away from taking that first step into collaborative computing. A step that's a leap far into future.