decoration decoration
Stories

GROKLAW
When you want to know more...
decoration
For layout only
Home
Archives
Site Map
Search
About Groklaw
Awards
Legal Research
Timelines
ApplevSamsung
ApplevSamsung p.2
ArchiveExplorer
Autozone
Bilski
Cases
Cast: Lawyers
Comes v. MS
Contracts/Documents
Courts
DRM
Gordon v MS
GPL
Grokdoc
HTML How To
IPI v RH
IV v. Google
Legal Docs
Lodsys
MS Litigations
MSvB&N
News Picks
Novell v. MS
Novell-MS Deal
ODF/OOXML
OOXML Appeals
OraclevGoogle
Patents
ProjectMonterey
Psystar
Quote Database
Red Hat v SCO
Salus Book
SCEA v Hotz
SCO Appeals
SCO Bankruptcy
SCO Financials
SCO Overview
SCO v IBM
SCO v Novell
SCO:Soup2Nuts
SCOsource
Sean Daly
Software Patents
Switch to Linux
Transcripts
Unix Books
Your contributions keep Groklaw going.
To donate to Groklaw 2.0:

Groklaw Gear

Click here to send an email to the editor of this weblog.


To read comments to this article, go here
Answering Microsoft: Comments on Microsoft's Letter to MA ~ by David A. Wheeler
Sunday, October 30 2005 @ 11:24 AM EST

Comments on Microsoft’s Letter to Massachusetts
by David A. Wheeler, October 29, 2005

As most readers probably know, Massachusetts has chosen the OpenDocument standard as their standard for office suite data exchange. Massachusetts has published the comments about this decision from many sources, including Adobe, Corel, IBM, Microsoft, and Sun Microsystems. Microsoft was extremely unhappy that the OASIS OpenDocument standard was selected instead of their proprietary XML format, aka the Microsoft Office Open XML format, that is under development.

Pamela Jones asked me to comment in more depth on Microsoft’s response, because I’ve had past related experience (e.g., with standards, XML, and even OpenDocument specifically). Many people have made brief comments about this letter, but she thought a more detailed commentary would help those who are less familiar with this topic. So, here’s my attempt at commenting on Microsoft’s letter in more detail.

But first, a few disclaimers and general comments. These are my personal thoughts, not anyone else’s, and I speak for no one. I have tried to give references to justify many of my comments, though, which I believe justify the statements well. I’m independent; I have no financial stake in this outcome. I am not interested in Microsoft-bashing, so for those who were looking for that, sorry. I do believe that governments should try to create as level a playing field as possible, so that suppliers can fairly compete for government business. I also believe that it’s a very good idea to have standards; we could not have a technological society without them!

Below is the Microsoft letter text (or summaries of it), which I’ve indented, italicized and put in bold text and interspersed with my comments.

It’s a long letter, and commenting it makes even longer. Feel free to skip ahead to a few of the discussions you may be particularly interested in (I repeat key points in some places so that skipping ahead will make sense):

With that said, let’s look!

September 8, 2005

(To) Secretary Eric Kriss Executive, Office for Administration ... (and) Mr. Peter Quinn, Chief Information Officer/Director Information Technology Division

Re: Proposed Revisions to Information Domain-Enterprise Technical Reference Model

Dear Secretary Kriss and Director Quinn:

Microsoft respectfully invites you to consider its responses to the proposed revisions to the Enterprise Technical Reference Model-Information Domain published on August 29, 2005 (ETRM) which, as currently framed, mandates exclusive use of a designated office document format within all executive agencies of the Commonwealth of Massachusetts by January, 2007.

This is a good, conventional beginning of an appeal to government officials, which is basically what this letter is. Right up front they identify the subject they wish to discuss, which is always a good idea. At the end you’ll notice that this letter is cc’ed to their boss, Governor Mitt Romney, presumably to try to get the Governor to overturn his employees’ decision.

They don’t give the name of the format being discussed, so I’ll re-state it here: Massachusetts plans to use the OpenDocument format, as standardized by OASIS. Microsoft is unhappy about this, because they wanted Massachusetts to use Microsoft’s new proprietary XML format also or instead.

They use the word “exclusive” here, which is interesting. Literally stated, it’s true; Massachusetts is mandating one standard to the exclusion of other specifications. But that’s good; you want to pick a single standard for a given purpose, to the exclusion of others, as long as all suppliers can implement the standard without unrelated restrictions. If we had two signal light standards, where red meant “stop” in one and “go” in another, we’d obviously have bad results. One of the biggest problems in information technology is that in some areas there are too many standards, instead of a single standard that everyone can agree on and use. Mandating a single standard for a given area is a very good thing, if all suppliers can implement the specification without legal, monetary, or other restrictions or discriminations. Massachusetts has really a strong case for selecting OpenDocument (the topic of this letter) as this standard, as we’ll see.

Be careful: the word “exclusive” is here, but no product is being excluded. As we’ll discuss, anyone can implement this standard without restrictions, including Microsoft. In fact, OpenDocument has already been implemented by several suppliers, both proprietary software and open source software, using all of their normal licensing and business models, so it’s demonstrably open for competition. (Note: in this document I’ll sometimes use the term “open source software”, and sometimes use the term “open source software / Free software (OSS/FS)”; in all cases I mean by these terms software whose license conforms to both the open source definition and the Free software definition).

By the way, some news articles and other various groups explained the issue very poorly, suggesting that this decision was fundamentally about “using open source software instead of Microsoft Office” or “throwing away Microsoft Office.” In this introduction, Microsoft correctly identifies the real issue: data formats. There’s nothing in this Massachusetts decision that mandates or even prefers open source software. This decision does not even mandate that Massachusetts stop using Microsoft Office, necessarily, though that decision is partly Microsoft’s. If Microsoft implements Massachusetts’ key requirement (OpenDocument), then great; if not, that is a future possibility. Data formats may seem like an esoteric thing to debate about, but whoever controls the data format controls all the data in that format. Once this letter is understood as a battle over power and sovereignty over all documents (and the information in them) ever created, the strong positions of these organizations all makes sense.

Microsoft strongly supports the efforts of the Information Technology Division (ITD) of the Executive Office for Administration & Finance (ANF) to bring the benefits of XML to executive agencies of the Commonwealth of Massachusetts. We recognize that governments are challenged to be fully accountable for archived public records well into the future, and for ensuring that government agencies can efficiently handle data and documents across all technical and organizational boundaries. We share the opinion that XML is the ideal format for data interoperability and storage, management, and archiving of public records and endorse the direction to support open and agreed-upon specifications for data interoperability within government via XML standards. We share the proposal’s goals for data interoperability across government agencies and for assuring proper storage and maintenance of all public records. Consistent with this viewpoint, Microsoft has been deeply committed to supporting XML within Microsoft Office for a number of years and continues to work with many governments around the world toward these goals.

It’s common in a disagreement to start with a list of things you agree on. I suppose this can look like flattery, but it does have a value -- if people can find many shared positions, then perhaps there’s a way to find a solution that everyone can be happy with.

Why XML?

We should particularly notice Microsoft’s agreement that XML is where everyone is going for storing editable documents. This is a point of strong agreement between Microsoft, Massachusetts, and most of the rest of the information technology community (including office suite suppliers, other organizations who manage or manipulate data in documents, and customers that use or generate documents). XML is where everyone is going, not because it just happens to be there, but because XML can help solve some serious government problems like data interoperability and long-term storage.

That’s why staying with Microsoft’s old binary formats isn’t a long-term solution; they don’t support interoperability well, and they’re horrific for long-term storage. I have Office documents less than a decade old that are no longer readable by current versions of Microsoft Office, and governments think in terms of centuries. Microsoft themselves are abandoning their old binary formats as their primary storage format (Office 12 will no longer use them by default); they plan to use a new proprietary XML format that they have developed in isolation instead. Other office suite suppliers are choosing to support OpenDocument, which is also XML-based.

Compressed XML documents also tend to be much smaller than binary files, which is great for archiving. Loading and saving XML files tends to take longer than with binary formats, but the advantages of being able to actually use the documents later more than compensates for that (in most people’s opinion).

But XML cannot provide these advantages by itself; specific XML-based schemas have to be defined to store the data in a way that maximizes data interoperability and long-term storage. We are at a technology inflection point; everyone agrees that XML is where we’re headed, and thus it makes sense that Massachusetts has begun planning for that transition.

Microsoft agrees here that there’s a need for “open and agreed-upon specifications for data interoperability.” This is striking, because these are some of the very requirements that Microsoft’s proposed alternative fails to meet. These issues show up throughout the letter, so let’s deal with them them at length now; otherwise, what the letter says (and doesn’t say) will be hard to understand. Let’s look at the issues in two parts: “agreed-upon” and “open”.

Agreed-upon

Let’s start with the phrase “agreed-upon.” Microsoft has developed its own proprietary XML-based formats, first the XML format for Office 2003, and a later significantly modified version that is to be used by Office 12 when it is released next year. No one other than Microsoft has had any reasonable way to “agree” on Microsoft’s specification. In particular, competing suppliers could not review Microsoft’s format in some neutral forum where they could identify problems, work together to fix them, and agree without pressure on a final result. Users do not really count here; most users have no idea what is in their format and cannot rationally agree to anything. So, there’s never been an opportunity for agreement on Microsoft’s format. In contrast, Massachusetts’ choice, OpenDocument, has been agreed upon between many office suite suppliers and large-scale users, and vetted (agreed upon) by an independent standards body (OASIS).

This is a central issue. Wise customers (especially large ones like governments) don’t want a “standard” controlled by any one supplier, no matter who it is, because that tends to lock them into a single supplier. Once a supplier has a monopoly, their prices tend to go up and their service tends to go down... no matter who it is. That is not because they’re “evil.” In the U.S., commercial suppliers have a duty to maximize their profit; raising prices and lowering service (overhead) helps do both. To counter this, wise customers try to use standards developed and implemented by a number of suppliers, vetted and maintained by an independent body.

The European Commission's Valoris report explained why vetting by a neutral party is especially critical for XML-based document standards like the ones under discussion:

“It is quite trivial to add elements to an XML document that place processing requirements and restrictions on the document, thus preventing cross-platform processing capability... While properly developed XML should in theory be platform-neutral, experience has shown that suppliers who wish to maintain and protect their platform’s market will go to extents to encode elements that are capable of being processed only by their own application suites. The only counter-balance to this natural force is the development of open, cross-industry, widely adopted standards that serve to block the inclusion of application or platform specific encoding.”

Customers generally want the strong, supplier-neutral results that come from the usual give, take, and consensus-building in standards development, both for the original work and to control future direction. Usually an existing specification that works is presented to a large body of independent reviewers and competing suppliers, who work together to discuss various improvements or subsets to make the result appropriate for long-term standardization. Massachusetts has selected other specifications that are good examples of this. Adobe’s PDF has gone through ISO, it has multiple independent implementations, and it has a specification whose future is controlled by a neutral party. HTML is standardized by the W3C, and again, has multiple independent implementations and a future controlled by a neutral party. In contrast, no independent standards body has ever reviewed Microsoft’s specification, it’s not clear that there will even be multiple independent implementations (and if there are they will be unnecessarily constrained), and there is no neutral party that can control the direction of later versions of the specification.

Some people have taken a brief independent look at Microsoft’s specification, and reports from them suggest that there will be serious problems doing so. Gary Edwards’ article “*You’re Kidding, Right?*” (posted by Groklaw on Sep. 25, 2005) reports on Microsoft’s XML format (calling it MSXML), and he found one issue that may be an example of what the Valoris report warned about:

“From an interoperability - transformation perspective, MSXML fails miserably. Every MSXML file has a binary key in the header. The binary key holds the critical layout definition for styles, or what would otherwise be called the ‘presentation’ aspects of the file...

If your purpose with XML is to have all your information in a structured file format that enables the aggregation, reuse, and re-purposing of massive stores of information, the binary in MSXML is a deal breaker...

[It’s true that] XML is intended to be eXtensible.... [but if] the eXtensions are not transparently declared, or worse, critically important aspects like the layout definition are locked in a binary key - and comes with a governing license that threatens and looms over any reverse engineering attempts, then this simply isn’t XML.

.. . [I]nteroperability and transformation is broken right from the git go. Because of the binary key in MSXML, [someone other than Microsoft cannot] transform a MSXML file to OpenDocument. And those transformation ‘filters’ Microsoft promised the European Union in November of 2004? Forget it. They don’t exist.

Since OpenDoc is Open XML, and an Open Standard, it would be easy for Microsoft to perfect a transformation from OpenDoc to MSXML. But until and unless the reverse engineer experts break that binary key, it is not possible to perfect in any useful way a transformation from MSXML to OpenDocument...

Information can go in. But it can’t go out. Least ways not without Microsoft’s permission. This is one-way, arbitrarily limited interoperability...

With the key in place they can leverage their monopoly on the desktop deep into the realms of Open Internet servers and back end data-transaction processing systems. The stockholders are going to be happy if they pull off this coup. But how this works for the state of Massachusetts, or anyone else for that matter is beyond me.

If Massachusetts were to accept this low level of non-transformational, non-interoperable XML, they would have to kiss their SOA goodbye, and accept the cost of an all Windows, all the time from now on far into the future.”

It is not clear to me exactly what version he reviewed; perhaps this has been fixed, or will be fixed. But that’s not the point. The point is that there are lots of problems that can be embedded in a specification when there’s no independent review; this is just one example. The need for independent review is very, very real.

Back in 2004, the European Union’s Telematics between Administrations Committee (TAC) asked “Industry actors not currently involved with the OASIS Open Document Format consider participating in the standardization process”; Microsoft chose not to. TAC also specifically stated that “Microsoft should consider the merits of submitting [its] XML formats to an international standards body of their choice.” Microsoft chose not to, again. Microsoft has every right to choose to do what it wants, but if a supplier won’t do something that is important to their customers, especially after being clearly told to do so, customers typically start looking at alternative suppliers.

Open

When you think “open”, think “open competition” or “open market.” An open standard should allow competition based on merit, instead of limiting customers’ suppliers to one particular supplier or a subset of suppliers based on their business model, development model, licensing model, and so on. You should be able to replace one product that does the same function for another, as long as they meet the same open standard, and achieve at least the same basic function provided by the standard (though some may perform better or have additional features -- think of plug-replaceable components).

The Valoris report (noted above) defines an open standard this way:

“The minimum requirements for an open standard are that the document format is completely described in publicly accessible documents, that this description may be distributed freely and that the document format may be implemented in programs without restrictions, royalty-free, and with no legal bindings.”

Indeed, if a specification or its license is designed to make it very difficult or essentially impossible for plausible competitors to implement it, then it isn’t really an open standard. The Valoris report notes, for example, that widespread public review of a specification is needed to ensure that a specification isn’t unnecessarily difficult for another party to implement it.

Governments have a special duty to use open formats. Europe’s Telematics between Administrations Committee (TAC) said:

“Because of its specific role in society, the public sector must avoid that a specific product is forced on anyone interacting with it electronically. Conversely, any document format that does not discriminate against market actors and that can be implemented across platforms should be encouraged. Likewise, the public sector should avoid any format that does not safeguard equal opportunities to market actors to implement format-processing applications, especially where this might impose product selection on the side of citizens or businesses. In this respect standardization initiatives will ensure not only a fair and competitive market but will also help safeguard the interoperability of implementing solutions whilst preserving competition and innovation.”

ZDnet concluded that, “[when] open standards exist which are capable of supporting the work the state does, this should be an unexceptional decision; accessibility for as broad a range of citizens and [organizations] as possible is a primary responsibility for any government.”

To be an open standard, anyone should be allowed to implement it. And “anyone” means “anyone.” A format that locks out a large part of the supplier market is, obviously, not open. In particular, now that open source software has become a major market force, it is clear that open source software (using their most common licenses) must be allowed to implement the specification, or the specification is not an open standard. Quoting obsolete guidelines doesn’t duck the issue; anyone is anyone.

Microsoft’s letter has many words (especially on page 9 and on) on many things they’ve done. And I agree that a number of things that they list in their letter are indeed very good things, and some are unusual steps for Microsoft. Microsoft acknowledges they’ve applied for patents, but they proclaim in particular that they grant use royalty-free, in perpetuity, and that anyone can read the specification and its license.

Why would anyone have a concern? Because that wasn’t the question.

Microsoft’s conditions on the use of their XML format are fundamentally incompatible with many of their competitor’s licenses or business models, according to a large number of people who have examined the conditions imposed by Microsoft. They certainly fail to meet the definition of “open” as defined above in the Valoris report (which required no restrictions and no legal bindings). There’s no practical way many of Microsoft’s competitors could use Microsoft’s format given the license Microsoft is imposing with it (in countries like the U.S. that tolerate software patents).

Basically, if you choose Microsoft’s XML format, you have decided against open competition, in perpetuity. A naïve reader might not realize this, but Massachusetts has been researching this issue for a long time and has learned that the details matter.

For about two years, Massachusetts has been working to identify its requirements, discussing its options with others, and asking people what the impact would be of various alternatives. They found out that the impact of accepting these conditions would be severe: they would probably lock themselves into a single supplier or small set of suppliers in perpetuity, and they would certainly forbid real competition. Massachusetts does not want to be locked into one supplier, or a small set, in perpetuity; they want anyone to be able to develop software for them to work with their data. And they can’t keep this fundamental right if they use Microsoft’s XML format.

This isn’t just my opinion; it’s the considered analysis of people who have reason to know. Groklaw has posted a lengthy analysis by Marbux, a retired lawyer (updated April 1, 2005). His detailed analysis found that Microsoft’s specification excluded competition, in contrast with Microsoft’s public claims. “Competitors are... effectively precluded from bidding against Microsoft or its suppliers for any... contract specifying use of Microsoft’s software file formats.” He first noted that the patent license for the format “is structured to be read restrictively, in Microsoft’s favor... it states that: ‘All rights not expressly granted in this license are reserved by Microsoft. No additional rights are granted by implication or estoppel or otherwise.’ This is not the customary ‘all rights reserved’ phrase more commonly encountered... If you cannot find words in the license explicitly stating that you have the right to do something, you don’t get that right.”

Then, by examining the patent license in detail, he found a number of omissions and conditions that suppress competition: there is no integration clause, no license for the schemas themselves, no grant of copyright was included in the patent license, no commitment to delivering any future changes to the schemas or right to develop software implementing them under the same or more liberal license (this particular issue may have been resolved later by Microsoft), no identification of the Microsoft patents involved, no identification of third-party patents, no right to sell or sublicense implementing software, a prohibition against sale and licensing of implementing software, a prohibition against software having functions other than to read and write files using the specification without modification, no license to convert files to and from other formats, no right to write files using the schemas, vagueness and ambiguities will deter implementation by developers and adoption by end users, and a discriminatory incompatibility with F/OSS licensing, and discriminatory incompatibility with proprietary software competitors. In short, he believes Microsoft’s license prohibits effective competition from using the format.

This analysis is supported by others. Richard Stallman, the author of the GNU General Public License (GPL), states that Microsoft’s license was “designed to prohibit all free software. It covers only code that implements, precisely, the Microsoft formats, which means that a program under this license does not permit modification... The freedom to modify the software for private use and the freedom to publish modified versions are two of the essential components in the definition of free software. If these freedoms are lacking, the program is not free software [as defined by the Free Software Definition].”

Indeed, let’s look at a particularly important example -- software that is written and released using the GNU General Public License (GPL). It turns out the GPL is a really good “acid test” for determining if full and open competition is possible. This isn’t an academic exercise; there are already open source software office suite programs that are licensed under the GPL, such as KOffice and Gnumeric. That’s not surprising, because the GNU GPL license is the most popular license, by far, for open source software.

For example, SourceForge.net reported on November 10, 2003, that the GPL accounted for 71% of the 45,736 projects it hosted with open source licenses; the next most popular licenses were the LGPL (a variant of the GPL) at 10% and the BSD licenses at 7%. Billions of dollars worth of effort have been invested in software released under the GPL. These programs cannot realistically change to another license; they have a massive amount of code, they reuse other code that requires the GPL, and their development model depends on the GPL.

Can these competitors use Microsoft’s format? No, and here’s more evidence of this (see Peter Galli’s eWeek article for more):

  1. Dan Ravicher, executive director of the Public Patent Foundation, says that “If [Microsoft has] rights and a license is needed, then the term in the license that requires attribution by the licensee of all of its downstream licenses is, in fact, not compatible with the GPL.”
  2. Even Jean Paoli, senior director of XML architecture for Microsoft, acknowledged that their attribution requirement might preclude any program that uses the file formats from being used in open-source software licensed under the GPL. Paoli admitted, “The GPL may not allow code that is attributable to another company like Microsoft to be included.”

    Now, perhaps Paoli doesn’t know for sure, but that seems unlikely; Microsoft has lots of lawyers, and people have been asking them point-blank if really any competitor is allowed to use their format since at least early 2004. Many believe that Paoli does not wish to admit that Microsoft’s XML license is incompatible with the GPL, because that admission would make it even more obvious that the license prevents real competition.

Such discrimination is unacceptable, runs counter to many customers’ policies, and it’s certainly not in the interest of customers. It’s particularly absurd if such restrictions control access to public documents (which is what government documents are).

If a specification cannot be implemented using the GPL, it discriminates against open source software (because the GPL is the most common such license). If a specification discriminates against open source software implementations, then it is not a specification that allows open competition. This was not as big an issue decades ago, when large-scale open source software systems were uncommon, but it sure is now.

Some examples from the U.S. federal government are probably instructive. Back in 2002, MITRE’s “Use of Free and Open Source Software in the US Dept. of Defense” (DoD) [PDF] found that “banning [open source software / Free software (OSS/FS)] would have immediate, broad, and strongly negative impacts on the ability of many sensitive and security-focused DoD groups to defend against cyberattacks.” The report also found that the GPL so dominates in DoD applications that a ban on just the GPL would have the same strongly negative impacts as banning all OSS/FS. MITRE noted that OSS/FS “plays a far more critical role in the DoD than has been generally recognized.” The U.S. DoD has a formal policy of neutrality between OSS/FS and proprietary software. The U.S. federal government also has a position of neutrality, as discussed in the Office of Management and Budget memorandum M-04-16 of July 1, 2004. Massachusetts is not bound by these federal documents, but they have many of the same issues. Using a format that cannot be implemented by OSS/FS programs, using the most common license for them, is clearly discriminatory against them and is not neutral.

For more perspectives, you can see Ken Krechmer’s “The Meaning of Open Standards,” which notes that open standards can be viewed from three perspectives:

  1. The formal Standards Setting Organizations (SSOs), as organizations representing the standards creators, consider a standard to be open if the creation of the standard follows the tenets of open meeting, consensus and due process.
  2. An implementer of an existing standard would call a standard open when it serves the markets they wish, it is without cost to them, does not preclude further innovation (by them), does not obsolete their prior implementations, and does not favor a competitor.
  3. The user of an implementation of the standard would call a standard open when multiple implementations of the standard from different sources are available, when the implementation functions in all locations needed, when the implementation is supported over the useri’s expected service life and when new implementations desired by the user are backward compatible to previously purchased implementations.
Krechmer references Bruce Perens’ earlier “Open Standards: Principles and Practice”, which identifies these principles: Availability, maximum end-user choice, no royalty, no discrimination, ability to create extension or subset, and ability to prevent predatory practices.

Microsoft’s letter quotes the Valoris report from 2004, which said “the MS license provides access to the schemas and full documentation,” and also quoted some positive European Union statements about their license based on the Valoris report. But the Valoris report, page 51, said this as its primary statement about the legal issues:

“The associated legal terms seem to create a lot of controversy. As we are not qualified to make a judgement on this basis, we will simply highlight the main points hereafter, and recommend examining carefully the legal aspects of the license.”

Catch that? The Valoris report was a market analysis with some technical evaluation, and it was specifically not a legal analysis. It is unreasonable to depend on the Valoris report for a legal analysis; the Valoris report specifically said it was not a legal analysis, the writers frankly state that they were unqualified to do a legal analysis, and they recommended that a legal analysis be performed as follow-on work. We want to know if any competitor can legally use the schema, not if people can read it. Groklaw posted the most detailed public analysis I know of, updated on April 1, 2005, and it found a massive number of problems. At the very least this license discriminates against the most popular license for open source software; not even Microsoft is willing to deny this. And those who’ve examined this license the closest have found a vast number of restrictions in it that prevent or inhibit competition.

Note that this is fundamentally different than the issues surrounding the old binary formats of Microsoft Office (.doc, .xls, and .ppt). It was legal for competitors to implement these binary formats; the problem was that the specifications for these binary formats were never published for competitors to use. Because the specifications for those formats were not published, and because Microsoft kept changing the formats internally, competitors have had to spend many man-years figuring out those formats so that they could read and write them and maintain interoperability. But in fact, they have managed to do so; essentially all office suites can read and write these formats. No office suite, not even any particular version of Microsoft Office, can read and write these binary formats perfectly, but all are free to try.

The difference now is that patents are involved. Microsoft has patented some aspects of its new XML format; if these patents are truly valid, the law makes it illegal to use Microsoft’s XML format except under the terms Microsoft has created. Many are especially concerned about this in the case of Microsoft, because of the secret Microsoft documents now named Halloween I and Halloween II that were exposed to the world a few years ago. These documents were developed in secret through a collaboration with key people in Microsoft, and their bottom line was a recommendation that Microsoft suppress competition by “de-commoditizing” protocols (i.e., creating proprietary formats that could not be used by others) and by attacking competitors through patent lawsuits. The use of a patent here does not necessarily mean that is Microsoft’s intent, but it does explain why Microsoft’s actions with patents are watched so closely. Since patents can be used to enforce terms that are otherwise unenforceable, such patents become a key issue. The issue is not that patents necessarily disqualify a format; the issue is that patents can enforce terms that then need examination.

Microsoft hopes to have Office 12 out next year, so there aren’t really any full, mature implementations yet of the XML format that Microsoft wants Massachusetts to standardize on. ( eWeek’s Steven J. Vaughan-Nichols, among others, notes that “the so-called open Office 2003 XML formats aren’t going to be Office 12’s formats.” -- they are incompatible.) Are there multiple implementations? No. How about one using the GPL (a really good acid test for competition)? No, and unless there are license changes, there won’t be one. So, there is no evidence that there can be free and open competition using the Microsoft XML formats.

Bottom line on open, agreed-up standards: Competition permitted

This isn’t complicated at all, really. The point of all this is full and open competition, with a range of competitors actively in the market. Are some valid suppliers prevented from entering the market, or at an unnecessary disadvantage?

Few leading vendors ever really want to have an agreed-on open standard, because that opens the market to competitors. But once a technology stabilizes, wise customers demand standards that let them pick between suppliers, and customers are the ones paying the bills. This happens all the time, in all technology areas. In this case, office documents have been around for decades, so it’s a very mature technology and long overdue for an open standard with a competitive market.

Massachusetts explained at their Sep. 16, 2005 “Town Hall” meeting what being open required. They included no or minimal legal restrictions (so anyone can implement it), it’s published and peer reviewed (so problems can be fixed and it’s easy to implement), and joint evolution (so future changes are not controlled by a single entity or pair of entities, but by all). These are good and necessary requirements, but I would add one more: evidence of unfettered, active competition (in the form of implementations with a variety of licenses).

It’s all too easy to meet some list of requirements, yet fail to really provide what the customer wanted. The best proof of competition is in the pudding: are there competing implementations, under a sufficient variety of licenses (proprietary through GPL), that proves that there is no discrimination? Are those competitors reporting that they are disadvantaged by the standard? HTML is a good example -- there are a variety of readers and writers of the HTML format (with proprietary and open source software licenses, including the GPL), and there is a neutral body (W3C) which created, published, and maintains the standard. PDF does very well by this measure, too. There are multiple PDF readers and multiple PDF writers, in both cases ranging from proprietary through open source software (including those using the GPL). ISO acts as a neutral body to review and maintain PDF specifications. Later in this paper I discuss PDF in more detail. OpenDocument is easy to prove the same way: there are multiple OpenDocument implementations already, some proprietary, some open source software (including those using the GPL), with more on the way; OASIS is acting as the neutral party supporting peer review and joint evolution.

Obviously, it is possible to have competitors even when all others are at a competitive disadvantage. Which is why the other points Massachusetts raised (no legal restrictions, published, peer reviewed, and joint evolution) are still important to examine. But the real point is to foster full and open competition.

Choosing OpenDocument does not mandate open source software, nor does it mandate proprietary software. OpenDocument is a choice that lets you choose. Choosing Microsoft’s XML format locks you into one supplier for now and possibly forever; it certainly doesn’t lead to a road with full and open competition. Even if it is legal to implement the format, which the legal analysis earlier makes doubtful, and even if competitors implement it fully (which may be technically difficult since it was not reviewed through a consensus process), the complete control of Microsoft’s XML format by Microsoft into the future serves as a deterrent to competition. Microsoft appears to have a right to limit who can use their proprietary XML format in the U.S., according to current U.S. law; but customers have a right to avoid it, too.

If we take this part of the letter at face value, by itself it disqualifies Microsoft’s XML format from consideration. Microsoft’s format was never reviewed by an independent body with representatives from multiple suppliers. It can’t be implemented by all suppliers, either. Even if there are suppliers who can and will use it, they would be unwise to depend on it, because it would be directly controlled by an organization with a financial incentive to harm them (e.g., through capricious and unforewarned changes). Governments have been asking Microsoft to change their approach to their XML format for a long time, to no avail. Microsoft no doubt has business reasons for its decision, but Massachusetts has the right to make decisions that are best for it too.

Now that we understand some of the real issues underlying this letter, let’s move on.

We have substantial concerns, however, with the definition of “open formats” in the current proposal. This definition mandates adoption of a single, immature format for office documents throughout the Commonwealth’s executive agencies...

And here we come to the nub. They completely disagree with Massachusetts’ choices, and try to squeeze as many complaints as they can into one paragraph.

It may seem odd that they start by complaining about a definition, of all things, but it makes sense. As any government should, Massachusetts started its decision-making process by identifying its requirements. In this case, Massachusetts identified one of their key requirements as the need for an open format, including a definition of that term. Microsoft is unhappy with this requirement. By saying that the definition “mandates” the choice, they acknowledge that Microsoft’s format does not meet the Massachusetts requirement as currently defined, nor do they want to perform the necessary changes so it would meet the definition. Since they do not want to perform the changes necessary to meet the requirement, they want Massachusetts to change the requirement.

This kind of request is not necessarily unreasonable, depending on the strength of their argument. However, governments often identify their key requirements and try to meet them directly, so the argument needs to be really strong. The rest of the letter tries to give that argument. In my opinion, their argument is inadequate.

They also complain that the format is “immature”. All formats are immature, in the sense that there will be future improvements, clarifications, and so on. But OpenDocument has lots of evidence justifying that it is mature for its intended purpose. We’ll examine that evidence in more detail below, in two parts: is the specification mature (in concern (iii)), and will the products be mature in the timeframe Massachusetts is requesting (in concern (v))? Let’s get back to the letter.

and effectively requires deployment of a single office application technology within those executive agencies.

Microsoft says here that OpenDocument would require deployment of a “single office application technology.” This is a bizarre argument, because it’s a good reason to avoid Microsoft’s XML format (which they’re proposing instead). To my knowledge, there is only one application technology and one supplier who has committed so far to Microsoft’s new XML format -- Microsoft. Certainly Microsoft’s format has far fewer suppliers backing it than OpenDocument. This text seems to suggest that Microsoft’s XML format should be disqualified, by their own rationale.

In contrast, there are multiple suppliers of OpenDocument already. OpenOffice.org and StarOffice do share a technology. But KOffice already implements OpenDocument with a completely different implementation. See my discussion about concern (v) for more information. Perhaps someday there will be multiple office suites that implement Microsoft’s XML format, but since all the other suppliers are working on OpenDocument, it’s quite reasonable to think that OpenDocument has a much more promising future.

Besides, Massachusetts is clearly trying to get many suppliers to implement this requirement by 2007, not lock themselves into the choices that are available now. Massachusetts has announced a requirement that won’t be imposed for 16 months. The most likely reason for this lengthy lead time is to encourage even more suppliers to add support so that they can compete for Massachusetts’ business. This is how any big customer gets its needs met -- it announces its requirements, with enough lead time so competitors can meet the requirement, and then it picks the best available solution. The text about concern (v) discusses implementations in more detail.

Massachusetts could choose to use two incompatible standards for the same thing, but nobody wants to do that. The costs and problems are high, with very little benefit to Massachusetts. If Massachusetts used Microsoft’s new patent-encumbered format at all internally, then essentially they’d be locking themselves into a single supplier for years to come to process those documents. Choosing two formats gives them all the drawbacks without any of the benefits. Requiring a single format -- as long as anyone can implement it -- levels the playing field.

As such, this unprecedented approach...

I respectfully disagree. It quite normal for a government to determine its requirements, pick a standard for meeting them, announce the standard, and then make purchasing decisions; governments do that all the time. In fact, this announcement is unusually generous; it gives plenty of time (16 months) for interested parties to implement the standard. OpenDocument can be implemented by anyone (without restriction), and was developed by an industry consortium of multiple suppliers. This is the ideal case, actually; governments sometimes have to make many more compromises to get things done, and are usually delighted when they have such a clear option available to them.

It’s not unprecedented to change office programs, or even to have a market-leading office program get dethroned. Remember WordStar? VisiCalc? It’s not even unprecedented to stop using Microsoft Office, if that turns out to be the ramification; other governments have done so.

In fact, the history of standards and governments has many similar examples. The U.S. government imposes a standard for the distance between rails in a railroad track, for example. There was a significant one-time cost for doing this, but the continuing advantages have been tremendous.

Having a lock on the current market is no guarantee of the future, in general; change is certainly not unprecedented. David Halberstam’s book “The Reckoning” gives an example from carmaking:

“The US Big Three automakers thought that they could dictate what their ‘captive’ market could buy, but the US public proved that assumption to be false, in the mid seventies. The only survivors from that era of heavy, rear wheel drive land yachts (albeit in much reduced and much improved forms) were the Ford Crown Victoria/ Mercury Grand Marquis and the Lincoln Town Car. Every other passenger vehicle is some variation of the K-car.”

It’s also quite quite common for a group of suppliers to overtake a big market-leading supplier by agreeing on a (more) open standard; the resulting competition tends to produce superior products at lower prices, and customers quickly flock to the open standard. Look at the videotape standards war of VHS vs. Sony Betamax in the 1980’s. Sony was a big company, trying to control an industry through a format it created -- Betamax. But the rest of the industry chose VHS, which allowed many different suppliers (Sony tightly controlled who could implement Betamax, while the VHS specification was far more open to implementation by others). The group of VHS suppliers quickly competed with each other, while staying true to the standard, customers preferred formats where there were competing suppliers, and as a group the VHS suppliers demolished Betamax’s market share.

There’s even a slang term based on this; “to betamax” means “to deploy a proprietary technology format that gets overwhelmed in the market by a more open format that allows multiple competing manufacturers.” There’s more to that story, of course, but my point should be clear: open standards, supported by competing implementations, tend to win in the marketplace. The fact that there’s a slang term for this complex situation tells you something else: this script happens repeatedly.

The market forces are clear; often smaller competitors (who fear being driven out of the business) will band together with customers (who fear having a sole supplier) to develop and promote a standard that is not (as) proprietary. Eventually the broadly-created open standards tend to win, because customers make the final decisions, and few customers want to be controlled by any single supplier. Having broad input into the standard’s development also helps make sure that all important needs were addressed, as well as enabling equal competition.

not only prevents impacted state agencies of the Commonwealth from using many critical and well-established technologies, ...

And there it is! The real reason they believe Massachusetts shouldn’t do this is right here. Microsoft doesn’t want to support OpenDocument, even though they can. If Microsoft won’t support OpenDocument, and Massachusetts sticks to its requirements, then Massachusetts may eventually need to migrate to a different office suite other than Microsoft Office. Microsoft doesn’t want Massachusetts to use the standard, nor do they want to implement the standard. I suspect many will read this text as a threat to punish Massachusetts for daring to choose the international standard instead of Microsoft’s proprietary specification.

This is circular reasoning. Microsoft has the power to implement OpenDocument in Microsoft Office, if they choose to do so. It’s not a lack of money, certainly, and the specifications are free for anyone to implement (including Microsoft). Sixteen months is enough time to implement it, and if they don’t want to do it themselves, they could license the code from someone who has already done it. It is not Massachusetts who is deciding to stop using Microsoft Office; it’s Microsoft who is threatening to not do business with Massachusetts. Massachusetts does not want to be locked into any single supplier, so it is choosing a format that any supplier can implement. If Microsoft decides to not implement the requirements that its customers demand, then Microsoft shouldn’t be surprised if customers switch to suppliers more willing to meet customer needs. It’s how competition works.

Wise customers choose markets with many competing suppliers. The competition acts as an impetus for both innovation and cost reduction. Think Betamax, for example.

Massachusetts’ Frequently Asked Questions (FAQ) makes some interesting points here. This selection won’t really impact the current well-established technologies, such as Microsoft Office; as they note, “use of existing MS Office licenses is allowed as long as agencies use a method permitting the saving of documents in Open Document Format.” They apparently already have a process for doing this. Their policy also limits “the applicability of the Open Document Format standard to documents created by the Executive Department in the future” so they won’t have to convert what already exists.

but also runs afoul of well-established procurement norms without due consideration for the enormous costs and technical challenges that stem from the proposal.

Not at all. We’ll talk more about procurement norms as part of concern (i), but a few comments are appropriate now. If there were only a few days to discuss this, I agree that’d be a problem. But this has been going on for a long, long time. Paula Rooney reports in InformationWeek that “The state and software company have locked horns on this issue for more than two years.” There were major statements in January 2005, and really all through the year.

We simply do not believe that the proposed mandate for this exclusive document format is the best solution for achieving the Commonwealth’s laudable goals.

It’s always fair to state that you have differing beliefs, so this is perfectly reasonable. But clearly, we need to look into why they believe that, so let’s look at their key concerns.

Microsoft’s key concerns are as follows:

Now we have a short list of “concerns,” followed later by more detail. It’s hard to respond to a claim without seeing the supporting information behind it, so I’m going to show not only each of their concerns, but I’ll extract some details from the rest of the letter to see what they really mean. Unfortunately, this letter is not well-organized. In the front, it has a list of 7 concerns numbered (i) through (vii), but the details of its arguments are numbered 1 through 5 with different headings. Since the organization of the rest of the letter doesn’t exactly match the list of key concerns given in the front, it’s more awkward than most letters like this. Let’s soldier on, looking at each of their 7 concerns and then the closing text in the front part of the letter.


(i) ANF did not provide sufficient time for review and comment on the proposed policy, nor a robust process for addressing comments. Due process requires much more, particularly given the unprecedented nature of the proposal and the potentially adverse consequences it could provoke,

This clearly maps to the details listed under “reason 1,” which is later in the letter. Reason 1 claims that the “Executive Office for Administration & Finance did not provide sufficient time for review and comment on the proposed policy, nor did it provide a robust process for addressing comments.”

Microsoft’s letter gives more details, and quotes various laws, which I don’t want to bore you with. In short, this claim doesn’t seem to hold water.

First of all, Massachusetts has been publicly discussing its needs for quite some time, not just a few days. As I noted before, Paula Rooney reports in InformationWeek that “The state and software company have locked horns on this issue for more than two years.” On January 15, 2005, Eric Kriss made many public statements about open formats, and a vast number of meetings and public statements have been made throughout 2005.

David Berlind’s ZDNet makes a number of observations:

“There’s no question that Microsoft’s Office XML Reference Schema was once on Massachusetts’ “list,” and then, as a matter of public discourse--some might say democracy-- removed from that list. Going back to the notes from Massachusetts’ June 9, 2005 meeting with industry-- referred to by Quinn as the Open Formats Summit--it would be impossible for anybody, Microsoft included, to assume that any format that was once on the “list” was guaranteed of staying on that list. In the very last section of those meeting notes, the next steps were clearly defined as:
  • Identify a continuum of acceptable open document standards for the Commonwealth.
  • Revise the standards and publish the revision for public comment; then finalize the standards.
These action items practically scream out that, at the time, Massachusetts was undecided on a standard and that the due diligence for choosing one was far from over... Microsoft’s team on the job--Stuart McKee, Brian Burke, and Leslie Tan--didn’t have to look far to see some of the criteria for openness that the State was considering. It was displayed in a footnote on the bottom of the June 9, 2005 meeting notes and stated as follows:
‘Specifically, [Ken] Krechmer, [Fellow, International Center for Standards Research, University of Colorado] notes, standards creators typically consider a standard to be open if the creation of the standard follows the tenets of open meeting, consensus and due process; implementers of an existing standard would call a standard open when it serves the markets they wish, it is without cost to them, does not preclude further innovation (by them), does not obsolete their prior implementations, and does not favor a competitor; and users of an implementation of the standard would call a standard open when multiple implementations of the standard from different sources are available, when the implementation functions in all locations needed, when the implementation is supported over the user’s expected service life and when new implementations desired by the user are backward compatible to previously purchased implementations.’

[Noting that Microsoft’s format would fail this test,] With the writing on the wall, the question that sticks out in my mind is why didn’t Microsoft sound its internal fire alarm sooner?”

It’s certainly not a last-minute decision; the issues and their ramifications have been publicly discussed for a long, long time. It’s not just Massachusetts; a lot of work was done by the European Union in 2004 (and probably even earlier), and governments talk together about things like this. There’s absolutely no evidence that any new information is forthcoming; the issues at this point are well-understood. Decision-makers are paid to make decisions, not to stall forever. At some point, there comes a time to make a decision; not making a decision is, in the end, a decision.

In their FAQ, Massachusetts explains why this didn’t require a regulation review process -- because it’s not a regulation. “In issuing the draft Final ETRM version 3.5, ITD is not proposing a ‘regulation’ as that term is defined under the state’s Administrative Procedures Act, Mass. Gen. L. ch. 30A. Section 2 of that Act defines ‘regulations’ for which agencies must conduct formal notice and comment proceedings to exclude ‘(b) regulations concerning only the internal management or discipline of the adopting agency or any other agency, and not substantially affecting the rights of or the procedures available to the public or that portion of the public affected by the agency’s activities’. Under the Final E TRM 3.5, ITD would require use of Open Document format only for documents create by Executive Department agencies. It would not require use of such formats by citizens, businesses, and other governments who communicate with the Executive Department. ITD is a control agency with no regulatory authority over any citizen, business or governmental entity outside the Executive Department. In that the Final ETRM 3.5 does not constitute a ‘regulation’ under the APA, ITD was not required to engage in formal rulemaking in connection with the issuance of this standard. It offered an information notice and comment proceeding voluntarily and in the spirit of cooperative leadership that ITD promotes.”

Their FAQ discusses several other legal points, justifying their belief that the law was followed. It appears that Massachusetts is quite within the law. In fact, it appears that they went far beyond all legal requirements; they had an unusually lengthy public comment period.

It’s important to note that this is not a procurement, anyway. Eric Kriss made this very clear at the Sep. 16, 2005 “Town Hall” meeting, and it was clear before that time too. Massachusetts is warning suppliers that a procurement will come in the future, and that a particular standard will be required in that procurement. This is just what suppliers want to know, so that they can make their plans. By letting suppliers know the requirements ahead of time, suppliers can plan and work to meet those requirements, if they want to be considered for the procurement that will come later on. It makes no sense for Microsoft to complain that they’re being warned about a future requirement; instead, Microsoft should be thanking Massachusetts for giving clear direction and a long lead time.


(ii) the proposed policy would create significant costs and problems for state agencies, for the private sector, and for its citizens,

Well, any decision can involve costs and problems; even not doing anything can involve costs and problems. To figure out what they mean, we’ll need to look at the details later on in the letter. Here’s what they say in their “reasons” section, which gives more details:

2. The proposed policy would impose enormous costs on the Commonwealth of Massachusetts and on its citizens and the private sector.

If the proposed policy were put in place, the fiscal impact on the Commonwealth, its citizenry, and the private sector would be substantial.

First, there would be significant, and entirely unnecessary, costs incurred by all state agencies, departments, cities, counties, and school districts to procure new software applications that support the OpenDocument format for their individual users. Many state agencies already have licenses for Microsoft Office and other software products that do not support the OpenDocument format, and the expense already borne by these state agencies for Microsoft Office and such other products’ licenses would be wasted by disallowing use of these products after Jan. 2007. As a result, costs to taxpayers would rise as executive agencies would be forced to toss out software they have already paid for, that they already know how to use, and that they can already use for archiving in open standard XML formats.

Second, every state agency, department, city, county, and school district would face enormous document and/or application conversion costs and would need to invest in training and support activity in order to make this conversion, with potential risks arising from conversion errors in these public documents.

Massachusetts’ FAQ has some simple answers to this. Most obviously, this directive “applies only to the Executive Department, and then only to documents created by the Executive Department. Implementation plans will take into account the need to maintain interoperability through the use of a variety of acceptable formats.” Yes, they already have licenses; “use of existing MS Office licenses is allowed as long as agencies use a method permitting the saving of documents in Open Document Format.” They already have such methods available, which gives them a graceful way to implement this plan. Besides, they note that they will implement this in a phased approach. By doing things in phases, they can dramatically reduce the effort.

Though it doesn’t appear in their FAQ, I know that a survey found that far more people just read documents than write them. I would certainly expect that most documents created by the Executive Department would be read more often than modified by those outside of it. In that case, this issue is completely irrelevant; people will just want the PDFs anyway.

Possibly most important, it’s not a question of cost vs. no cost; it’s a question of where the money should go. Their FAQ notes that “the Executive Department will face significant costs over time in connection with office application upgrades. There is no evidence that migrating to office applications that support Open Document Format will be any more costly than upgrading current applications.”

(Footnote) The impact of this proposal extends far beyond Microsoft. For example, agencies that have chosen to make use of Corel software, such as the Massachusetts court system, will face similar challenges, and it is unknown how the proposed policy will adversely impact existing guidelines in place for such agencies, such as the Massachusetts court system’s electronic submission guidelines.

We’re talking here about Massachusetts’ executive branch. The judiciary was not in the executive branch, so this is irrelevant. In fact, Massachusetts’ courts determined about 30 years ago that, by the rules set in Massachusetts’ Constitution, one branch of Massachusetts’ government could not set the information technology rules for another branch.

Corel is one of the original authors of OpenDocument, anyway; For more information about Corel, see the discussion about concern (v).

(Footnote) Some may argue that lower licensing costs associated with software technologies supporting the OpenDocument format counters the cost associated with the migration. Recent Gartner analyst reports, however, have documented examples where organizations who have closely evaluated the issues conclude that a move to alternative software has no defensible ROI; in fact, those organizations have concluded that the preferred approach was to maintain and continue deployment of Microsoft’s most recent software technologies. See: A Financial Institution Sees No ROI on Desktop Linux. In many cases, companies license technologies for “free” or at very low sales price in the hopes of making money on other related products and services including sales of complementary proprietary software and hardware and service contracts. There are a number of examples of government entities that migrated away from some of the software that will be supporting the OpenDocument format due to total cost of ownership (including testing and installing, and training costs) among other factors.

This is a completely irrelevant study. Massachusetts is requiring a file format, not any particular product. The referenced article talks about “Desktop Linux”, and this whole footnote is talking about GNU/Linux -- which isn’t even the topic. The issue being addressed by Massachusetts is a standard file format (OpenDocument) that can be used on Microsoft Windows, Apple Macintosh, GNU/Linux, Unix, whatever. GNU/Linux is an interesting topic, but irrelevant, unless you wish to note that Microsoft’s file format is not supported on Unix and Linux (and thus inhibits the free selection of products). However, that’s yet another argument for avoiding Microsoft’s XML format, and using OpenDocument instead, if that is the point.

Even if this were relevant (it’s not), reusing ROI or TCO studies is inappropriate. The purpose of a TCO or ROI study is give an answer for a particular organization; such studies are so sensitive to the specific organization’s situation that you usually can’t reuse them for anyone else. Others’ ROI or TCO studies can be helpful to point out what might happen or might be true, so as “examples” it’s fair to say it’s possible. But they are useless for proving what will happen for someone else.

Even if this were relevant (it’s not), and even if reusing ROI/TCO studies made sense (it doesn’t), real TCOs and ROIs for governments look completely different from commercial organizations. So in particular they can’t be reused for this case. The reason is that governments plan to be around for centuries, and they plan to have to deal with their old documents in those future centuries. When your time horizon is in decades or centuries, you have to assume that you’re going to need to upgrade many times, and that you’ll have to change suppliers many times. Today’s transition costs matter, but are dwarfed in these timeframes. What standards will help you handle supplier changes many times over centuries? Probably not one controlled by a single supplier. And governments have requirements, like open access to all their constituents (and not just the rich ones), that a commercial company does not need to address.

Besides, a real TCO has to look at the whole picture, not just the office suite software. As Carr notes, Massachusetts is “launching a serious and comprehensive initiative to replace its fragmented, inefficient set of traditional information systems with a modern, coherent, and flexible IT architecture that allows data to be shared and reused easily.” The cost reductions from replacing a fragmented set of systems with a flexible architecture that allows full data sharing will probably dwarf any additional costs for the office suites themselves, if they even have an additional cost. And the flexibility of using any supplier, which Microsoft’s XML license does not grant, will probably lower the costs of implementing this.

But it’s actually pretty unlikely that the suites will cost more. In fact, it appears this move will also save them money on the suites themselves, even though that is not the primary purpose. At a Sep. 16, 2005 Town Meeting, Eric Kriss, the state’s Secretary for Administration and Finance, revealed some very interesting numbers. The state operates about 50,000 desktops, mostly Windows 2000 and Office 11. They roughly estimate that to upgrade to Office 12, which Microsoft is offering as the way to implement their format, would cost $50 Million (including software licenses, upgrading operating systems as needed, newer hardware in some cases, and training). In fact, an Office 12 upgrade will apparently require them to upgrade a bunch of operating systems too, and since its user interface has substantial changes, there will be a substantial training cost no matter what they do (the alternatives may actually have a lower training cost too). They estimate the hypothetical cost to install OpenOffice.org instead (one of the programs that implements OpenDocument) as $5M with comparable components. Even though these are crude estimates, the scale of the difference suggests that even getting better pricing from suppliers and discovering an unaccounted-for cost is not likely to make Microsoft the less expensive bid. That obviously includes more than the purchase price, because OpenOffice.org is available at no cost, in perpetuity. Even more tellingly, it’s reasonable to believe that many of the $5M costs are one-time costs, not per-upgrade costs; it would presumably cost much less for all upgrades after that. Even a great vendor discount can’t hide the cost difference. When you factor in upgrades every 2-3 years, over centuries, the difference becomes extreme.

Besides, as Kriss notes, cost is not the key reason for this decision. Sovereignty is. They’re willing to pay more, if it’s necessary to meet their fundamental needs. If cost was the only reason to make a decision, we would all be driving the cheapest car available. Sometimes it is worth paying more, to acquire something else.

Third, extensive work would have to be done deep within the IT infrastructure of the Commonwealth to manage conversions of “non-compliant” documents and mapping of processes that work well today to new, untested systems. On a daily basis, state agencies would need to work with private sector organizations and citizens to devise ways to convert documents back and forth and to troubleshoot problems. One could also assume that other branches of the Commonwealth’s government would incur substantial expenses in order to adapt IT systems to be able to interface with the overhauled systems of the executive branch.

As noted above, no conversion is needed; this policy only applies to new documents. There are freely-available programs that do this conversion, if it’s needed (OpenOffice.org even has a built-in “bulk converter” does that does it very painlessly). In fact, it’s hard to not notice that Microsoft’s own letter was quickly translated to OpenDocument format when it was posted by Massachusetts.

And again, this argument can be turned around. If switching to an XML format is this hard, you certainly want to choose a format that everyone can implement, because you don’t want to get stuck with a single supplier or a small set of suppliers. Otherwise, you risk having that supplier raise their rates once you have committed to them.

Fourth, new costs and problems will also be imposed on those doing business with the state, including organizations, businesses, and citizens, as the proposal could take away their choice of the software they may want to use to interact with the government to, for example, bid on a government project, submit filings, or correspond with government officials. Further, Massachusetts companies who currently sell products that do not comply with the proposed policy to Massachusetts agencies will be cut off from a major customer base.

The way to allow choice is to pick a specification that all suppliers can implement. OpenDocument meets that criteria, Microsoft’s does not. If the supplier chooses to not implement the international specification, that’s the supplier’s choice, not the customer’s.

Indeed, the proposal itself acknowledges the current pervasive deployment throughout impacted agencies of technologies not compliant with the proposed policy and the magnitude of the resulting costs that would be associated with the migration effort: Given the majority of Executive Department agencies currently use office applications such as MS Office, Lotus Notes and WordPerfect that produce documents in proprietary formats, the magnitude of the migration effort to this new open standard is considerable.

This will be a big change; Massachusetts has lots of office suite applications. They will need to be upgraded, replaced, or have extensions/plug-ins added to support the new format. How extensive it will be will depend on the suppliers, to some extent, and will greatly depend on wise planning. But big changes often have the biggest payoffs, if they’re handled well.

Note that Microsoft concedes here that their current Office suite produces documents in proprietary formats.

There is simply no principled basis for causing the foregoing costs to be borne by the Commonwealth, its citizens, and the private sector, ...

Not true. Massachusetts has repeatedly emphasized why it needs standards that are widely vetted by an independent body, and that are implementable by all.

At the Sep. 16, 2005 Town Meeting, Eric Kriss summarized the reason very simply: sovereignty. Kriss noted that without public documents, you don’t really have a democratic government, and increasingly many government documents are only available electronically. The fundamental question here is, who owns Massachusetts’ documents? The supplier, or the state? If it’s the state, then it must be able use its data however it chooses, using any supplier’s products. “Public documents must be as open and as unencumbered as possible... because it’s a democratic imperative.” I.E., if the state wants to be able to exercise its rights, then its data needs to be in a format where it can actually exercise its rights. Microsoft is preventing this free exercise, by not allowing its XML format to be reviewed and controlled by an independent body, and by only licensing their format so that data in the format cannot be used by many of its competitors. Massachusetts has determined that this limitation is incompatible with Massachusetts’ sovereignty, and thus must be rejected.

Marc Wagner’s “Microsoft and public access” noted that this debate “has one component that no one in the public sector should be dismissing -- that of access to public information. Obligating a citizen to download a free document reader (one that runs under any operating system available today) is one thing, but obligating a citizen to own a Windows-based system and MS-Office is quite another. So what if 400 million people use MS-Office everyday? We can say with absolute certainty that there are people in Massachusetts who own Macintosh computers and Linux computers and UNIX computers. I didn’t see anything in David’s article that suggested the Commonwealth of Massachusetts objected to Microsoft protecting its IP. All they were saying is that if Microsoft wanted them to use its document formats, that Microsoft needed to make those formats accessible to those who did not own Windows and Office. Seems reasonable to me.”

Peter Winston added yet another (and simpler) explanation at the Sep. 16, 2005 “Town Hall” meeting (at 01:21). These formats are data interchange formats; it simply makes no sense to select a specification that inhibits arbitrary data interchange.

The practical, specific needs for this are spelled out in articles by Nicholas Carr. Carr describes their needs this way:

“I assumed [this] was just another case of anti-Microsoftism, an assumption backed up by the press’s ‘Massachusetts Dumps Office’ headlines. I was wrong. This isn’t about Microsoft. It’s about a state government launching a serious and comprehensive initiative to replace its fragmented, inefficient set of traditional information systems with a modern, coherent, and flexible IT architecture that allows data to be shared and reused easily. The adoption of OpenDocument as a standard is just one element of the state’s ambitious plan. As described in a comprehensive technical paper, called the Enterprise Technical Reference Model (ETRM), the state aims to make a transition ‘from siloed, application-centric and agency-centric information technology investments to an enterprise approach where applications are designed to be flexible, to take advantage of shared and reusable components, to facilitate the sharing and reuse of data where appropriate and to make the best use of the technology infrastructure that is available.’”

Carr argues that:

“The proposal for adopting consistent, open standards for document formats needs to be understood in the context of this broad, long-term effort. The state, like other government bodies, is struggling to meet three particular challenges with regard to the large number of digital documents it produces and stores: (1) sharing the information in those documents seamlessly among diverse agencies (overcoming the incompatibilities of its various existing systems and data formats), (2) ensuring the accessibility and integrity of those documents in perpetuity, and (3) guaranteeing that information can be dispersed quickly in the case of a disaster or other emergency. (The problems that governments had in responding to 9/11 and Hurricane Katrina, some of which can be traced to an inability to share data among diverse computer systems, reveal the urgency of this last goal.) To accomplish these goals, it is requiring that all government agencies use a common set of non-proprietary, XML-based data standards, including OpenDocument for office documents.”

Gary Edwards makes similar points.

In short, Massachusetts believes it has fundamental principles that are the basis for such costs, if they turn out to be costs. This approach is necessary to maintain the State’s sovereignty. So let’s keep reading.

particularly given a) the significant flaws with the OpenDocument format, and b) the availability of more cost effective alternative ways to achieve the Commonwealth’s principal data interoperability objectives. These issues are discussed in turn in the following two sections.

Microsoft has had many years to prove that there are significant flaws in OpenDocument, and it has still failed to do so. OpenDocument’s predecessor has been in use since 2000, it has received detailed multiyear review from multiple experts, and it’s received worldwide public peer review as well. Are they really saying that Corel (Word Perfect), Sun (StarOffice), OpenOffice.org, IBM, Adobe, KOffice, and others can’t figure out how to store documents, even though they’ve been doing it for so many years? They can say it, but there’s no evidence to believe it.

No doubt OpenDocument will be improved as all such formats are. For example, many people (including me!) believe that the format for spreadsheet formulas should be described in more detail (there’s material in the specification, but more detail would be very helpful). But all agree that the format is able to exchange the necessary information, and Microsoft’s XML specification is no better in this area anyway. Indeed, all evidence suggests that OpenDocument is perfectly up to the job that Massachusetts has for it. Microsoft has failed to demonstrate that there are “significant flaws” that are so serious that it cannot be used by 2007.

For a cost-effective alternative, we need to see one. Microsoft’s old binary formats are probably the best alternative; at least they’re widely implemented. But they do not provide the processing or long-term storage advantages of XML, and they’re not even specified publicly. Even later versions of Microsoft Office occasionally fail to read this format; it’s clearly a failure for long-term storage. The sooner Massachusetts moves away from the binary formats, the faster they can use XML processing and stop losing data. Even Microsoft is abandoning their old binary format; no one believes that the binary format has a long-term future. Microsoft’s XML format is proprietary, with no independent vetting and with a license that forbids many competitors from implementing it, as I noted before, so it’s not a real alternative.

Now let’s go back to the list of concerns in the front of the letter...


(iii) the document format designated in the proposed policy is new to the marketplace, still subject to potential revision, and not widely deployed or tested in a wide variety of product or usage scenarios,

This seems to partly map to this later assertion later in the letter (remember, they didn’t organize this letter clearly):

3. The OpenDocument format is immature and not widely accepted in the industry or public sector, and mandating the adoption of this format would present affected state agencies with significant technical obstacles.

This text suggests a lack of understanding in how standards, customer plans, and industry adoption work. There are two parts to maturity: (1) is the specification mature enough to be implemented, and (2) will implementations be mature enough for their intended use when they will be used?

If you step back and see how things normally work, it’s actually quite straightforward. Here’s the normal progress of events when a standard begins to develop:

  1. Independent standards bodies, gathering various suppliers and users, work to create a specification that can be implemented by the various implementors. Usually they start with a specification of something that’s already in use and closely meets the needs of standard (as a “base document”); OpenOffice.org’s old XML format was used as the basis for OpenDocument. Then they make changes based on the experience and needs of other suppliers; those changes often make the specification far more general and useful than any one of the original suppliers’ formats. Usually a few of the eager suppliers actually start to implement it before the specification is complete; this helps ensure that the specification will be mature (implementable) when it’s finally approved as a standard. When it’s finally approved, that generally means that the specification is mature enough to be implemented.
  2. Customers examine the standard and, if they like what it will do for them, talk with their suppliers and warn them that they will need to implement the standard by some future date if they want to supply products in the future. Suppliers, in turn, decide if or when to implement the standard. Suppliers who implement early may have an edge, since they’ll be out to market first. But sometimes customers don’t demand a standard; suppliers will sometimes wait to see if any customers really care about the standard, and only implement it if anyone actually asks for it. Suppliers who choose to not implement the standard may decide that they can still have a market without it, or decide to withdraw from the market altogether. This is the stage we’re in now; customers are looking, and in this case Massachusetts is saying that they want what the standard can do for them.
  3. Suppliers complete implementation of the standard, do tests, and so on; at that point, the implementations are mature.
  4. Customers buy the implementations that meet their standards (and shun the products that fail to meet them).

There’s lots of evidence that the specification is mature enough to be implemented:

  1. The most obvious is that OASIS has formally approved OpenDocument, over half a year ago (May 1, 2005). Formal approval by a recognized standards body is usually considered prima facia evidence that the specification is mature and ready for implementation.
  2. Multiple organizations who implement the products (office suites in this case) cooperated to develop the standard. That includes Sun/OpenOffice.org, Corel (Word Perfect), IBM and KOffice. Peer review by multiple organizations is one of the best ways to get a mature standard, because it’s a lot less likely that important things will be overlooked.
  3. Multiple users who have stressing needs were involved in developing the format. These include Boeing (who create complex, large documents), the National Archive of Australia (who need to be able to retrieve documents long after development), the New York State Office of the Attorney General (both), and the Society of Biblical Literature (who need to manage large multilingual documents). Intel is developing a set of sample documents to be used as a test suite. When diverse user needs are represented, the result is much more likely to meet a variety of needs.
  4. The group started with a pre-existing format that was already in use, and already known to generally meet the group’s requirements: OpenOffice.org’s version 1 format. OpenOffice.org began as StarOffice, from the StarDivision company founded in 1986. The XML format itself was released in 2000, and has already seen a great deal of use ( an estimated 25 million users).
  5. Office documents are not a new technology. Markup languages have been used to transmit documents for decades, including the predecessors of SGML, SGML, HTML, and XML.
  6. The OpenDocument format has undergone two and a half years of multi-supplier discussion and public commentary, making a number of improvements over that time. The first official OASIS meeting to discuss the standard was December 16, 2002; OASIS approved OpenDocument as an OASIS standard on May 1, 2005. That’s actually quite a bit of time.
  7. There are no real non-proprietary alternatives for an XML office document format, which should speed implementation. This is the only XML-based document standard accepted by a neutral standard-setting body. The closest “competitors” are Microsoft’s binary format (which is binary and unspecified), XHTML (which simply doesn’t have the functionality required), and PDF (which is essentially read-only).

Microsoft says that “the open document committee of the OASIS umbrella organization did not include any government representatives.” OASIS’ umbrella organization does not have government representatives, but the OASIS OpenDocument committee formally included the National Archive of Australia and the New York State Office of the Attorney General. In addition, the OASIS process allowed anyone (including any government representative) to submit a public comment on their drafts. Massachusetts has had ample time to examine the specification themselves for suitability. Governments will have one last chance to examine the specification at the ISO level, too; ISO votes are by country, not by company. At this point there’s every reason to expect ISO will accept OpenDocument.

Microsoft says that participation in the OASIS standards process was only from a “narrow set of companies” and suggests that it cannot work across a “range of organizations with varying infrastructure, skills, requirements, and needs”; this is clearly untrue. I see Sun/OpenOffice.org and IBM, yes, but I also see Corel, KDE/Troll Tech, Intel, Boeing, the National Archive of Australia, the New York State Office of the Attorney General, and even the Society of Biblical Literature. In addition there was the lengthy public comment period, when anyone could submit comments.

Big IT companies like Sun, IBM, Corel, and Intel are widely-known, but let me focus in some of the more interesting players. Boeing isn’t usually considered an IT company, but creating a plane requires so much documentation that a single plane couldn’t carry it, and those documents are often very complex. Boeing’s involvement greatly increases confidence that complex documents with technical material can be handled.

The National Archive of Australia and the New York State Office of the Attorney General have needs strikingly similar to Massachusetts; it’s far more likely that OpenDocument will meet Massachusetts’ needs, because similar organizations were involved in its development.

My favorite organization in this list, though, is the “Society of Biblical Literature.” Why? Because I would never have anticipated that such a group would show up at a standards body... yet in retrospect their involvement strongly validates the result. This organization has to worry about complex multi-lingual documents, with all sorts of internationalization problems. Hebrew and Aramaic run right to left (not left to right), and they routinely create documents that have multiple languages in the same document with differing conventions. What’s more, many of their documents are extremely long-lived (they think in terms of millennia!), and accuracy is absolutely critical (they believe that in many cases they are dealing with God’s word). (see this introduction of their representative, a simple sample comment, and this example of an interesting need you might not anticipate). If a document format can meet their needs for accuracy and multi-lingual support, it’s ready to be a true international standard. Clearly, we have a variety of perspectives and usage scenarios represented here!

In contrast, only one company was allowed to control Microsoft’s XML format; if this is the criteria, Microsoft’s XML format is disqualified.

Microsoft complains that some companies were “promoting their own technologies.” They even note that there were two employees of Sun, and two employees of IBM, on the committee on page 12 (though the participation of the many other organizations and individuals is not mentioned). Well of course each representative promoted their own technology! That’s why neutral bodies, with representatives from multiple suppliers, are so critical -- by creating a neutral forum, everyone’s interests can be heard, and through consensus they can create an impartial result. One obvious retort to that is that KOffice (not associated with Sun or IBM) managed to implement the specification first; if we assume that Sun or IBM controlled everything to their liking, why was a completely different organization able to implement the specification first? But let’s follow the complaint; imagine that we accept the idea that it’s bad when a company promotes their own technology (I do not!). It’s easy to notice that there’s no neutral party with a multi-supplier effort covering Microsoft’s XML format; by this logic, we would be forced to presume that Microsoft’s format is designed to only promote sales of their technology (Microsoft Office) and is not supplier-neutral. By this logic, Microsoft would disqualify its own format first.

It’s worth noting that there was a lengthy public review comment period as well for OpenDocument. Though I wasn’t on the committee that developed OpenDocument, during their public comment period I read the specification myself and sent in a few comments. I was generally pleased with it after I reviewed it; there’s a lot of good to say about it. It’s actually quite clear to read, as these things go. And the careful crafting, and review by many, shows in the result. It’s smaller than it might otherwise be, because they reused pre-existing standards instead of rolling their own (that is generally a very good idea). It isn’t perfect; no human product is. But it’s a reasonable specification to use.

Microsoft notes that ISO could radically change OpenDocument. In theory that’s true, but in actuality that’s unlikely. OpenDocument is being submitted through the “PAS” fast-track process of ISO; OASIS is one of the few organizations allowed to use this process. This process rarely calls for any substantive changes at all; this process is designed for documents that have already undergone widespread, careful, and public review as OpenDocument has done. Future work will no doubt add new capabilities and clarifications, as they always do, but it’s not expected to invalidate what has already been done.

What about implementations? I’ll discuss OpenDocument implementations in depth in concern (v). The simple answer is that there are already several implementations on the market, and many more are expected by the time the policy comes into effect (2007). As you can see, Massachusetts is doing things in the usual way: an international specification is created and approved, governments and other customers then mandate the standard starting at some future date (giving a timeframe), and then suppliers have time to implement the standard in an orderly way (knowing that it’s worthwhile to implement the standard).

Microsoft claims that Massachusetts’ ETRM does not address issues such as “embedded pictures, audio, video, maps, voice...” At the Sep. 16, 2005 town meeting, Peter Quinn noted it’s a matter of priorities; “We are focused only on documents for now; it’s the pressing issue; other media [will be addressed] in [the] future.” Besides, OpenDocument already handles embedded pictures and audio (including voice). As for the rest, this is actually quite straightforward. OpenDocument is just a “zip” file format, so it can contain such information trivially, the same way that bitmapped pictures and audio are handled now.

All formats are subject to potential revision, so that concern in the letter is a red herring. Microsoft hasn’t committed to never changing their proprietary XML format, either. In fact, Office 12’s XML format will be radically different from Office 2003’s. Besides, since formats are subject to revision, they need to be controlled by a neutral third party to ensure fair upgrading. That’s a condition OpenDocument meets and Microsoft’s does not.


(iv) there are substantial technical challenges associated with implementation of the proposed policy. For example, there are issues associated with converting documents saved in the well-established, existing document formats which apparently have not been considered, including the possibility that the new policy will lock out citizens and organizations which use software applications supporting these existing formats from Commonwealth systems or services, or significantly change countless legacy documents that are not fully supported by the newly designated format,
In their FAQ, Massachusetts note that “The Final ETRM Version 3.5 applies only to new documents, and does not require conversion of documents existing before the implementation date.” Nicholas Carr has some interesting comments about conversion:
“Microsoft is, in essence, arguing for the maintenance of the status quo. It says that ‘Commonwealth agencies should be allowed to choose the technologies that best suit their needs, particularly in a context where, as here, multiple open and competing technologies/formats are available and supported in the marketplace, with many document conversion utilities already available and with no licensing barriers to future conversion software.’ But this reveals the flaw in Microsoft’s position. The state has determined that the status quo is neither desirable nor sustainable. It believes that the lack of standardization in technology and data is undermining its ability to do its work effectively on behalf of its citizens. The state, in other words, has made a conscious decision to endure short-term disruption in order to achieve a flexible and efficient IT architecture for the future. Indeed, when Microsoft points out the difficulties inherent in translating existing documents in a variety of incompatible, proprietary formats into a single open format, it is inadvertently backing up the state’s argument. The only way to guarantee that the state’s document archive will be accessible in the future is to move to a common standard today. If translating all those documents will be hard now, as it no doubt will be, imagine how much harder it would be if the state put the job off for another decade or two. The status quo is the problem the state has to solve; it is not a solution to the problem.”

Sam Hiser puts it this way:

“As a migration expert, I can state from personal and client experience that it is actually not that hard. It is certainly no more difficult than managing an organization of Massachusetts’ size (approximately 60,000 desktops) with as many as four different versions of Microsoft’s Windows operating system and possibly four different versions of Microsoft’s Office suite -- each of which has a different file format that is not backward-compatible. Additionally, any possible difficulties changing away from the Microsoft file formats make the case itself for the immediate change, for this is certainly a ‘pay-me-now... [or pay-me-later]’ situation...

Citizens who produce documents in an OpenDocument-ready office suite applications, like OpenOffice.org or StarOffice, do not need to convert their documents because they natively save as OpenDocument files by default. Statistics of common PC desktop workflow practices reveal that users often do not access a majority of their old documents and, therefore, most legacy documents don’t ever need to be converted. More than 95% of legacy documents are simple, one-page memoranda, notes, correspondences, short reports, or fax documents with minimal formatting. These simple documents convert perfectly and naturally in one click from the several different MS Office formats into the OpenDocument format, once opened in an OpenDocument-ready office suite. Trouble with document version compatibility is overstated by Microsoft for obvious reasons. It is important to keep in mind that users of MS Office have more difficulty with file format version incompatibilities than users of OpenOffice.org or StarOffice, since the latter suites open all legacy versions of MS Office documents.”

It is important to preserve important formatting, but we also need to have reasonable expectations. None of the formats being discussed here are designed for extremely high format fidelity (where the the document layout is exactly preserved across all platforms and computers, all versions, no matter what). The Valoris report’s section 2.4 notes quite clearly that the Microsoft Office binary formats, Microsoft’s XML format (they evaluated its predecessor WordML), and OpenDocument’s ancestor (OpenOffice.org 1.0 format) are all designed to be medium-fidelity formats: important layout information is preserved, but none guarantee the exact same layout on all systems. For example, Microsoft Word may reformat a document (with new spacing, line breaks and so on) simply when you change printers or upgrade a printer driver; it certainly won’t guarantee the exact same formatting if you transfer the file to a different machine (Windows to Mac or even Windows to Windows), if you upgrade the suite, or if you change to a different competing suite. For high fidelity needs, PDF is a better choice; PDF is considered a high fidelity format, though there are even cases in PDF where fidelity is not preserved. But PDF is primarily read-only, so it’s not very appropriate when you want to edit documents. Microsoft implies that there might be reformatting problems with OpenDocument if it did not handle some special-case construct in Microsoft Office, but indeed, even Microsoft Office does not guarantee the kinds of formatting expectations this implies. OpenDocument can handle today’s documents.

And of course, if there is a serious omission in OpenDocument, it can be updated to fix it. Microsoft has every right to submit comments, or join the group, like anyone else; they are even a member of OASIS.

Microsoft’s XML format is less widely used than OpenDocument and its predecessor, and it has not received the same widespread review. It’s fair to say that Microsoft’s proprietary XML format is far less mature than OpenDocument.

ZDNet puts this very bluntly: “Does OpenDocument, which is the result of a lot of hard work from people fully versed in contemporary corporate computing, really fail at the very things it was designed to provide?” and was very skeptical of this claim. There were a lot of smart people involved in OpenDocument’s standardization, many of whom understand Microsoft’s binary formats very deeply. I can say from personal experience that several of the products that implement or will implement OpenDocument can open old Office documents quite well, and they could not do that without first understanding the binary format well. So it’s reasonable to presume that the OpenDocument specification that they created does a good job with the important information in those binary files. Even Microsoft’s Office cannot always open older Microsoft Office files! Massive independent review is how you assure that a specification is good for its purpose. OpenDocument has had it; Microsoft’s XML format has not. The claim that OpenDocument will be completely unable to handle the documents already in existence seems unfounded.


(v) the policy would prohibit impacted agencies of the Commonwealth from taking advantage of innovations and solutions from a multitude of technology vendors, including vendors whose technologies are now widely deployed throughout the Commonwealth, thereby denying these agencies the benefits of future technological innovations,

which mostly maps to:

4. A preference for the OpenDocument format commits the Commonwealth to a single specific technology choice, which contravenes well-established public sector procurement practice, as well as various Commonwealth statutes and regulations.

Microsoft clearly states here that it’s important to have technologies from a multitude of technology suppliers, and I agree. So, does OpenDocument allow multiple suppliers, or one supplier? Are they really all one technology underneath? And will innovation be helped, or harmed, by having a standard? Let’s examine each question in turn.

Will anyone implement OpenDocument?

OpenDocument was specifically designed and licensed to allow anyone to implement it. there are a whole lot of implementations already, and in fact, a whole lot of governments already use them. The OASIS OpenDocument datasheet reports that the following organizations are already adopting applications that support OpenDocument: Singapore’s Ministry of Defense, France’s Ministry of Finance and its Ministry of Economy, Finance and Industry, Brazil’s Ministry of Health, the City of Munich, Germany, UK’s Bristol City Council, and the City of Vienna in Austria. According to the Wikipedia, as of Sep 26, 2005, the following products already implement OpenDocument: Abiword, docvert, eZ publish 3.6, IBM Workplace, Knomos case management, KOffice, OpenOffice.org, Scribus 1.2.2, StarOffice, and TextMaker. Corel’ is known to be working on an implementation of OpenDocument for Word Perfect (though their situation is more complex right now, as noted below). Let’s discuss a few of them.

OpenOffice.org version 2 reads and writes OpenDocument, and its older version 1.1.5 reads OpenDocument just fine. I often use OpenOffice.org myself, and it works just fine; Eric Kriss reports the same experience. It’s very capable, and it’s free -- you can download it from the OpenOffice.org website or install it from CD-ROMs such as the TheOpenCD.org CD-ROM. Sam Hiser points out that “OpenOffice.org and StarOffice with OpenDocument’s precursor [have between them] more than 25 million users around the world in more than 75 different language versions of the software. This means that OpenDocument has close to the magnitude of the number of users as any of the several MS Office legacy formats, since the MS Office user-base is fragmented across the new version and several older versions of that discontinuous series of products.” A CSC study determined that in 2004, 14% of the large enterprise office systems market are using OpenOffice.org, with over 16 million downloads and countless CD installations. While many people still don’t know about OpenOffice.org, Microsoft sure does. Microsoft’s Form 10-K report ending June 2005 notes that OpenOffice.org’s competition is a significant risk factor for them; in fact, it’s part of the very first risk they identify.

Sun’s StarOffice is a derivative of OpenOffice.org. On Sep. 27, 2005, Sun’s StarOffice 8 was commercially released, and it supports OpenDocument. In fact, in a previous version of this article I wrote that “people widely expect StarOffice to implement OpenDocument soon” -- I didn’t even have a chance to finish writing this article before that prediction came true.

KDE’s KOffice has been upgraded, and it now supports OpenDocument. In fact, they implemented it first, before OpenOffice.org and StarOffice did; independent implementations like this give you lots of confidence that the specification is a good one. Today, KOffice runs only on GNU/Linux systems, not Windows systems; but its underlying Qt engine runs on Windows, and their developers could easily accelerate porting work so that they can compete on Windows too. The announcement of OpenDocument support gives them a reason to do so. Inge Wallin’s Open Letter to Alan Yates of Microsoft says that he believes KOffice will be running on Windows within a year.

But what’s interesting is not what implements it now, but what will implement it in time to be considered by Massachusetts -- in 2007.

I expect Corel to announce a Word Perfect implementation of OpenDocument at some point, though when this will occur is unclear. Corel has not made a formal announcement that Word Perfect will implement OpenDocument, but the evidence that this is likely isn’t hard to see. Corel is an original member of the OASIS Technical Committee on the Open Document Format, and Paul Langille, a senior Corel developer, is one of the original four authors of the OpenDocument specification. Corel’s letter to Massachusetts strongly supported Massachusetts’ selection of OpenDocument, saying, “Corel strongly supports the broad adoption of the open standards Massachusetts has outlined, including XML, the OASIS Open Document Format and PDF.... Corel remains committed to working alongside OASIS and other technology vendors to ensure the continued evolution of the ODF standard and the adoption of open standards industry-wide.” Corel confirmed this strong endorsement of OpenDocument on October 25, 2005. Many find it improbable that Corel would invest so much effort, and say that they will work to ensure adoption, without implementing the standard themselves. After all, they’d be locking themselves out of that the market, based on a letter that they wrote themselves! More likely, Corel has been waiting to see if anyone would want a product with OpenDocument.

No supplier wants to waste resources -- they want to know what their customers want. Corel need a financial reason to finish adding OpenDocument support to Word Perfect... and this announcement gives them that incentive. Massachusetts’ announcement is just what they’ve been looking for... confirmation that there are customers who want (yea, demand) it. Corel’s corporate statements have recently become rather confusing, with one spokesperson saying they don’t have any specific plans to release a version of Word Perfect with OpenDocument support, while other spokesmen are simultaneously saying Corel strongly supports OpenDocument. But people with close insights into the company say that Corel is already implementing OpenDocument. Steven J. Vaughan-Nichols of eWeek says without caveats that Corel is implementing OpenDocument. IBM mentioned at the Sep. 16, 2005 “Town Hall” meeting (at 01:27) that Corel and IBM were both implementing OpenDocument, and that they routinely talked with each other while they were implementing OpenDocument in their respective products. After investing time to develop it, and more time to implement it, it seems likely to me that they will release it.

Microsoft states on page 8 that “the policy clearly calls for Corel and Microsoft products to be phased out...” which is not true. The policy requires that all products that perform a certain job use the same standard for storing data. Suppliers who meet the standard can be used. Microsoft and Corel products don’t need to be phased out; they just need to implement the standard. Massachusetts has given every indication that they would like everyone, including Microsoft, to implement the standard.

Microsoft’s failure to commit to meeting the standard important, because they’re currently the dominant supplier. But office suites are now a commodity; there are lots of other products that do the job, if only you could share data between them. With OpenDocument, you can use any office suite... and suddenly the dominance of one vendor is not as relevant. And as Massachusetts notes, there are various ways that people can continue to use Microsoft Office yet store data in OpenDocument, which can act as a useful transition approach if Microsoft chooses to not implement OpenDocument.

In fact, I believe that it’s very likely that Microsoft will eventually implement OpenDocument in Microsoft Office. I’ll explain why at the end of this article; it’s my punchline.

Oh, one funny thing: If you go to Massachusetts’ website, they’ve already posted Microsoft’s comments, in OpenDocument format. The notion that there is no software to manage this is nonsense; the software’s already available. Anyone who doesn’t have such software can go download OpenOffice.org, which is free, very capable, and available for just about any operating system in use. By waiting until 2007, Massachusetts can work with suppliers and do all sorts of evaluations and tests to lower any residual risk.

Are all implementations actually the same program?

Microsoft’s letter, in their supporting text for point 4, says, “The draft policy identifies four products that support the OpenDocument format: Sun’s StarOffice, OpenOffice.org, KOffice, and IBM Workplace. In reality, these products are slight variations of the same StarOffice code base, which Sun acquired from a German company in 1999. The different names are little more than unique brands applied by the suppliers to the various flavors of the code base that they have developed. In essence, a commitment to the OpenDocument format is a commitment to a single product or technology.”

This is wrong; Microsoft just didn’t do a good job fact-checking here. It is true that StarOffice and OpenOffice.org share almost all the same code. This is a fact that is clearly explained by both StarOffice and OpenOffice.org, in numerous locations, and not hidden anywhere. Nor is Massachusetts likely to consider this a disadvantage, for it still gives them two choices that share many technologies. OpenOffice.org is available at no cost, for the price-sensitive (and is an easy way to quickly get a reader and writer); StarOffice costs money, for which you get some additional capabilities and support.

But KOffice is demonstrably a completely independent implementation. I asked David Faure (who is sponsored by Trolltech and very familiar with KOffice) how much code KOffice and OpenOffice.org shared, and his answer was, “None at all!” He also shared with me a lot of technical details that prove, quite clearly, that they are completely independent implementations. Here are some of those details: “The KOffice support for OpenDocument is brand new code, based on Qt’s QDom parser for loading, my own XML writer for saving, and many KOffice classes that were written in order to support the OpenDocument file format (style collections, style stack, helper for writing out styles, manifest writer, list handling, variables etc.) plus of course the application-specific code.” For proof, you can go see the KOffice code, e.g., ZIP implementation, XML writer, Class for generating automatic styles on saving, Class for loading settings, and the Class for loading styles. Inge Wallin’s Open Letter to Alan Yates of Microsoft makes the same point, noting clearly that KOffice is a completely independent implementation.

And that is perfect. The best way to be sure that a specification is really understood is to have multiple independent implementations, just as we have here.

Perhaps when they say “technology” they mean “open source software.” But that can’t be true; StarOffice is proprietary, while OpenOffice.org is not, so their earlier statement makes even less sense.

What about innovation?

Microsoft claims that standards somehow prevent innovation ( “denying... benefits of future technological innovations” ). This is nonsense, even bizarre. Standards make innovation possible! This isn’t just me; IBM stated at the Sep. 16, 2005 town hall meeting that common standards like this enable, not inhibit, innovation. Let’s look at why:

  1. Innovators need standards to build on. If developers had to reinvent the wheel every time they made an innovation, then nothing could get accomplished. Standards for data formats let developers commoditize the non-innovative stuff, and concentrate on the innovations. We would never have had a World Wide Web if we had not first had an Internet; innovations build on other innovations.
  2. Customers need standards so they can transition to the new innovation. Without standards, customers cannot switch to whatever products provide the innovations important to them when they become available. If a customer is into one supplier (e.g., through a proprietary format), then they cannot switch to another supplier when they have an innovation.
  3. Standards create an environment where innovation is rewarded. If a customer is locked in to one product due to the lack of standards, their supplier has no reason to really innovate. After all, customers would buy the product and its upgrades whether or not the supplier was innovative.

If all office suite products used the same standard format, then it’s reasonable to presume that this would greatly increase innovation. Innovation speeds up when there’s a vendor-neutral standard and competing suppliers who implement it; just look at past examples like Betamax or TCP/IP. If Microsoft innovates while using a standard neutral format, then everyone who wants that innovation can switch to or upgrade to their product, no matter what they used before.

Innovation is often oversold, anyway. I love great innovations, I support research, and I believe supporting innovation is a good idea; I’ve even written an article about The Most Important Software Innovations. But people get software products to solve problems, not to get innovation for innovation’s sake. Innovation “just to be different” is often a cause of problems, not a solution. For example, Joel Spolsky’s “Joel on Software: User Interface Design for Programmers” notes that “consistency is a fundamental principle of good [user interface] design... [but] there is a dark force out there that fights against consistency, and that is the natural tendency of designers and programmers to be creative. Now, I hate to be the one to tell you “don’t be creative,” but unfortunately, to make a user interface easy to use, you are going to have to channel your creativity into some other area.” That’s particularly true for mature areas like office suites, where millions already have expectations of how things should work. Many argue that Microsoft is not innovative, but non-innovative companies can still be very successful, as long as they solve customer needs. Let’s face it; few people use Microsoft Office or any other office suite because it’s “innovative.” Many people choose to buy Microsoft Office because they want to read and write files in Microsoft’s proprietary binary formats, not because of any special feature in the product that’s unique to it. Many people who have an office suite do not bother to upgrade for many, many years, because they just don’t need the capabilities of later versions. Office suites are an extremely mature market, and few people use even a significant fraction of today’s office suite capabilities.

Other comments

In this section, Microsoft states that they spent over “five years building its XML capabilities into its current generation products.” They seem to be saying that everyone should use their formats, because they invested a lot of money in their proprietary formats. That’s not a good reason to use something; just because a lot of money was invested in it does not make it appropriate for someone else to use. And more importantly, the European Union, Massachusetts, and others have been repeatedly telling Microsoft that they do not want a proprietary format and specifically instructed Microsoft to not make them proprietary. The whole point of XML is to enable interoperability. Microsoft’s proprietary XML format shuns neutral standards bodies and includes a license that prevents customers from manipulating their own data in arbitrary ways. These limitations subvert the whole point of using XML. Microsoft chose to invest lots of time in something the customers said they didn’t want. Now, as it’s getting ready, the customers say they still don’t want it. Suppliers that reject their customer’s requirements as irrelevant often waste their investment dollars; this may become yet another example.

Microsoft notes that the Commonwealth has statutes requiring the “stimulation of competition.” Good point. They then amazingly complain that a common file format would reduce competition. Nonsense. By requiring a single neutral standard everyone can meet, a level playing field is created that everyone can implement, stimulating competition.


(vi) the proposal appears both inconsistent and discriminatory in that it approves use of one “proprietary” document format as an alternative to the OpenDocument format, while excluding others, and

This is referring to Portable Document Format (PDF); more details of Microsoft’s argument are in its details under “reason 5.” Microsoft argues that the conditions for its format are no different than PDF... but it’s not so. It’s true that Microsoft has posted their specifications for free download, just like Adobe.

But there the similarities end. PDF has been reviewed by neutral standards bodies, multiple times, and there is every evidence that future evolution of it will involve other parties in a neutral setting too. In additional, PDF has been implemented by a wide variety of different implementations, from proprietary licenses to the GPL, demonstrating that its license does not constrain competition or use. In short, there’s lots of evidence that there is open competition in PDF implementations. Let’s look at the evidence; it turns out there’s a vast array of it.

Massachusetts wants the specification to be reviewed by an independent standards body; this kind of review reveals biases for a platform or supplier. Wikipedia briefly explains that this has already occurred with PDF, and other information sources show this in spades.

For various reasons, the international standards community has decided to standardize PDF by creating several different standards, each of which is a subset of Adobe’s specification and tailored for a different reason. Adobe’s letter to Massachusetts briefly mentioned these different standards. Anyway, the main groups are:

  • PDF/Archive (also called PDF/A) has just been approved by ISO as the standard ISO 19005-1. This specification is specifically designed for document archives, and is probably the most relevant standard for most of Massachusetts’ intended uses.
  • PDF/X is for reliable exchange of press-ready, high-end graphic information (like color advertisements), and it’s defined by the ISO 15930 series (more in a moment).
  • PDF/UA is ongoing work to define accessible characteristics for PDF (so disabled people can access PDF files).
  • PDF/E is ongoing work to exchange engineering data.
Both PDF/UA and PDF/E are currently in the ISO working group phase, under AIIM sponsorship.

PDF/X has actually been around for a while now as an ISO standard, but some of its aspects seem to confuse people so I’ll try to explain a little further. The PDF/X FAQ explains PDF/X in more detail. The PDF/X format is a subset of PDF designed specifically for reliable prepress data interchange. There are three levels of PDF/X, with increasing capability but increasing requirements on implementations; normally the level is included in the name to avoid confusion . One oddity is that the levels of PDF/X (from the point of view of increasing functionality) are numbered 1, then 3, then 2 (not 1,2,3); this historical oddity was caused by the development process of CGATS and ISO. People who choose option 2 have be technically adept anyway, so this odd numbering (while a little unfortunate) doesn’t cause problems in practice. Anyway, here are the levels of PDF/X:

  1. PDF/X-1a. The first PDF/X-1a standard went through the standardization process years ago, and is now published as ISO standard 15930-1:2001. I’ll let the FAQ explain its purpose: “The PDF/X-1a standard addresses blind exchanges where all files should be delivered in [the color system] CMYK (and/or spot colors), with no RGB or device independent (color-managed) data. This is a common requirement in many areas around the world and in many print sectors.”
  2. PDF/X-3. PDF/X-3 is a superset of PDF/X-1a; it adds the ability to contain color managed data. Sometimes the CMYK color system isn’t good enough for print work; “others are better served by transferring [color] data in other spaces, such as CIELab or RGB with a profile attached.” It’s important to note that PDF/X-3 is a superset of PDF/X-1a -- a PDF/X-1a file meets all of the requirements of PDF/X-3 except for the label that says “I’m a PDF/X-3 file.” ISO recommends that all tools designed to read PDF/X-3 should also be able to read PDF/X-1a files, since there’s no extra work in doing so and this makes PDF/X-1a files extremely portable. The first PDF/X-3 standard (aka PDF/X-3:2002) is published as ISO standard 15930-3:2002.
  3. PDF/X-2. PDF/X-2 is a superset of PDF/X-3. PDF/X-1a and PDF/X-3 define file formats for blind exchange. Sometimes you don’t need blind exchange, and instead you’d like additional control over file formatting. PDF/X-2 (PDF/X-2:2003) enables an “OPI-like” workflow, and supports data exchanges where there is more discussion between the supplier and receiver of the file. In this case there is a single “master” file that refers to others. I looked at ISO’s list of related standards and found that PDF/X-2 has been published in 2003 as ISO 15930-5:2003.

But what about the future, and modifications to the format? It’s true that Adobe has its own specification, posted on its own website, and that Adobe could update its own specification. But ISO has been updating things as well; there seems to be excellent evidence that ISO is free to take what Adobe does and accept or reject pieces, or even add new things. In practice, Adobe and the other participants seem to be acting collegially, but that’s a good thing. Their process is not terribly surprising: Adobe creates a modified specification as a proposal, and then the independent body examines it and creates a vetted version of the specification. This lets Adobe create new ideas, but it also lets people around the world refer by name to a specific set that is internationally vetted and approved. For example, the PDF/Archive committee noted that, since they had completed their task of creating ISO 19005-1, they had “begun work on ISO 19005-2 based on PDF Reference 1.6” and were asking for anyone interested to join. Clearly there seems to be no problem in working with neutral bodies into the future to manage PDF. I’ve found no evidence that the neutral standards bodies are forbidden from updating their specifications independently, or adding completely different capabilities, if they found a strong need to. If Adobe decided to abandon PDF, PDF could still go on.

Are there licenses that forbid arbitrary implementation of PDF? No, the licenses allow anyone to implement it. You can build partial implementations, for example; you have to use a different name (other than PDF) when referring to partial implementations, but you can do it. Permitting partial implementation is critical; for many applications a partial implementation is all you need (and can afford), and many projects have to implement incrementally and thus must be able to have partial implementations. There do not appear to be any traps or snares in the license that would prevent someone from implementing PDF.

Massachusetts wants anyone to be able to implement the specifications they choose. In fact, I think that’s the best test -- are there multiple implementations, under a variety of licenses? Wikipedia reports that PDF readers include the Adobe Reader by Adobe Systems (proprietary, no cost). There are also open source readers, including Xpdf for POSIX-like systems with the X Window System, derivatives of Xpdf (including KPDF, GPdf, and Evince), and Ghostscript. Ghostscript has historically had both a proprietary and GPL’ed branch, so there has been an alternative proprietary implementation too. And there are multiple writers. Adobe’s Distiller is a wildly popular writer, but Ghostscript includes ps2pdf (historically available under both a proprietary license and the GPL), and OpenOffice.org includes a PDF writer built into the office suite. PDF readers are available for just about every platform out there, including Microsoft Windows, Apple MacOS, Linux, Unix, and PalmOS.

So, in short, PDF is nothing like Microsoft’s XML format. PDF has already undergone multiple reviews through neutral international standards bodies. Even more importantly, it already has multiple independent implementations on a vast range of platforms, using both typical proprietary licenses (like Adobe Distiller’s and Acrobat’s) and typical open source software licenses (like the GNU GPL). Remember, the reason to have the review is to be sure that arbitrary implementations are possible, and that no platform-specific encodings are hidden in there. The ability to have multiple implementations on multiple platforms has been proven in spades.

Microsoft cannot say this about their XML format. There’s been no independent international review. There are no multiple implementations under a variety of licenses, and legal analysis says that competitors are severely constrained because many common licenses used in industry are forbidden by the excessively restrictive legal conditions. Competition is constrained by their license and the lack of neutral forums. Case closed.


(vii) there are less costly, less limiting, non-preferential policy options to achieve the proposed policy’s stated goals.

which sort of maps to this (except for the text about PDF):

5. The current proposal constitutes a significant and unjustified departure from the Commonwealth’s prior policy, adopted earlier this year, under which de facto format standards, such as Microsoft’s Office XML Reference Schemas, could also qualify as “Open formats”.

The Commonwealth seems to respectfully disagree that there are better options. I think the Commonwealth made the right decision here, and it can easily justify its decision. In short, Massachusetts believes it should own the data it creates, and be allowed to choose any supplier at any time to manipulate its data. OpenDocument seems to be the only choice available to it for editable office documents, given the Commonwealth’s fundamental need to own its own data.

I already addressed most of this earlier. Microsoft’s letter points to lots of irrelevant details, while avoiding the main point: Anyone can implement OpenDocument, without changing their business model, licensing structure, and so on. Microsoft’s XML format has never been submitted to any independent body, the format stays in control of a single party (putting all competitors at a permanent disadvantage), and its licensing conditions make it impossible to select arbitrary alternatives.


The front part of the letter continues this way:

... In particular, we respectfully recommend that the ANF: 1) reinstitute its prior definition of “open format” that properly allowed for agency purchase of products based on openly licensed and widely deployed de facto standards as an equally effective means of fostering data interoperability,

This proposal makes little sense. Massachusetts already examined this, and found out that Microsoft’s XML format prevented competition. This “open” seems to means “open unless you’re a competitor I don’t like,” which is not really open. Also “de facto” is odd in this context, because this entire discussion is about the XML-based formats. Very few people use Microsoft Office 2003’s XML formats, and Microsoft Office 12 isn’t even due out til next year (and it will be significantly different). Neither are de facto standards, in any sense. The binary formats are a “de facto” standard, sure, but everyone wants to move on to XML; Massachusetts wants the processing XML supports, and Microsoft is abandoning its binary formats too.

And that’s a deal-breaker. Massachusetts is trying to establish a new and far more efficient process for managing its documents. To do it, they need a format that anyone can manipulate. And Microsoft doesn’t want to provide one. I understand Microsoft’s business reasons for making that decision, but it is fairly obvious why Massachusetts would not find that decision acceptable for its purposes.

2) reinstitute its prior conclusion that Microsoft’s Office XML Reference Schemas qualify as open formats under the Commonwealth’s policy (under this approach, the OpenDocument and PDF formats could also remain as viable alternatives), and

But Massachusetts doesn’t want two office formats. Who does? It wants one, and only one. There’s only one specification that is an international standard. Only one that can be implemented by all suppliers. Only one that already has commitments from many suppliers. Yes, the current market leader doesn’t implement it. But WordStar and Lotus 1-2-3 were once market leaders; when technology changed, they lost that perch. Assuming that the market leader will always be around is an extremely dangerous assumption in technology, especially when your records for all time are on the line. Here, a new technology is finally ready -- XML -- which promises the ability to share and manipulate data in novel ways. But XML’s promises only work if anyone can manipulate the data.

Note that they imply here that somehow PDF doesn’t meet the criteria. It does, quite nicely, as I explained in concern (vi). For example, PDF has both proprietary and GPL’ed implementations. Microsoft cannot say the same.

3) incorporate a process into the ETRM that makes clear how additional formats or standards may be added to the Commonwealth’s “accepted” list as developments and innovations arise in the future.

They already have such a process; it’s embedded in the definition of open format. The problem is that Microsoft’s format does not meet the criteria, not that there’s no way to add new developments.

David Berlind’s Sep. 22, 2005 article notes that at the “Town Meeting,” Massachusetts once again explained their requirements:

  1. It must have no or absolutely minimal legal restrictions attached to it.
  2. It must be published and subject to peer review
  3. It must be subject to joint stewardship
They were open to reconsidering Microsoft’s format, if they would meet their requirements, but that window of time seems to dwindling away. Massachusetts can’t wait forever for Microsoft to decide to meet these requirements, they have problems that need solving.

In the next part of the letter, Microsoft asks for more delay, after two years of discussion. It seems to me that’s enough. There’s absolutely no evidence that any new information is likely to surface, and Microsoft has had plenty of time to make licensing changes.

They also demand inner details of Massachusetts’ cost and time estimates, which they have no right to know. I’m sure Massachusetts has some estimated costs, and they have stated in public forums some simple figures that show that they’ve considered the issue. But they’d be foolish to announce too many details in public right now. If you walked into a car dealership and said, “I’m planning to pay up to $50,000 for a car,” you can be sure that the cars you see won’t cost $20,000. Massachusetts can make some public observations, but I think they’ll be coy when pressed for details. Governments try to identify their requirements, and then let the market bid to meet them. By having competing bidders, governments tend to get better functionality at lower cost.


The letter goes on with details, but I’ve dealt with their highlights above. If you want more information, a good place to start is the Wikipedia article on OpenDocument. You can’t get more authoritative, in terms of Massachusetts’ responses, than Massachusetts’ FAQ on OpenDocument. One of the simplest and straightforward articles about this topic is ZDNet UK’s article “Microsoft must drop its Office politics” (September 2, 2005). Much was discussed at a “Town Hall meeting” on Sep. 16, 2005. Groklaw has an article referencing a number of comments on Massachusetts’ decision.

Jason Brooks’ article “Massachusetts vs. Microsoft?” (eWeek, Sep 9, 2005) opines, similarly to me, that Massachusetts’ “Plan to move toward open formats isn’t foolhardy if the state thinks it can better serve taxpayers by escaping proprietary lock,” and then says, “I don’t blame Microsoft for doing what it believes is best for its business -- but let’s not blame Massachusetts policy makers for doing what they believe is best for their state, either... Massachusetts has chosen to shift its default file formats to those for which any developer or firm has an equal chance of building an equally good application to create and consume these documents, thereby ensuring choice and flexibility for itself and for its residents. Where’s the controversy or zealotry here?”

In the end, Microsoft has written a long letter, but I think it fails to make its case. This is in some sense a surprising development. Microsoft has the largest market share, and if it had performed the reasonable actions its customers asked for, it should have been able to easily meet their requirements and just carry on. Europe told Microsoft what to do back in 2004, including getting involved with OpenDocument. Massachusetts has been having a public dialogue since January 2005, and apparently has been having private discussions even before that. Microsoft’s license clearly limits how users’ data can be used, yet supporting arbitrary processing is the whole point of XML. Microsoft gambled that no one would be willing to choose a standard that they didn’t already implement. The gamble failed.

No doubt Microsoft will try all sorts of tactics to overturn this. I think they’re unlikely to overturn this, though it’s possible.

But whether or not they manage to overturn Massachusetts’ decision, the dominoes are beginning to drop in lots of other places. For example, BECTA (British Education Communication Technology Agency), is the UK agency in charge of defining IT policy for all schools in the United Kingdom. They recently published a comprehensive document describing the IT policy for infrastructure in schools, including a definition of the standards for infrastructure for all the schools in the country. In this new policy (pages 25-26), schools are required to use software that saves files in open formats. Look at their list:

Text documentsOpenDocument (.odt), plain text, RTF
SpreadsheetsOpenDocument (.ods), CSV
DatabasesOpenDocument (.odb), CSV
PresentationsOpenDocument (.odp), HTML, SMIL
OpenDocument is notably present; notably absent are Microsoft’s binary formats (such as .doc, .xls, .mdb, and .ppt) and both of Microsoft’s proprietary XML formats. This is no accident; BECTA explains saying: “Any office application used by institutions must be able to be saved to (and so viewed by others) using a commonly agreed format that ensures an institution is not locked into using specific software. The main aim is for all office based applications to provide functionality to meet the specifications described here (whether licensed software, open source or unlicensed freeware) and thus many application providers could supply the educational institution ICT market.” Groklaw includes an article discussing BECTA’s decision to use OpenDocument in more detail.

But that’s just a drop in the bucket compared to what is likely to come next.


What’s Next? Europe.

In my opinion, the next big move is Europe, and there the choices are starker. OpenDocument should sail through the ISO fast track process; OASIS is one of the very few organizations who have been granted the right to submit specifications directly to ISO, and OpenDocument meets all the criteria for a quick “yes” vote in ISO. The European Union has already said that they plan to pick whatever specification becomes an international standard; OASIS is probably sufficient for some, but the Europeans will probably wait until ISO accepts OpenDocument before making a declaration. There is only one real option: OpenDocument. In contrast, Microsoft never even chose to compete. Even if Microsoft removed all the license restrictions and started a standardization process in a neutral body, it would take years for a neutral group to go through the specification (to find and fix problems like constructs not neutral to a supplier). What’s more, there is no evidence that suppliers are interested in doing all of it again; there’s already a widely-vetted standard for this purpose.

The European Union creates lots of documents, and is keenly interested in having an exchange standard for them. Their calculus is really obvious: do we choose the one and only international standard, one with multiple implementations? Or do we break our normal rules, abandon our announced plan of 2004 to crown the international standard, support a specification that many of our domestic organizations won’t be allowed to implement, and entrap our data in a specification controlled by a foreign company... all to enrich a foreign company? I think some have a visceral dislike for Microsoft, and Microsoft has been in Europe’s courts quite often recently; neither will help Microsoft’s case. But many Europeans will be dispassionate decision-makers... and end up with the same conclusions. They’ll worry about transition costs, of course, just like Massachusetts has. But transition costs are a one-time cost, while governments plan to be in business, handling documents from their past, for many centuries. As noted above, the transition costs appear to be surprisingly small, and the only long-term alternatives are even more expensive. They primarily use Microsoft Office’s binary format, not Microsoft’s XML format... so now, while they do not have pre-existing commitments to any XML format, is the time to make that decision. Making a decision later will just raise the transition costs later, and keep them from exploiting XML’s potential while they wait. That decision will certainly involve lots of back-room politics, as these things do. But the likely outcome is already clear: OpenDocument.

If Massachusetts had not chosen to use OpenDocument, I think the European decision would be the same. But I suspect Massachusetts’ decision will embolden Europeans and many other governments, enabling them to decide even faster than they would have otherwise. It’s easier to make a decision stick if you’re not the first one to make that decision.

But even worse for Microsoft’s hopes to get everyone to use its proprietary XML format, the information Massachusetts has gathered and made public will make deciding to use OpenDocument and not Microsoft’s format incredibly easy for all other governments. Governments worldwide all have the same issues that led Massachusetts to this decision, and Massachusetts has done a massive amount of work in analyzing the alternatives. I think David Berlind’s September 26, 2005 post, “Did Microsoft send the wrong guy to Massachusetts’ ODF hearing?”, is particularly insightful on this point:

“Microsoft may see Massachusetts as just one state with 80,000 employees across 173 separate agencies along with a handful of contractors that it can let go. But, if you take a step back to look at the volume of diligence that the Commonwealth undertook before it made its decision final -- most all of which is public [--] you can’t help but wonder whether Massachusetts just created an online wizard that will make it easier and less expensive for other governments to embark on similar projects. Lest Microsoft think other governments aren’t paying attention, it may want to consider who launched the Government Open Code Collaborative (Massachusetts, Rhode Island, Pennsylvania, Utah, Kansas, Missouri, West Virginia, and the cities of Gloucester [MA], Worcester [MA], and Newport News [VA]) in June of 2004 and how the current list of GOCC members and observers has blossomed well beyond the founding group. Not only is the GOCC clearly keeping an eye on things in Massachusetts... (see this call for participation from the GOCC), judging by a presentation given by Massachusetts Information Technology Division (ITD) policy and architecture director Claudia Boldman, the Commonwealth considers its involvement in the GOCC an integral part of its overall open standards initiative (not to mention that the presentation was given at the Conference sur les Logiciels Libreset les Administrations Publiques in Quebec, Canada and one can only imagine who else was paying attention!). Take the digital ecosystem that lives around Massachusetts’ 173 agencies and multiply that times some number of other states. Throw in some cities and counties and then a dash or two of corporations that see a reflection of themselves in what those governments are doing, and suddenly, instead of a defector, Microsoft has an exodus on its hands.”

Lots of other states are seriously looking at these decisions (see the Sep. 16, 2005 “Town Hall” meeting, 01:21). And many other countries are discussing this, too; Harvard recently hosted a meeting of many countries (ranging from Canada to China to many others) discussing these very issues.

Is the data good enough to convince others to do the same thing? Look how David Berlind views the quality of Massachusetts’ data:

“The clarity, the mission, the thoroughness and the goals under which the Massachusetts officials were operating were the stuff that IT case studies should be made of. This is a group of people that in no uncertain terms (1) understood exactly what their employer’s needs were (for example, preserving the state’s sovereignty), (2) figured out how best to meet those needs from a strategic IT perspective, (3) anticipated the pain points and costs and (4) very clearly articulated and communicated the plan and the progress, soliciting feedback from all interested parties along the way.”

What should really concern Microsoft is that Europe is usually a lot more serious about standards than the U.S. is. Europe has so many interoperation problems (due to multiple sovereign states, multiple languages, and so on) that they treat any standard as a Godsend. Any improvement in interoperability due to a standard is a real help, given the fundamental challenges they face. As I noted above, it’s almost certain that OpenDocument will become the EU standard for governments; it’s a good standard, supported by lots of suppliers, and there are no alternative international standards. The EU uses supplier-controlled specifications when there are no alternatives, but once there’s an international standard, controlled by a neutral body and with at least one implementation, the EU is usually quite purposeful (some might say ruthless) in abandoning nonstandard products and switching to the international standard.

Even back in 2004, the EU said that “Where by choice or circumstance only a single revisable document format can be used[,] this should be for a format around which there is industry consensus, as demonstrated by the format’s adoption as a standard.” “Standard” here only means a “formal standard vetted by an truly vendor-independent body” -- they are so serious about formal standards that European governments have rules forbidding them from even using the word “standard” for a supplier-proprietary specification. The “choice or circumstance” language looks more flexible, but supporting multiple formats raises costs, and the obvious alternative prevents the use of arbitrary suppliers. It’s not particularly likely that this “choice” will survive in the long term. At this point, it’s extremely likely that OpenDocument will become the required document standard for all EU governments. And it won’t be with a wink, while accepting an alternative... their agencies typically implement the standards once picked, all the way through.

The real question is, will Europe allow the sale of Microsoft Office 12 inside Europe if Microsoft fails to implement OpenDocument? It’s not unusual to forbid the sale of products that don’t meet international standards, after all. Just try selling many other products in Europe that fail to meet various international standards; their rules are generally very reasonable and clearly defined in publicly accessible documents, but they’re serious about them. And Europe is already dictating terms involving Microsoft Windows to allow competitors to compete; such a move would not be out of character. If consumers routinely used Microsoft’s XML format, and then sent those files to the government, the government would not be free to choose which supplier to use in reading or updating those files. At the very least, governments could require that only OpenDocument be allowed for documents sent to them. But it’s sometimes hard to predict when a document will go to the government. The obvious way to quash that problem is to mandate the OpenDocument format, Europe-wide, for all documents, with some sort of phased-in timing to deal with the old Microsoft binary formats. That’s certainly interesting to speculate, and not outside the realm of possibility. Even if sales are permitted (I think that is likely), and if the national governments require OpenDocument for their own use (which I also think is likely), and Microsoft refuses to implement OpenDocument, then all the EU governments will abandon Microsoft Office... and a lot of their citizenry will quickly follow.

It’s hard to imagine Microsoft continuing to not implement the international standard at that point. The risk of losing the entire EU market would quickly lead to a worldwide collapse of its market share, and that just isn’t worth it. They’ll probably start with a quick rushed implementation, which will be really bad (inevitable with rushed implementations), and then later come out with a useful version. The longer they wait to implement OpenDocument, the greater the risk that they will lose significant market share. It’s speculation, but it’s reasoned speculation. Others have come to the same conclusion; Steven J. Vaughan-Nichols says, “OpenDocument is here to stay. Microsoft can either gracefully accept and support it, or fight a losing battle against it.” David Berlind of ZDNet says, “My hunch is that there are plenty of government agencies, both domestic and foreign watching this one and that, in this game of chicken, Microsoft will not win.” Lots of people are appealing to Microsoft to support OpenDocument now; Massachusetts certainly is, and other organizations such as ZDNet are imploring the same. OpenDocument Fellowship has been running a petition for Microsoft to implement OpenDocument, and they already have signatures from thousands of people representing legions of computers (as of October 29, they had 5780 signatures representing 171807 computers). If that’s so, the question will be, will they do it fast enough before they start to lose significant market share because of this lack? There are lots of smart people at Microsoft. It’s my hope that they change course quickly, implement OpenDocument well, and not suffer the likely consequences of ignoring standards when customers and competitors switch to them.

We’ve been here before, ten years ago. Microsoft owned the client market in 1995, yet they were on the verge of losing everything. Why? They had completely committed themselves to their own proprietary networking protocol. But people didn’t want a proprietary protocol; they wanted the freedom of the TCP/IP and World Wide Web standards, which could be implemented by anyone and thus were a hotbed of innovation. In 1995, Microsoft finally abandoned their proprietary network format and embraced the industry standards that anyone could implement (and that everyone else did). This story of a giant’s proprietary standard getting felled by a less proprietary standard happens over and over again. The World Wide Web, TCP/IP, ASCII, Betamax -- they are all just a few examples of the same basic story. This isn’t about Massachusetts “rejecting” Microsoft; I expect they’ll continue to work with Microsoft, and that they will continue to use many Microsoft products. And I expect that Microsoft Office will eventually implement OpenDocument directly, because the market is already requiring it.

--- David A. Wheeler

This article is released under the Creative Commons Attribution-NonCommercial 2.0 license.


  View Printable Version


Groklaw © Copyright 2003-2013 Pamela Jones.
All trademarks and copyrights on this page are owned by their respective owners.
Comments are owned by the individual posters.

PJ's articles are licensed under a Creative Commons License. ( Details )