I asked Daniel Carrera, an OpenOffice.org volunteer, if he'd please explain to us the OpenDocument format. How does a format get chosen? And is OpenDocument on the list when governments like the State of Massachusetts make up such lists of acceptable formats for governmental use? If not, what can be done to change that? He graciously agreed. Because we are all concerned about proprietary formats and standards, and more and more governments are adopting policies requiring open standards, it's a very important subject.
Daniel calls OpenDocument "our best chance to fight vendor lock-in associated with proprietary formats." It's currently being considered by the EU Commission as a candidate for an official format. Note that you can also download
this article in various formats, including, naturally, OpenDocument, from links at the end of the article. In some browsers, you may need to Save As, instead of just clicking on the links.
Daniel Carrera is a mathematics PhD student at the University of Maryland,
as well as an active OpenOffice.org volunteer. He serves in the OOo
community council as the "community representative". Daniel also
facilitates the OOoAuthors project, where most English documentation is
developed. He will be speaking at the OOoRegiCon in San Diego on Feb 9, the day before and leading into the
" Desktop Summit 2005", a conference about the present and future of open source and desktop Linux technologies being held in San Diego, CA February 9-11.
He is speaking opening day on the subject, "INREACH, Building an Active Community."
The complete list of speakers is here, and I must say, it sounds like fun. Mitch Kapor and Doc Searls are two of the keynote speakers at the Desktop conference. Sun's Simon Phipps kicks off OOoRegicon, and I've heard he is an excellent speaker. You can find out more here. It's limited seating, and the page says the price goes up after February 5.
The Future is Open: What OpenDocument Is and Why You Should Care,
~ by Daniel Carrera
What Is OpenDocument?
Quiz: Complete the sentence,"OpenDocument is ..."
(a) An open, XML-based file format.
(b) An open standard, supported by the OASIS
and ISO standards groups.
(c)The default file format for the upcoming OpenOffice.org 2.0
and KOffice 1.4.
(d) A top prospect for an official format for
the European Commission.
(e) Our best chance to fight vendor lock-in
associated with proprietary formats.
(f) All of the above.
The correct answer is (f) All of the above.
This article links to reports written over a period of several months.
To make things easier to follow (in particular, the name of the format),
I include an approximate timeline.
OpenOffice.org creates an open file format called "OpenOffice.org
The European Union commissions Valoris to report on open file formats.
The OpenOffice.org 1.0 format is submitted for OASIS standardization.
KDE and Corel join the OASIS Technical Committee and expand the format to
cover a wider range of applications.
The new OASIS format is called "Open Office XML".
OpenOffice.org and KOffice both commit to making the format their
The Valoris report is published. Microsoft and Sun respond to it. The
European Union TAC makes recommendations.
The format is submitted for ISO standardization and changes its name to
the OpenDocument format.
In order to avoid promoting an outdated name, I will always refer to the format
The European Union and the Valoris report
The key player here is the European Union's Telematics between Administrations
Committee (TAC). The TAC hired the Valoris consulting group to explore the
possibility of using Open Standards to create a more competitive marketplace.
The Valoris report was well researched and very thorough. They recommended the
adoption of an open XML format backed by a recognized open standards group.
The Valoris report is a very detailed and most compelling document describing
how open XML technologies are impacting government information systems and
It's difficult to summarize a 78-page document. I encourage you to
read it yourself [PDF].
But over all, it is well-researched and rather encouraging of open source,
OpenDocument, and EU hopes for an open marketplace.
The Valoris group closely monitored the development of the OpenDocument
format, and by November 2004 every one of the concerns and issues they raised
were directly addressed by OASIS TC. As a case in how governments and
organizations go about setting information system requirements policy, the
Valoris effort provides an unparalleled insight.
For the report, Valoris compiled an exhaustive list of existing file formats.
These were compared against a series of requirements. Only two formats performed
well, and these were selected for further analysis.
Being familiar with these criteria is important for us. It teaches us what a
government requires before it will adopt an open XML format. I divide these into
Open: At this stage of analysis Valoris
only looked for openness in the sense that a public,
royalty-free specification is available. For example, PDF
and MS XML both met this requirement.
Non-binary: Binary formats get in the way
of neutrality. A format that depends on Windows components
will be hard to support on GNU/Linux and Mac OS.
Cross-platform: An obvious requirement for
Preserve format fidelity: This refers to
both presentation and structure. For many applications,
format fidelity is an absolute imperative.
Modifiable: This excludes formats like
Support current word processor features:
An open format is no use if it can't represent your data.
This list included Unicode support, bi-directional
(Hebrew), and scripting, among others. Abiword and KOffice
failed this requirement.
Support emerging requirements: Digital
signatures, access rights, version control, etc. Almost
every format failed this requirement.
This sounds odd. Why is this important? Shouldn't we pick the best format
The answer is simple: Have you heard of the ODA? I didn't think so.
In the 80's Europe made an attempt to define an open standard format called
Open Document Architecture (ODA). It received a lot of support from
European institutions including the European Commission. It was also an ECMA
standard and an ISO standard.
ODA failed miserably.
ODA had no backing from the industry. The format was complex, and the
companies preferred to support more pragmatic standards such as SGML and RTF.
Conclusion: Industry support matters. This is why the W3C never accepts a
standard unless it has at least one working implementation.
"widely adopted" does not mean "dominant". It means that there
is enough industry adoption to sustain the format. For example,
OpenDocument meets this requirement. Unlike the ODA effort,
OpenDocument enjoys the support of two office suites as well as
major players like Sun, IBM, HP, Novell and even Adobe.
No format met all the criteria perfectly well. But two formats stood out
above the rest:
OpenDocument vs MS XML
- OpenDocument - main failing:
lacked some support for emerging requirements.
- Microsoft XML - main failing:
medium fidelity, unsure about cross-platform status.
The only apparent advantage of MS XML over the OpenDocument format was
the presence of custom-designed schemas.On the other hand, OpenDocument
had several advantages:
- More open: No legal constraints, and support from OASIS.
- Reuse of existing open standards when possible (SVG, Dublin
Core, MathML, etc).
- Higher format fidelity.
- Friendly to XSLT and other XML-based tools.
Reactions To The Report
Microsoft Responds to the Valoris Report
Microsoft's response [PDF] essentially comes down to trying to
convince the reader that custom designed schemas are really
really important. This isn't surprising since it seems to be
the only card they could play.
Sun Responds to the Valoris Report
Sun's response [PDF] expresses several concerns about Microsoft's
approach to XML. I recommend reading the full response for
details, but here are a few:
The EU TAC Makes Recommendations
MS XML doesn't cover all of MS Office (for example, it
doesn't cover PowerPoint).
MS XML doesn't support some of the advanced MS Office features.
MS XML can contain binary objects that depend on MS Office and Windows
(e.g. OLE and VBA) and those lack complete documentation.
MS did not commit to make future changes to MS XML available to
the public, only the current one.
After getting responses, the EU TAC wrote their
final recommendations. Those too are very encouraging. They
stood firm on the importance of file formats:
"...the public sector should avoid any format that does not
safeguard equal opportunities to market actors to implement format-processing
applications, especially where this might impose product selection on the
side of citizens or businesses..."
Their response to Microsoft's argument on custom-defined schemas was
interesting. Instead of arguing whether the feature is really valuable, they
chose to challenge OASIS to add custom-defined schemas to the format to match
Microsoft's argument. The TAC also had some suggestions for Microsoft, finding
that the MS XML and MS XML Reference License were simply not open enough.
Recommendations for OASIS:
- Add custom-defined schemas to the format.
- Submit the format to ISO for a more official seal of approval.
Recommendations for Microsoft:
- A public commitment to keeping future versions of MS XML public.
- Submit the format to an international standards body of their choice.
- Remove non-XML components from the format.
Recommendations for other industry players:
- Participate in the OpenDocument standardization process to encourage
wider industry consensus around the format.
- Include filters to support both OASIS OpenDocument and MS XML.
- Provide tools to help the public sector migrate its documents to XML
Recommendations for the general public:
Microsoft responds to the TAC recommendations
Microsoft's response [PDF] essentially comes down to:
Sun responds to the TAC recommendations
They agree to publish future versions of MS XML under
They say they will "vigorously" work on documenting the non-XML
elements of the format, though they disagree that they should exclude
all non-XML elements. They claim that OpenDocument also has non-XML
elements (such as images) but fail to point out that in OpenDocument
those are in a separate directory and in MS XML they are embedded
throughout the XML tags.
They also say that supporting various formats is important, but
fail to say that they'll support OpenDocument.
They insist that their licensing scheme is just fine, and that
"royalty-free licensing programs have a role to play alongside formal
standards". Translation: No, they won't submit it to a standards body.
Sun's response [PDF] reiterates the importance of formats that are truly open,
in every sense. They like the idea of ISO support and are confident that the
format will become an ISO standard.
IBM responds to the TAC recommendations
IBM's response [PDF] was also positive. There was a tiny bit that
... we believe [that standard] technologies should be
published without restriction (other than reasonable royalties for
Well, I don't really know much about this, so I'll let other people decide
if the stuff in brackets is significant. But other than that, IBM informs the EU
- They will join the the OASIS Technical Committee.
- They already offer products (Workplace) that conform with OpenDocument.
This last bit isn't surprising, since Workplace is based on OpenOffice.org code,
but it's a good reminder.
Where We Are, Where We Need To Go
The OpenDocument format is arguably the single most important step for the FOSS movement after software
patents. It is unique in that:
This Is a Battle We Can Win
The OpenDocument format is in a unique position because it has a real chance
Technical merit: With the inclusion of custom schemas, OpenDocument can
meet and exceed every technical hurdle that can be thrown at it.
Adoption: OpenOffice.org, KOffice, IBM Workplace, StarOffice.
OpenDocument is not a dream. It is a real format with enough support to
present a real alternative.
OpenOffice.org is cross platform, easy to download and install, and is
preparing to ship version 2.0. Sun, IBM, HP, Red Hat, Novell,and even Adobe
were able to announce full support of the EU open XML standards requirements
based on the inclusion of OpenOffice.org components in their products and
services. That both OpenOffice.org and KOffice are open source with open
component frameworks enables the entire marketplace of vendors to easily
comply if they choose to do so.
OpenDocument is more than an open format, it is a open standard
. That is, it is backed by standards groups, ISO and OASIS.
It is not controlled by any company or product. Not by Sun or IBM. Not
by OpenOffice.org or KDE. Is Microsoft willing to match that?
Pushing OpenDocument as a Standard : The
European Union is pushing for a policy of open standards. So is the state
of Massachusetts, USA. The time is right to push for OpenDocument as the
standard for government documents. People are listening.
What OpenDocument Needs To Succeed
The Valoris report makes it clear that any open format that is to succeed
as an official standard for government documents needs four things:
Technical merit: Most open formats that were evaluated
did not make it to the final round because they lacked important features.
Any technical shortcoming, no matter how theoretical, gives Microsoft a
Adoption: No adoption, no success. Learn from ODA.
Open standard: Why is this so important? Why not just
any open format like the old OpenOffice.org 1.0 format or KOffice?
Because Microsoft would argue unfair treatment. Yes, I know, ha ha. But
still, Microsoft could effectively lobby against any format controlled by
one application supplier.
This is why OpenOffice.org and KOffice are both switching to OpenDocument
as their primary format (in versions 2.0 and 1.4 respectively).
The Fight for Open Standards, OpenDocument As a Standard: The fight for open standards will not be
easy. It will take will power from politicians, and the public who
elects them, to come through. Open XML technologies are the future
of collaborative computing, but the proprietary efforts to compromise their
implementations, or lock them up with insidious patents and restrictive
What You Can Do To Help
Keep your focus on the four items above. Let's go through each and see
what we need.
If you find a significant failing in the format, join the
OASIS TC and participate. Or, you can follow the example of XML
expert David Wheeler. Even though David is not a member of OASIS,
he routinely participates in discussions and contributes to the
specification through the public forum. The OASIS TC also maintains
liaisons with other open XML efforts such as the OASIS UBL and the
W3C XForms groups.
Then there is always the option of working through one of the many
OpenOffice.org project groups to directly impact the specification.
Examples of this include the Bibliography and DocBook groups who
labored long and hard to prepare the specification to perfect the
demands of the Library of Congress' very complex "MODS" schema.
Write filters for Abiword, Gnumeric, Sodipodi and Inkscape.
Promote applications which support OpenDocument.
Help at OpenOffice.org or KOffice.
|Pushing for OpenDocument
Help convince the EU and MA to support OpenDocument.
Write letters, lobby.
Inform people. Feel free to use this article to do so.
Become informed yourself.
Read the Valoris report and the documents listed here.
If you live near San Diego, California, go to the
OpenOffice.org RegiCon on Feb 9 and hear Gary Edwards from
OASIS talk about OpenDocument:
The Shot Heard Round the World: How OASIS Open Document Changes
Description of the talk: "Now two years in the making, the OASIS Open Document XML file format specification is making its mark. This session will discuss how the OASIS TC made dramatic enhancements to the file format specification to meet the next generation information system requirements of the European Union. Enhancements such as XForms, SVG, and SMiL were needed to meet the "custom-defined schema" requirement. Submission of the specification to ISO (the International Standards Organization) was another challenge met by the TC. This discussion will also cover the dynamics of Microsoft's arguments to the EU that MSXML file formats and licenses are "open enough". The EU held their ground though. The requirements for " Open Standards based Open XML file formats" are now in the EU purchase cycle. A fact which quickly led to the capitulation of Microsoft in mid November of 2004, when they agreed to comply with the requirements in future releases, as well as to work with Sun to provide OASIS/ISO file format filters as plugins for current MS Office applications. Since IBM's WorkPlace desktop environment is based on OpenOffice.org components, IBM, Adobe, and over a 150 other IBM affiliated information technology vendors were also able to announce compliance with the EU requirements in November. When open standards policies find their way into the purchase cycle, the marketplace becomes competitive in every way imaginable. More importantly though, open standards based policies enable computational consumers take ownership of both their information, and the information processes they depend on. And that changes everything."
Attend the above talk and provide a report for Groklaw.
Gary Edwards, from the OASIS TC, reviewed this article for technical
accuracy. Jean Hollis Weber edited the final draft. Any errors or omissions
are entirely my own.
Valoris report, along with TAC recommendations and all
This article is licensed under the same
Creative Commons License as PJ's articles. Daniel Carrera can be reached at dcarrera at openoffice.org
You can download this article in
OpenDocument format as well as
OpenOffice.org 1.0 and