Mastodon
Where is there an end of it? | All posts tagged 'JTC1'

Document Format Standards and Patents

This post is part of an ongoing series. It expands on item 9 of Reforming Standardisation in JTC 1.

Background

Historically, patents have been a fraught topic with an uneasy co-existence with standards. Perhaps (within JTC 1) one of the most notorious recent examples surrounded the JPEG Standard and, in part prompted by such problems there are certainly many people of good will wanting better management of IP in standards. Judging by some recent development in document format standardisation, it seems probable that this will be the area where progress can next be made …

Most recently, the Fast Track standardisation of ISO/IEC 29500 in 2007/8 saw much interest in the IPR regime surrounding that text, with much dark suspicion surrounding Microsoft's motives. However, the big development in this space – when it came – was from an unexpected direction …

The i4i Patent

Back in the SGML days I remember touring the floor of trade shows and noticing the S4-Desktop product from Candian company Infrastructures for Information, Inc (i4i). Like a number of other products at the time (including Microsoft's own long-forgotten SGML Author for Word, or Interleaf’s BladeRunner) it attempted to make Word™ a structure-aware authoring environment, based on the (accurate) belief that while many companies wanted structured data they didn't want to have to grapple with pointy brackets.

Keen to avoid the phenomenon that Rob Weir describes whereby

There is perhaps no occasion where one can observe such profound ignorance, coupled with reckless profligacy, as when a software patent is discussed on the web.

I will avoid any punditry about the ongoing legal course of this patent. Those interested would do well to read IP lawyer Andy Updegrove's post (and follow-up) on the legalities of this matter.

On the technical merit of the standard though, there appears to me to be unanimity among disinterested experts qualified to judge. For example Jim Mason (for 22 years the chair of the ISO committee responsible for all-things-markup) commented:

[T]his technique did not originate with i4i. It was already established in other commercial products and was, in effect, standardized in ISO/IEC 8613, Office Document Architecture. ODA essentially described a binary format for word-processor document representation, which worked by pointers into a byte stream. Its original interchange format, ODIF, started as a representation of that structure, but it was extended to have an alternative SGML stream, exported by a process similar to that described in the i4i patent. So there was prior art, specifically prior art described in public standards.

This point was expanded on by markup veteran Rick Jelliffe, who concluded:

By the end of the judgment I was left thinking "what interactive XML system with any links wouldn't be included in this?" which is utterly ridiculous.

I was creating SGML systems from 1989, and the i4i patent is just as obvious then as it is now.

In a Guardian Interview i4i chairman Loudon Owen seemed to make it clear that the patent would not be licensed on a reasonable and non-discriminatory (RAND) basis (at least – or especially – where Microsoft are concerned):

On licensing to Microsoft, Owen sounds on the edge of anger: "No. No. This is our property. We are going to build our business. There's no right for Microsoft to use it and go forward." But i4i could license it at some humungous, eye-watering price that Microsoft might have to pay, surely? No, says Owen.

The Wider Context

As part of its amicus brief (PDF) in the Bilski case pending before the Supreme Court, IBM offered what might be termed the orthodox pro-patent position. In a section headed “Software Patent Protection Provides Significant Economic, Technological, and Societal Benefits” we thus find a footnote quoting this text:

Given the reality that software source code is human readable, and object code can be reverse engineered, it is difficult for software developers to resort to secrecy. Thus, without patent protection, the incentives to innovate in the field of software are significantly reduced. Patent protection has promoted the free sharing of source code on a patentee’s terms—which has fueled the explosive growth of open source software development.

While it is somewhat surpising to learn here of the affinity between FOSS and patents, the point is of course that the idea of patents is not wholly without foundation: that a state-sanctioned restraint of trade (for such is a patent) is justified in allowing innovators to monetize their inventions. However, increasingly when we listen to the voices of actual FOSS (and non-FOSS) people the view seems to be that any advantages are outweighed by the problems of patents. For example Mike Kay (developer of the superb Saxon family of XSLT, XQuery, and XML Schema processing products) in an open letter to his MP argues against software patenting in a piece which is well-worth reading in its entirety:

The software business does not need incentives to innovate. If you don't innovate, you die. [...] [I]n the software business, patenting of ideas benefits no-one: certainly, it does not benefit society or the economy at large, which is the only possible justification for governments to interfere with the market and grant one company a monopoly over an idea.

And, in specific reference to the i4i patent:

recently an otherwise unsuccessful company has been awarded a similar [i.e. 9-figure] sum against Microsoft, for an idea which most people in the industry considered completely trivial and obvious.

More colourfully Tim Bray lists some horror-story cases (again well worth reading) and opines that the whole patent system is "too broken to be fixed". He also addresses the question of whether patent activity benefits society, and comes down firmly against:

And here are a few words for the huge community of legal professionals who make their living pursuing patent law: You’re actively damaging society. Look in the mirror and find something better to do.

The Myth of Unencumbered Technology

Given the situation we are evidently in, it is clear that no technology is safe. The brazen claims of corporations, the lack of diligence by the US Patent Office, and the capriciousness of courts means that any technology, at any time, may suddenly become patent encumbered. Technical people - being logical and reasonable - often make the mistake of thinking the system is bound by logic and reason; they assume that because they can see 'obvious' prior art, then it will apply; however as the case of the i4i patent vividly illustrates, this is simply not so.

Turning to document format standards, we can see there most certainly are known and suspected patents in play. For example:

  • the i4i patent mentioned above (which, in his Guardian interview, the i4i Chairman refuses to rule out as applying to ODF)
  • 45 unspecified patents which Microsoft has claimed OpenOffice.org infringes, some number of which may relate to the ODF specification (and which Sun and Microsoft agreed a cease-fire over until 2014 - at least as far as Sun is/was concerned)
  • an unknown number of unspecified patents which have led IBM to include ODF under its Interoperability Specifications Pledge
  • an unknown number of unspecified patents which have led Microsoft to include OOXML under its Open Specification Promise (though presumably clear OOXML-specific patents such as US Patent 7,676,746 are in scope here)

Now, as is clear from the above, large corporations have a preferred means of neutralising their IP stake in standards: by "promises", "covenants" and the like.

The question for standardizers remains: is the current situation acceptable? and if not, what can be done to improve it?

The ISO Rules (and Are They Followed?)

Since 2007 the "big three" International SDOs (ISO, IEC and ITU-T) have operated a common patent policy predicated on the wholly reasonable premise that standards should be "accessible to everybody without undue constraints". The policy is implemented in detail by JTC 1 (which joins the forces of ISO and IEC) and which – as we know – governs the International Standards ODF and OOXML.

The Policy as implemented in the Directives has several aspects, which I would categorise as falling under the following headings …

Personal Disclosure

Anybody aware of an IPR issue has a duty to speak out:

any party participating in the work of the Organizations should, from the outset, draw their [sic] attention to any known patent or to any known pending patent application, either their own or of other organizations. (ISO Directives Part 1, Clause 3)

And indeed committee secretaries and chairs are routinely reminded by Geneva to issue a request for IPR disclosure at meetings, to jog people's memory.

Formal Disclosure in Standards

Readers of Standards can expect to have the IPR/patent situation made explicit in the text before them, and accordingly there are may textual items mandated for Standards to which patents apply. In particular it is stated, "[a] published document for which patent rights have been identified during the preparation thereof, shall include the following notice in the introduction:"

The International Organization for Standardization (ISO) [and/or] International Electrotechnical Commission (IEC) draws attention to the fact that it is claimed that compliance with this document may involve the use of a patent concerning (…subject matter…) given in (…subclause…).

Centralised Record-keeping

A JTC 1 "patent database" (served as a huge HTML document) is maintained in Geneva which gathers together all the patents applying to published standards, and the terms under which patent holders have agreed to make licenses available.

Clear Access Rights

Patent Holders who have signed the licensing declaration to ISO, IEC or ITU-T agree to license their patents under a clear regime: either RAND, ZRAND (i.e. RAND with a free-of-charge license), or – exceptionally – on a per-case commercial basis. Anybody accessing the patent database is able to see this and, by referring to the ISO/IEC governing documents, know what it means, not least because no deviations from Geneva's wording are permitted:

the patent holder has to provide a written statement to be filed at ITU-TSB, ITU-BR or the offices of the CEOs of ISO or IEC, respectively, using the appropriate "Patent Statement and Licensing Declaration" Form. This statement must not include additional provisions, conditions, or any other exclusion clauses in excess of what is provided for each case in the corresponding boxes of the form.

Problem Handling

And if things go wrong:

2.14.3 Should it be revealed after publication of a document that licences under patent rights, which appear to cover items included in the document, cannot be obtained under reasonable and non-discriminatory terms and conditions, the document shall be referred back to the relevant committee for further consideration.

Unfortunately, when we hold up the big two document standards of ODF and OOXML against the goals set out, we see there is work still to be done …

Moving Forward

While the "broken stack" of patents is beyond repair by any single standards body, at the very least the correct application of the rules can make the situation for users of document format standards more transparent and certain. In the interests of making progess in this direction, it seems a number of points need addressing now.

  • Users should be aware that the various covenants and promises being pointed-to by the US vendors need not be relevant to them as regards standards use. Done properly, International Standardization can give a clearer and stronger guarantee of license availability – without the caveats, interpretable points and exit strategies these vendors' documents invariably have.
  • In particular it should be of concern to NBs that there is no entry in JTC 1's patent database for OOXML (there is for DIS 29500, its precursor text, a ZRAND promise from Microsoft); there is no entry whatsoever for ODF. I would expect there to be declarations from the big US vendors who profess patent interests in these standards, and I would expect this to be addressed as a matter of urgency (perhaps in parallel with the publication of these standards' forthcoming amendments)
  • In the case of the i4i patent, one implementer has already commented that implementing CustomXML in its entirety may run the risk of infringement (and this is probably, after all, why Microsoft patched Word in the field to remove some aspects of its CustomXML support). OOXML needs to be referred back to its committee (this may be JTC 1, not SC 34) for a decision on what happens next. My personal guess is that CustomXML will be left in OOXML Transitional (patent-encumbrance will be just one more of the many warning stickers on this best-avoided variant), and modified in, or removed from, OOXML Strict
  • When declaring their patents to JTC 1, patent holders are given an option whether to make a general declaration about the patents that apply to a standard, or to make a particular declaration about each and every itemized patent which applies. I believe NBs should be insisting that patent holder enumerate precisely the patents they hold which they claim apply to ODF or OOXML, as this will give greater transparency about what is (or is not) covered and prevent the vague threat ("there may be patents but we're not saying what") which seems to apply at the moment.

There is obviously much to do, and I am hoping that at the forthcoming SC 34 meetings in Stockholm this work can begin. Certainly, anybody reading this blog post now knows there are outstanding IPR issues which we as standardizers have a duty to raise …

SC 34 WG meetings in Paris next week

Once again I feel that bubbling up of almost schoolboy fervour that presages a set of SC 34 meetings. In Paris no less (AFNOR shall be our hosts): the city of love, fine art and rognons à la moutarde.

What is tasty on SC 34’s menu? Well four working groups are meeting next week:

  • WG 1 (which I convene) will be carrying forward its work on foundation standards – particularly the schema languages of DSDL. We have two new (/proposed) projects to discuss: one a schema language focussed on cross-reference validation; one on associating schemas with documents using processing instructions. Probably our most successful schema language, RELAX NG, is due for an update and several new features are up for discussion: keep it coming!
  • WG 4 (OOXML) will continue its intensive maintenance on ISO/IEC 29500 – not least in handling a new set of approved corrigenda (just voted on) and dealing with the day-to-day grind of correction and improvement. There are larger questions to answer too, in particular those which concern the relationship between the Strict and Transitional forms of OOXML. I have led the preparation of a background paper on this which (thanks to the newly open WG 4 mail archive) can be accessed as a public document (PDF). I predict some lively discussion!
  • WG 5 (OOXML/ODF interop) will continue its work examing how (or not) the two formats may be used by systems which hope to interoperate. TR 29166 - dedicated to this topic - continues to take shape ahead of its projected finishing date in 2011.
  • WG 6 is the newly-created WG dedicated to the JTC 1 side of maintenance of ISO/IEC 26300:2006 (aka ODF v1.0). As a newly-created group there will no doubt be a certain amount of adminstrivia to be got through but there are more substantial issues looming too: defect reports to be advanced and the longer-term project of amending ISO/IEC 26300 to bring it into alignment with ODF 1.1 – there is general agreement that it makes sense to reduce marketplace confusion by reducing the confusing number of standard (and non-standard) “ODF” variants out there, and aligning versions between standards organisations.

Stay tuned (and follow hashtag #sc34 for real-time updates) …

SC 34 Meetings, Seattle - Day 1

At the invitation of ANSI, SC 34 is in sunny Bellevue for five jam-packed days of Standards meetings (Sunday-Thursday). This is a full and busy event, with around 60 delegates registered from 14 countries (Belgium, Brazil, Canada, China, Denmark, Finland, France, Germany, India, Korea, Norway, South Africa, UK, and the USA) and 4 liaison organisations (Ecma, OASIS, W3C and the XML Guild).

Maybe by the end of it a number of momentous questions will have been answered, including:

  • Whether the world needs a standardised way to associate XML documents with schemas
  • Whether OOXML Transitional should be evolved
  • How ISO/IEC 26300 shall be maintained within SC 34
  • How standard schemas should be licensed to users
  • How MIME types should best be used for identifying document formats

Stay tuned ...


SC 34 meetings, Copenhagen

This week I attended 4 days of meetings of SC 34 working groups. WG 4 (OOXML maintenance) and WG 5 (OOXML/ODF interoperability). Last year I predicted that OOXML would get boring and, on the whole, the content of these meetings fulfilled that prophecy (while noting, of course, that for us markup geeks and standards wonks “boring” is actually rather exciting). There was however, one hot issue, which I’ll come to later …

Defects

Since the publication of OOXML in November last year, the work of WG 4 has been almost exclusively focussed on defect correction. To date over 200 defects have been submitted (the UK leading the way having submitted 38% of them). Anybody interested in what kinds of thing are being fixed can consult the material on the WG 4 web site for a quick overview. During the Copenhagen meeting WG 4 closed 53 issues meaning that 71% of all submitted defects submitted have now been resolved. By JTC 1 standards that is impressively rapid. The defects will now go forward to be approved by JTC 1 National Bodies before they can become official Amendments and Corrigenda to the base Standard. Among the many more minor fixes, a couple of agreed changes are noteworthy:

  • In response to a defect report from Switzerland, for the Strict version (only) of IS 29500, the XML Namespace has been changed, so that consumers can know unambiguously whether they are consuming a Strict or Transitional document without any risk of silent data loss. This is (editorially) a lot of work, but the results will be I think worthwhile.
  • As I wrote following the Prague meeting, there was a move afoot to re-instate the values “on” and “off” as permissible Boolean values (alongside “true”, “1”, “false” and “0”) so that Transitional documents would accurately reflect the existing corpus of Office documents, in accord with the stated scope of the standard. This change has now been agreed by the WG.

The ISO Date Issue

The “hot issue” I referred to earlier is ISO dates. What better way to illustrate the problem we face than by using one of Denmark’s most famous inventions, the LEGO® brick …


f*cked-up lego brick
OOXML Transitional imagined as a LEGO® brick

More precisely, the problem is about date representation in spreadsheet cells. One of the innovations of the BRM was to introduce ISO 8601 date representation into OOXML. However the details of how this have been done are problematic:

  1. Despite the text of the original resolution declaring that ISO 8601 dates should live alongside serial dates (for compatibility with older documents), one possible reading of the text today is that all spreadsheet cell dates have to be in ISO 8601 format
  2. Having any such dates in ISO 8601 format is particularly problematic for Transitional OOXML, which is meant to represent the existing corpus of office documents. Since none of these existing documents contain ISO 8601 dates having them here makes no sense
  3. Even more seriously, if people start creating “Transitional” OOXML documents which contain 8601 dates, then a huge installed base of software expecting Ecma-376 documents will silently corrupt that data and/or produce surprising results. (My concern here is more for things like big ERP and Financial systems, rather than desktop apps like MS Office). Hence the odd LEGO brick above: like those bricks, such files would embody an interoperability disaster
  4. Even the idea of using ISO 8601 is pretty daft unless it is profiled (currently it is not). ISO 8601 is a big complex standards with many options irrelevant to office documents: it would be far more sensible for OOXML to target a tiny subset of ISO 8601, such as that specified by W3C XML Schema Definition Language (XSD) 1.1 Part 2: Datatypes
  5. Many date/time functions declared in spreadsheetML appear to be predicated on date/time values being represented as serial values and not ISO 8601 dates. It is not clear if the Standard as written makes any sense when ISO 8601 dates are used.

The solution?

Opinions vary about how the ISO date problem might best be solved. My own strong preference would be to see the Standard clarified so that users were sure that Transitional documents were guaranteed to correspond to the existing document reality – in other words that Transitional documents only contain serial date representations, and not “ISO 8601” date representations. In my view the ISO dates should be for use in the Strict variant of OOXML only.

If a major implementation (Excel 2010, say) appears which starts pumping incompatible, ISO 8601-flavoured Transitional documents into circulation, then that would be an interop disaster. The standards process would be rightly criticised for producing a dangerous format; users would be at risk of corrupted data; and guilty vendors too would be blamed accordingly.

It is imperative that this problem is fixed, and fixed soon.

Impressions of Copenhagen

Our meetings took place at the height of midsummer, and every day was glorious (all meetings started with a sad closing of the curtains). Something of a pall was cast over proceeding by thefts, in two separate incidents, of delegates’ laptop computers; but there is no doubt Copenhagen is a wondeful city blessed with excellent food, tasty beer, and an abundance of good-looking women. Dansk Standard provided most civilised meeting facilities, and entertainment chief Jesper Lund Stocholm worked hard to ensure everyone enjoyed Copenhagen to the full, especially the midsummer night witch-burning festivities! Next up is the SC 34 Plenary in Seattle in September; I’m sure there will be many more defect reports on OOXML to consider by then, and that WG 4's tireless convenor Murata-san will be keeping the pressure on to mantain the pace of fixes  …


Jesper Among the Beers
Jesper Lund Stocholm

SC 34 Meetings, Prague, Days 2, 3 & 4

What a Difference a Day Makes
MURATA Makoto, WG 4 convenor,
seems pleased with progress on the maintenance of OOXML

I had been intending to write a day-by-day account of these meetings but as it turned out there simply was not time for blogging (tweeting, on the other hand …). Another activity which suffered was photography: I had wanted to take a load of pictures of über-photogenic Prague – but somehow the work seemed to expand into all my notionally free time too. What I did manage to snap is here. And it is also well worth checking out Doug Mahugh’s photos for more.

WG 1 met again on Tuesday to finish its business (you can read the meeting report here) and then my attention mostly turned mostly to the activities in WG 4 (for OOXML maintenance) and WG 5 (which is for OOXML / ODF interop).

OOXML One Year On

Overall, WG 4 is making excellent progress. To date, 169 defects have been collected for OOXML (check out the defect log) and the majority of these have either been closed, or a resolution agreed. Amendments were started for 3 of the 4 parts of OOXML to allow a bunch of small corrections to be put in place, and the even-more-minor problems will be fixed by publishing technical corrigenda. Overall, I think stakeholders in OOXML can feel pretty confident that the Standard is being sensibly and efficiently maintained.

I was personally very pleased to see National Bodies well-represented (the minutes are here) – to the extent that I’d now ideally like to see some more big vendors coming to the table so their views can be heard. Microsoft (of course) was; but where (for example) are Apple, Oracle and the other vendors who participated in Ecma TC 45 while OOXML was being drafted? To them – and to anybody who wants to get involved in this important work – I say: participate!

Over at Rick Jelliffe’s blog Rick has been carrying out something of an exposé of the unfortunate imbalance in the stakeholders represented in the maintenance of ODF at OASIS (something which will become even more acute if Sun is, in the end, snapped-up by IBM). Personally I think Rick is right that it is vitally important to have a good mix of voices at the standardisation table: big vendors, small vendors, altruistic experts, users, government representatives, etc. WG 4 is getting there, but it too has some way to go.

Tougher Issues

Some more controversial topics were however not resolved during the meeting, and I think it may be worth exploring these in more detail:

The Namespace Problem

One issue in particular has proved thorny: whether to changes Namespaces in the OOXML schemas. This topic took a good slice of Tuesday, and then segued into a bar session afterwards; this then carried on over supper and by the time we broke it was midnight. And still no conclusion has been reached. WG 4 has issued a document outlining the state of discussions to date …

I have already expressed my own views on this; but during these Prague meetings some important new considerations were brought to bear. Go over to Jesper Lund Stocholm’s blog to get the thorny details

Personally, I think the “strict” format is a new format, and that changing the Namespace is only part of the solution. I would like to see the media type changes and for OOXML to recommend a new file extension (.dociso anyone?) to reduce the chances that users suffer the (to me) unacceptable fate of silent data loss that Jesper highlights.

Whither Transitional?

Another hot topic of discussion is what the “transitional” version of OOXML is really for. One interesting and slightly surprising fact that emerged after the BRM is that the strict schemas are a true subset of the transitional schemas. Should this nice link be preserved? Microsoft are keen for new features introduced in the “strict” version of OOXML to be mirrored in the “transitional” version – presumably, in part, because Office 14 will use transitional features.

Openness

When the maintenance of OOXML was being planned, one of the principles agreed on by the National Bodies was that the process should be as open as possible, consistent with JTC 1 rules. One aspect of this is the question of whether WG 4’s mailing list archive should be open to the public. Some NBs were a little nervous of this for the reason that their committee members might be less free to post candid comments if they were open to public scrutiny, and possible repercussions with their boss and/or the tinfoil brigade in the blogosphere. There was also the troubling precedent of the mailing list of the U.S. INCITS V1 committee, which opened its archive to public view during the DIS 29500 balloting period, only to see it die completely as contributors refused to post in public view.

The issue will now be put to public ballot, and I am hopeful that mechanisms have been put in place which will allow NBs to support opening of the archive. With public standards, public meeting reports, public discussion documents and a public mailing list archive I think WG 4 will demonstrate that an excellent degree of openness is indeed possible even within the constraints of the current JTC 1 Directives.

Overturning BRM decisions

The UK proposed an interesting new defect during the Prague meetings, which centred on one of the decisions made at the BRM.

Nature of the Defect:

As a result of changes made at the BRM, a number of existing Ecma-376 documents were unintentionally made invalid against the IS29500 transitional schema. It was strongly expressed as an opinion at the BRM by many countries that the transitional schema should accurately reflect the existing Ecma-376 documents.

However, at the BRM, the ST_OnOff type was changed from supporting 0, 1, On, Off, True, False to supporting only 0, 1, True, False (i.e. the xs:boolean type). Although this fits with the detail of the amendments made at the BRM, it is against the spirit of the desired changes for many countries, and we believe that due to time limitations at the BRM, this change was made without sufficient examination of the consequences, was made in error by the BRM (in which error the UK played a part), and should be fixed.

Solution Proposed by the Submitter

Change the ST_OnOff type to support 0, 1, On, Off, True and False in the Transitional schemas only.

The result of the BRM decision being addressed here was apparent in a blog entry I wrote last year, which attracted rather a lot of attention.

Simply put, the UK is now suggesting the BRM made a mistake here, and things should be rectified so that existing MS Office documents “snap back” into being in conformance with 29500 transitional.

This proposal caused some angst. Who were we (some asked) to overturn decisions made at the BRM? My own view is less cautious: this was an obvious blunder, the BRM got it wrong (as it did many things, I think). So let’s fix it.

Whither WG 5?

In parallel with WG 4, WG 5 (the group responsible for ODF/OOXML interoperability) also met. One of the substantive things it achieved was to water down the title of the ongoing report being prepared on this topic, changing it from:

OpenDocument Format (ISO/IEC 26300) / Office Open XML (ISO/IEC 29500) Translation

to:

OpenDocument Format (ISO/IEC 26300) / Office Open XML (ISO/IEC 29500) Translation Guidelines

Adding the word “guidelines” to the title should make it clear to anybody noticing this project that it is not an “answer” to ODF/OOXML interoperability, merely a discursive document. For myself, I have doubts about the ultimate usefulness of such a document.

It is disappointing to see the poor rate of progress on meaningful interoperability and harmonisation work. Of course these things are motherhood and apple pie in discussion – but when the time comes to find volunteers to actually help, few hands go up. In my view, the only hope of achieving any meaningful harmonisation work is to get Another Big Vendor interested in backing it, and I know some behind-the-scenes work will be taking place to beat the undergrowth and see if just such a vendor can be found.