Rhinoceros, Final Version
29 09 2008That’s official, and an interesting, sad, and probably ridiculous page of the I.T. History is about to be turned: OOXML has been published in its final version. Has it really? Well, it turns out that OOXML does have lots of tricks up its sleeve. It seems that the specification document is actually the same than the last time we saw it. Let me rephrase this: We know several versions of OOXML. Some succeeded to others, some others are concurrent; but we can roughly consider two major versions of OOXML. One is used inside Microsoft products such as MS Office 2007 and is thoroughly undocumented and not standard. Another one is the ISO standard, ISO-IEC 29500.
Today, we’ll focus on the “standard standard”, so to speak. This “standard standard” has just been officially published. But there’s a big surprise. The text seems to be exactly the same version of OOXML we once saw, the ghost version of OOXML (do you understand now why it took so many times for Microsoft to release Vista? The answer comes out clearly now: they were working on different versions of OOXML as if they were planning to shoot Pres. Kennedy on Elm Street).
Why does it matter? Because this ghost version, the “Final DIS text” is the one that was supposed to be the version to be approved by the various national standards committees after the BRM. Both the Ecma and the ITTF explained at the time (March 2008) that this text didn’t exist. But it does, and it was ready to be published ever since that time as it had been seen once on the ISO’s JTC 1’s server a while ago.
In short, national standards bodies voted on a text they never read, and the result was an astounding yes prompted by pressures of various kinds. The rest is history: The appeals that never got answered properly, the dubious voting procedure, the letter of protest sent by four countries to the ISO… Once again, this chapter is full of darkness, lack of transparence and maneuvers in dark alleys. Once again, the ISO has not hesitated once to dive in the mess and proudly follows what it believes is the reasonable way; so reasonable, in fact, that if told to define the Law of Gravity the ISO would now claim that any physical body falls on the ground if released from above not because of Gravity, but because it is reasonable.
All in all, the history of OOXML is not over. The maintenance phase is actually going to be hilariously complex, as you will see experts haggle over the countless existing issues and bugs of a standard that never got implemented ( so what? Can’t we be funny from time to time?) while probably trying to come up with a dubious workgroup on implementation conformance, if that role has not already been taken by the joint Afnor-DIN committee where nobody ever really meets but everyone receives emails from unknown superiors that preside over the work of that committee.
Yet, all the fun stuff set aside, something odd keeps coming back in my mind. Am I the only one to see that this new, final OOXML version was published -although it existed for months inside the ITTF- right at the time where the period for new appeals ended?
Call me disingenuous if you wish. But don’t call me an Ionesco; we already have similar artists in Geneva and in Redmond…
There are quite a few errors in this blog post:
1. You say that “One is used inside Microsoft products such as MS Office 2007 and is thoroughly undocumented and not standard”
The version used in Office 2007 is ECMA-376, which is standardized and fully documented.
2. You say that “The text seems to be exactly the same version of OOXML we once saw”
You seem to be claiming that the published verion does not contain any of the changes voted upon at the BRM. Is that what you are really claiming? If I am not misinterpreting you, that is completely false.
3. You state that “Both the Ecma and the ITTF explained at the time (March 2008) that this text didn’t exist. ”
Not quite. They said that an OFFICIAL version did not not exist. And there’s no logical way it could have, since the BRM had yet to make changes to it.
4. You claim that “national standards bodies voted on a text they never read”.
This is false. All national bodies were given copies of a version that contained the changes that ECMA recommended (which were the overwhelming majority of changes suggested by the NBs themselves). But, as I said in 3) above, that was and could not be the OFFICIAL version, which could only come about AFTER the BRM.
This is a very misleading blog post.
Ian, thank you for your comments. Here are my answers to each of your point (with the corresponding numbers).
1. You are completely wrong I am afraid. What is used in Office 2007 is not Ecma 376. It is something else. The best evidence of this is that OOo couldn’t use the Ecma spec to engineer the OOXML filters for OOo 3.0. The OOXML filters are filters for MS Office 2007 format called OOXML but it’s not the Ecma standard.
2.Actually I may have expressed myself in an unclear way, and this is why you have misinterpreted my words: the OOXML version “we once saw” is not the one before the BRM. it’s the “consolidated” version we saw after the BRM.
3. There you lost me. Any version could have been welcome, but the whole process was unclear and lacked transparency.
4. You areapparently mixing the dates. National standards boards met after the BRM, if you remember, and at that time there were not a single post-BRM version available that they had access to. I know what I am talking about, I was at the Afnor.
I don’t think it’s a misleading post actually. But claiming that Office 2007 implements the Ecma 376 is simply wrong, wrong, and again wrong. And there are lots of technical and non technical evidences of this.
[…] to yesterday’s strong criticism of ISO, Charles had this to say: In short, national standards bodies voted on a text they never read, and the result was an […]
Hi Charles,
We can engage in a he said / she said type of thing for ever about Office 2007 and ECMA-376, but I’ll take one last shot, if you don’t mind. You are wrong about this. OOXML was the name for the format before it was modified by Ecma International and then published as ECMA-376. During that modification, Microsoft had to issue a second beta of Office 2007 in order to keep it up to date with the revisions of the developing standard. (I know — I was one of the beta testers of Office 2007, and I had to download that update to handle ECMA-376). As for OOs inability to produce a filter, I know nothing of that, but it doesn’t prove that Office 2007 doesn’t follow ECMA-376. For example, the ECMA-376 standard allows for some deprecated features, such as VML. Office 2007 implements some of those deprecated features of the standard. If a filter did not take into account all the capabilties of ECMA-376, it would fail. Further, there is one area of the Office 2007 file format that is not covered in ECMA-376: macros. (That was done on purpose because only a tiny percentage of Word document have macros: less than 1%) Perhaps this is what you are referring to?
About the versions. There was an unofficial draft version produced by ECMA that was distributed to all NB’s PRIOR to the BRM. I understood your statement to be that the NBs didn’t see this prior to the BRM. If my understanding is correct, then your statement is wrong.
In any case, the maintenace phase of IS29500 is well under way, with NBs commenting on the post-BRM draft. When that phase is complete shortly, a final version will be published.
Ian,
One last shot from my side then.
OOXML was the name for the format before it was modified by Ecma International and then published as ECMA-376.You are absolutely right about this, and by the way I don’t see how it contraddict what I wrote in the post. But I am always happy to read the truth.
Microsoft had to issue a second beta of Office 2007 in order to keep it up to date with the revisions of the developing standard. (I know — I was one of the beta testers of Office 2007, and I had to download that update to handle ECMA-376)That’s rather surprizing. Ecma 376 has been tested against the actual format used in the final version of Office 2007 and there were hundred, if not thousands of insconsistencies that showed that MS Office 2007 not only was not implementing the Ecma Standard (let alone the ISO one), but was actually implementing something different; something so different, that implementation of the filters inside OpenOffice.org was and is difficult and that the engineers working on it have quit using anything in the way of the Ecma spec or the post BRM spec to develop the filters.
As for OOs inability to produce a filter, I know nothing of that, but it doesn’t prove that Office 2007 doesn’t follow ECMA-376. For example, the ECMA-376 standard allows for some deprecated features, such as VML. Office 2007 implements some of those deprecated features of the standard. If a filter did not take into account all the capabilties of ECMA-376, it would fail.Actually, if engineers are unable to implement an Ecma spec, then there is either a problem with the qualification and skills of engineers (which I think we can agree is not the case at hand) or a problem with the spec.
I can tell you it was not the VML that blocked them, although what you say about it is interesting: I was sworn by MS staff at the Afnor that VML was deprecated and “not a problem for implementation”. So now, VML that was a spec that got rejected as a standard is mandatory?
And no, I don’t mention the macros, that is something that falls into a different chapter unfortunately.
About the versions. There was an unofficial draft version produced by ECMA that was distributed to all NB’s PRIOR to the BRM. I understood your statement to be that the NBs didn’t see this prior to the BRM. If my understanding is correct, then your statement is wrong.No, I don’t refer to this document. I refer to the version with the consolidated comments that appeared just at the end of the BRM and that was collated by Rex Jaeschke. The version you refer to was I think communicated to the NBs but was it worth anything if it was one that was drafted before the BRM?
In any case, the maintenace phase of IS29500 is well under way, with NBs commenting on the post-BRM draft. When that phase is complete shortly, a final version will be published.Here we have a major problem, Ian. What we should be talking about is the final version, not a final draft. If we’re talking about a final draft then why was it blocked for such a long time at the ITTF?
OK, one last thing. Please give a reference to this independent test that found “thousands” of inconsistencies between ECMA-376 abd the Office 2007 formats. I have heard this rumor many times, usually from anti-OOXML zealots, and no one has EVER given a reference.
By the way, you are aware that Microsoft claims that Office 2007 is ECMA-376 compliant, right?
Regards, Ian
Ian,
I really hope that your figures are right that only 1% percent of Microsoft Binaries Formats bear macros. If so, it is right and good to assure that one can migrate 99% of their documents to OpenOffice and invest their money on their business rather than in trade-mark proprietary office licenced aplications.
regards,
Hi Charles,
I am still awaiting your respond to my question about proof for your claim that Office 2007 does not use ECMA-376? Where is this study that supposedly found thousands of errors? You really should not repeat such claims unless you know them to be true, by checking out the study yourself.
Ian
I am travelling right now and have not a lot of time. The study I am referring to has been linked (I think, but can’t remember) from this blog a while ago (like march or april 2008), and can be found either on groklaw or on noooxml. The study comes from the ECIS. I did check the study myself.
That being said, I am not going to do you a favour by giving into your language. I know the claims I am making are true, and am surprized you still question my integrity. You will noticed that I never questioned yours, so I would be happy if you’d keep your tone polite.
Thanks,
Charles.
Hi Charles,
I was simply assuming that if you knew of the study, you would have quoted it.
I did a quick search of your blog archives during the period you mentioned. Maybe I missed it, but all I found was a repeat of the same assertion, with no reference.
I looked for any ECIS “study” along the lines you mentioned. Again, maybe I missed it, but all I found was a public announcement making an unverified (and actually false) assertion that ECMA-376 contains windows dependencies.
When you do get the time, I would appreciate a pointer. I really would like to know where this came from, and why it is endlessly repeated without any evidence given to back it up (that NOT to say it doesn’t exist; don’t get me wrong!),
Thanks and have a nice trip. Ian
Hi Charles, I hope your trip is going well.
After about 4 hours online this morning, I believe I may have tracked down the source of your statement that it is well-known that Office 2007 does not use ECMA-376 for its file formats and that thousands of differences have been found. (I hope I have summarized your points accurately; if not, please correct me.)
The clue to finding it was your statement that Groklaw had said it. After trawling through endless groklaw and grokdoc articles, I finally found one that made this claim:
“I notice that MS Office 2007 is not using OOXML after all. There is yet another binary format that is silently introduced and touted as the preferred option.”
[Grokdoc Article “A History of MS’ Standards ‘Dirty Tricks’”]
They reference Rob Weir, in his blog post of January 08, 2007, entitled “The Formats of Excel 2007”.
In that blog, Weir talks about two things that he wasn’t aware of before, although they are by no means secret. They were talked about in Microsoft blogs before he posted. The two items are:
- The format of the source code for Excel macros is not defined in OOXML.
- There is an optional binary file format called XSLB for Excel that is not part of OOXML.
As I have summarized them above, both statements are perfectly correct.
As I had mentioned in my earlier comments to you, Office documents with macros, which represent less than 1% of all Office documents, are not covered by the ECMA-376 (or IS29500) spec. This was a deliberate decision by ECMA, because of their rarity and because ECMA had a lot of work to do. Note, however, that ECMA-376 DOES document where macros are to be stored; it just does not define their format. It has been mentioned that the source code format will be covered as part of maintenance; this makes sense to me.
As for the existence of the XSLB binary file format (NOT the default one) in Excel 2007, that was covered in a Microsoft blog long before Weir found out about it. The reference is:
http://blogs.msdn.com/dmahugh/archive/2006/08/22/712835.aspx
“New Binary File Format for Spreadsheets”, 22 August 2006.
The reason why Microsoft introduced this very rare, optional, file format for Excel in Office 2007 is given in that blog by Mahugh as follows:
“There is also another new file format for Excel spreadsheets that you’ll want to take a look at if you’re building unusually large or complex spreadsheets: the new XLSB binary format. Like Open XML, it’s a full-fidelity file format that can store anything you can create in Excel, but the XLSB format is optimized for performance in ways that aren’t possible with a pure XML format.”
He also adds the following in a comment to the Weir’s blog I mentioned above:
“It’s simply an option for users who have huge complex spreadsheets and are willing to forego standards in the name of performance. The kinds of spreadsheets that XLSB is used for are not viable in either Open XML or ODF, and those users have decided that they’d rather optimize performance than use an open standard format. Again, the user gets to choose, and the default format of Open XML is always there for Office users.”
Weir’s characterizations of these two Excel file formats are quite misleading, in my opinion. Check out his blog to see if you agree. Those characterizations have been picked up by Groklaw, further distorted, picked up elsewhere and distorted endlessly until they have acquired the status of a full-blown myth that Office 2007 does not use ECMA-376 and that there are thousands of differences.
If what I have found is indeed the source of your statements, then they are quite incorrect, and I believe a retraction is in order. On the other hand, if you have another source for your statements, I would be most interested in following it down.
Ian,
actually no, this is not what you are talking about; am still looking for it but what I can tell you is that it’s a document, (a powerpoint or impress file) that has been uploaded somewhere, last Spring and am looking for it as well.
Cheers,
Charles.
[…] OOXML is here […]