What Windows Home Server and OOXML Have in Common: They Corrupt Data

Dr. Roy Schestowitz

2010-02-22 18:17:12 UTC
Modified: 2010-02-22 18:17:12 UTC

Summary: Data corruption glitches inherent and more likely with Microsoft's sub-standard products that do not comply with industry standards

TWO YEARS ago I called Windows Home Server (WHS) "data corruption server" because it turned out that its unique feature (or antifeature) was that it silently destroyed people's data rather than make backups like it was supposed to. We wrote about the disaster which is Windows Home Server around that time; it's built upon pretty much the same codebase that makes up Vista 7.

According to this very extensive new review of the Asus TS Mini Windows Home Server, GNU/Linux is still miles ahead of Microsoft when it comes to so-called "home servers" (Microsoft terminology for the most part). To quote some portions of the text:

Network Attached Storage (NAS) devices are essentially small servers designed for use in the home, but generally use modified versions of Linux. It was only a matter of time before Microsoft got in on the action with Windows Home Server (WHS), which it introduced in 2007.

Most NAS devices run Linux on hardware based around embedded processors from manufacturers such as Marvell or Freescale, typically based on the ARM design. WHS, on the other hand, will run on standard PC hardware based around Intel or AMD x86 processors.

[...]

Linux is the obvious choice since many distributions are free and its reliability is well-documented. Installing, configuring and maintaining Linux can be a time consuming hassle though, even if you're already familiar with the OS.

[...]

Overall, Asus's Home Server TS Mini is a disappointment. The hardware's clumsy design makes adding or replacing a hard disk more difficult than it has to be. Asus' WHS plug-ins don't add much value either, although these can always be updated in future or just replaced with alternatives of your choosing. The sluggish performance is particularly disappointing though, limiting the TS Mini's usefulness.

All of this is a shame, since the WHS OS clearly has much potential, but it's not without its flaws either. It's disappointing that almost three years after its launch, there aren't easily accessible printer sharing options or RAID support.

There is one area where the failure of Windows Home Server is similar to that of OOXML. According to this new post from Rob Weir, Microsoft Office has data corruption problems that affect OOXML.

In this post I take a look at Microsoft’s claims for robust data recovery with their Office Open XML (OOXML) file format. I show the results of an experiment, where I introduce random errors into documents and observe whether word processors can recover from these errors. Based on these result, I estimate data recovery rates for Word 2003 binary, OOXML and ODF documents, as loaded in Word 2007, Word 2003 and in OpenOffice.org Writer 3.2.

My tests suggest that the OOXML format is less robust than the Word binary or ODF formats, with no observed basis for the contrary Microsoft claims. I then discuss the reasons why this might be expected.

It is not exactly surprising because OOXML has corruption written all over it, but Microsoft's crimes aside, there are clearly some technical deficiencies. Microsoft does not build software for robustness. The London Stock Exchange found this out the hard way [1, 2]. People inside Microsoft know this too. ⬆

'Eller and his team had written what they felt was some very good Windows code. When Konzen came over he appeared to want to counter this impression—he told the Windows team their code was garbage. They had completely misengineered the system, he said.

'"These Apple guys really know their graphics," Konzen told Eller.

'"They're better, faster, and simply easier to use. You chimps working on Windows don't have a clue."'

--Barbarians Led by Bill Gates, a book composed
by the daughter of Microsoft's PR mogul

Comments

Robotron 2084

2010-02-22 23:01:59

Naturally, most peoples predictions are based on where they stand on various issues. However, the extremist makes no predictions toward their enemy. Rather they simply declare guilt when the facts are lacking, or even in spite of them.

I have no ill feelings toward Microsoft, so I first suspected (not declared) the problems at the London Stock Exchange were due to the "TradElec" software designed by a company called Accenture. Steven J. Vaughan-Nichols, a man after Roy's own heart, citied a ".NET Crash" in one of his headlines despite the fact no one reported that to be the problem. Chalk that up to blogger vs journalist, I suppose.

According to a follow-up article at computer world: the LSE said it would not point the finger of blame at Accenture. It said TradElect was not at the source of the problem. Later a networking problem was given as the cause of the outage.

Jose_X

2010-02-23 03:38:24

What people need to note as well is that most and/or many more stock exchanges use Linux. The one or among the very few notable ones that used Microsoft technology ended up with network problems on various occasions and also was slower. At least these are details I think I remember.

SJVN and others also jumped on the LSE failures in part because Microsoft had been selling the LSE as a prime reason to go with MSdotnet. And it's not everyday that a major stock exchange has the kinds of problems LSE experienced.

It would be surprising, then, for Microsoft not to have helped out that situation in some way (eg, providing consultation, specialized optimizations or libraries, discounts, etc). It would be even more surprising to think Microsoft's software had nothing to do with the problems. Keep in mind that Microsoft already has a reputation, and people testing both platforms for high performance and reliability tend to go with Linux -- even when costs are not a major constraint (although if you can get more software for less, you can afford more hardware). Don't overlook this either: http://www.top500.org/stats/list/34/osfam . Of the top 500 supercomputers in the world, Microsoft has 5 spots while Linux has 446 (including some systems that trounce Microsoft's best entry).

BTW, note that the LSE themselves didn't say "let us fix the bugs of the code". They bought a new totally different solution that avoided Microsoft technology entirely. And they paid decent money for this (functionally) duplicate technology. So they didn't try to spend money (even with a healthy budget) to improve their existing situation, nor did they try simply to adjust their future purchasing while retaining legacy. This speaks volumes.

And it is also not surprising Microsoft would not be shamed in public. Microsoft might be a desktop partner or even sue for defamation (eg, Microsoft uses NDAs liberally and is a very aggressive "partner" with supposedly huge bank accounts and levers).

Given all of this, let me repeat, it seems likely that the underlying technology has some problems believed to be unfixable in the near term. I mean, aren't there other major dotnet vendors that could have stepped in? Microsoft, themselves, could have stepped in to save the day if their was a near-term salvation, but they didn't.

My point is that the benefit of the doubt can't keep lying with Microsoft over and over against what statistics suggest (even if there could be reasons).

As for "extremists" biased towards Linux and "consequently" clearly wrong to a large degree:

Microsoft has been at their unethical games and has been producing software with various types of glaring flaws (at least when it comes to various types of applications) for many many years. It's unfair to criticize people here as if they did not have good reasons to be upset. I would love a compromise (eg, some give Microsoft more credit while others give them less), because among the compromise would have to exist a condition where desktop market share lies much more towards Linux than is the case today. Linux is very comparable to Windows overall and is a heck of a lot cheaper over the years. Also, most of its "faults" have more to do with the trade secret nature of and levers being exercised by the dominant platform.

And, yes, Microsoft software can/does lead in a number of ways, but you'd expect such a wealthy company as Microsoft (and its partners) to spend on polish and marketing much more than the traditionally small Linux companies or users (unfortunately, at the expense of other things). With more market share, and hence money, moving to the Linux side, Linux will clean up the loose ends much more and move forward even faster. Linux has at least gotten to the point that a major new ambitious and promising player has emerged to compete on the desktop (Google).

Jose_X

2010-02-23 03:55:52

Let me add that a "network problem" can in fact be a dotnet failure (or multiple ones or a run into limits of the software, etc).

In any case, I'm not denying that there is jumping to conclusions by "extremists"; however, over time it won't just be bloggers and extremists. Neutral customers also will start to avoid certain companies or software or classes of software after they get burned one time too many. This has worked against both Linux and Microsoft.

The main difference is that Microsoft has much more lock-in and huge revenue streams (anchored by monopolies held in part by unethical/illegal practices) so they can overcome huge mistakes [Microsoft themselves admitted this much when it came to their software's lock-in keeping their revenues in place: http://boycottnovell.com/2009/06/25/ms-definition-of-embrace-and-extend/ "You could argue that the [Windows] API is too hard to use, that not every library is as fast as it should be, or other serious imperfections... it is so deeply embedded in the source code of many Windows apps that there is a huge switching cast to using a different operating system instead... In short, without this exclusive franchise called the Windows API, we would have been dead a long time ago."].

Meanwhile, Linux is open, so Linux overcomes mistakes because there is no single or terminal point of failure, and both new and old users/developers/sellers simply keep improving it and adjusting their techniques.

Robotron 2084

2010-02-26 04:18:41

Your first reply was fine and presents some interesting points without going off-topic too soon, but still WAY too long.

Besides that, I wouldn't expect you to agree on the problems with Linux/FOSS extremists. Your past history on this website and others coupled with the extensive amount of time you put into Linux easily puts you into that category. Any views you have would be heavily tainted.

your_friend

2010-02-23 07:19:34

I'm not sure why anyone would think NAS is a chore.

Installing, configuring and maintaining Linux can be a time consuming hassle though, even if you’re already familiar with the OS.

The same and worse could be said of WHS if a person were to try to build it from scratch themselves. Installing and configuring a NAS with GNU/Linux is no more difficult than installing GNU/Linux for any purpose. If there are problems, it's thanks to Microsoft's continued efforts to make hardware painful. Once it's up, there's next to no maintaining to be done, it just works, and there's a wealth of good applications to learn for your tasks. Windows, on the other hand, is notoriously difficult to install from scratch and sucks life once you have it. The fact of the matter is that most people have a GNU/Linux box already - it's their WAP, their TV, their phone and many other appliances they can take for granted because they require less upkeep than a toaster. Microsoft's recent patent attack on a NAS maker shows that Microsoft is running scared of it's technically superior competitors. Free software is easier no matter how you get it, but it is zero effort when it comes out of a box working they way you want. Microsoft is not going to catch up anytime soon.

Roy Schestowitz

2010-02-23 22:30:23

I think Melco is a good example here. Microsoft reportedly sued them to sign a patent deal.

uberVU - social comments

2010-02-25 01:31:15

Social comments and analytics for this post...

This post was mentioned on Identica by schestowitz: What #Windows Home #Server and #OOXML Have in Common: They Corrupt Data http://boycottnovell.com/2010/02/22/windows-data-corruption-server/...
Yuhong Bao

2010-02-22 20:17:34

These are different kinds of data corruption, however.

Roy Schestowitz

2010-02-22 20:26:58

Of course.

Also, formats and software are not exactly the same.

Roy Schestowitz

2010-02-25 21:34:18

I notice that opponents of ODF are still using diversion to avoid the truly important issues (the "IBM is evil" defence). Some hours ago I received the following:

Try this: Launch OOWriter. Go to Tools > Options > Load/Save > General. Set "ODF format version" to 1.0/1.1. "ODF Format Version" to "1.0/1.1 (OpenOffice.org 2.x)". Notice that a warning has just appeared below: "Not using ODF 1.2 may cause information to be lost."

So when it comes to ODF 1.0/1.1, OOo does exactly what Microsoft Office 2010 does, throws a warning that using the format can result in data loss. The recommended "ODF 1.2" of course is not yet a standard and is still evolving. (The international standard is ODF 1.0 Second Edition.) I can guarantee you that this will be the situation for years to come. I'm in touch with folks at JTC 1's SC 34 and there is no way that ODF 1.2 will be allowed to slide through JTC 1 without very substantial revision. ODF 1.0 got a pass at JTC 1 largely because many national standardization bodies wanted it approved to put pressure on Microsoft to submit its own formats to standardization. That goal was achieved, and a significant number of N.B.s want the ODF interop mess cleaned up in ODF 1.2. The OASIS ODF TC isn't cooperating on that so the work will happen at JTC 1. ODF 1.2 will emerge in altered form.

Meanwhile, ODF has no interoperable implementations. Zero, zilch, nada. Even the OOo clones have significant loss of data when exchanging documents. Take a look at the quotes and stats I collected in a comment here. http://www.nwprogressive.org/weblog/2009/01/review-free-openofficeorg-writer.html?ext-ref=comm-sub-email . It's a very sad situation.

Now try this in OOWriter. File > New > HTML Document. Now compare the menus with the menus when the format in use is .odt. Whole bunch of features available in .odt that aren't even on the menus when the format .html. That's because Sun implemented a *compatibility mode* for HTML. No data loss so long as you are working only in HTML with a document originally created in that format.

Both Microsoft and Sun/IBM could have implemented a compatibility mode for ODF 1.1 that makes features unavailable in the UI that aren't supported by ODF 1.1. But they did not do so. And the only implementation of ISO/IEC:26300 with noticeable market share is Google Docs, which uses so few core features that it wouldn't matter if they were throwing in the ODF version file header for any version of ODF.

I'm not all that disturbed by Microsoft's failure to implement a compatibility mode for ODF 1.1. Why? Because there are no interoperable ODF implementations to begin with and Microsoft isn't hoisting the FOSS banner over its products. I am far more disturbed by Sun/IBM's failure to do so because they're the ones out there waving the FOSS and open standards banner and claiming that ODF apps are already interoperable, the latter of which is a bald-faced and colossal lie.

Now you might wonder why Sun and IBM would deliberately avoid well known compatibility mode techniques to provide only lossy support for ODF 1.1. The answer is rather simple: forced upgrades, the business model that Microsoft developed into a fine art with MS Office.

Although never reported anywhere to my knowledge, Sun undeniably granted IBM the rights to use the OOo 3.x code base in IBM's proprietary apps. See e.g., http://www-03.ibm.com/press/us/en/pressrelease/25912.wss ("the Symphony roadmap for 2009, when future generations of Symphony will be developed entirely on the ODF 1.2 and OpenOffice 3.0 software code base, bringing it in line with the newest OO technology").

Strong circumstantial evidence pins the date of the deal somewhere around August of 2007. So despite OOo's LGPL licensing for the rest of the world, OOo 3.x is soon to have several proprietary clones, StarOffice, Lotus Symphony, and IBM's Notes/Domino product line. (Are you getting an inkling yet about Novell's reasons for refusing to submit patches to OOo under the Sun contributor's license?) That assumes that Oracle does not launch still other proprietary clone products but there are strong indications that at least one is in the works, a cloud computing clone based on the OOo code base that implements JavaFX code.

This situation raises the incentive for OOo to become the crippleware loss leader to get customers to migrate to the proprietary, non-free as in beer clones where the companies charge money for licenses. How might that be accomplished? Very simply, by embracing and extending the OOo flavor of ODF. As OOo users receive documents containing ODF extensions that break compatibility, they are incentivised to switch to the proprietary apps where there is no loss of fidelity. This is the heart of the method that Microsoft used to force users to upgrade to the latest and greatest flavor of Office. And Sun/Oracle/IBM have the same incentives to do so that Microsoft had, the revenue stream from forced upgrades.

Now consider the monkeywrench thrown into that strategy if OOo users could simply stick with ODF 1.1 without fear of data loss because a compatibility mode for ODF 1.1 was implemented in OOo. Say a government mandates that all of its documents be generated using ODF 1.1. Bye-bye, forced upgrades. Governments could simply tell the public that if they want their documents sent to the government to be processed without data loss, turn on the ODF 1.1 compatibility mode in OOo.

Likewise, consider how Sun might force users of proprietary StarOffice to do paid upgrades to the latest and greatest StarOffice. Obviously, one can't leave OOo capable of writing to ODF 1.1 without data loss, else customers simply switch to the free OOo instead. No, ODF 1.1 support must be lossy, setting the stage for the StarOffice extensions to ODF 1.2 that force the switch from OOo 3.x+ to StarOffice. So no ODF 1.1 compatibility mode for OOo.

We now stand more than 50 years from the date IBM claims it invented the word processor. (There are valid claims to priority by others.) But in those five decades, word processor users still have only the same option they have always had, whose word processor codebase they want to be locked into. And with scant exceptions, the users must also choose to endure forced upgrades.

Ironically, the only word processor I know of that doesn't force upgrades is nearly dead from lack of market share, WordPerfect. Since version 6.0, all versions of WordPerfect have been able to exchange documents with each other without data loss using a compatibility framework that allows unforeseen new features to be preserved when the document is processed by older versions. One advantage OOXML has over ODF is that OOXML includes a compatibility framework very much like that used by WordPerfect. It's generally referred to by its acronym, MCE. However, it needs much more work to enable round-tripping of documents without fear of data loss. I do not suggest here that the world should switch to OOXML. Generally, it's a crap standard and is patent encumbered. The only realistic hope is to fix ODF.

But you might take a look at Rob Weir's recent opposition to adapting MCE to ODF, in the comments here. http://broadcast.oreilly.com/2010/02/microsofts-ooxml-extensions-fo.html . His excuses are bullshit and he ends with yet another ad hominem attack, his favored tactic when he can't respond on the merits. Kill the messenger bearing ODF bug reports. That's been IBM's entire strategy for keeping ODF grossly under-specified.

In truth, there is no ODF standard. There is a partial specification but it is so grossly underspecified and elastic that the only way to create interoperable implementations is to clone the same code base. Creating interoperable implementations will never happen under the strategy IBM professes; IBM's strategy is just about creating the illusion that serious interop work is under way. It is not. I am thankful that Microsoft chose to implement its OOXML spreadsheet formulas rather than copying OpenOffice.org's. That incident forced realization on a host of people that ODF is grossly under-specified, which should translate into the specification being prepared sooner. And no damage was done thereby. There are no interoperable ODF implementations anyway and Microsoft has said it will implement OpenFormula once it is stable.

I understand that you view OpenOffice.org as vital to FOSS. I do not share that view. To me, OOo soaks up just about all available developer mindshare that might otherwise be devoted to developing an office suite and related apps that are based on true open standards and not controlled by a sole vendor. In my mind, OOo is doubly evil because the specifications necessary to interoperate with it continue to be concealed.

But regardless of how you feel about OOo itself, I do wish that you and other OOo advocates would clearly delineate your support of OOo from your support of ODF in its present form and direction. ODF is in need of drastic revision if we are to build a connected world of FOSS applications. ODF is a giant bug in the free information infrastructure. And it is the constant drivel about ODF being an open standard that creates the public shield wielded by IBM and Sun to keep from having to repair both ODF and OOo to enable interoperability.

Consider the fact that we could have had OOo and Microsoft Office talking ODF to each other without loss of data several years ago using the OpenDocument Foundation's Office plug-ins, but were blocked not by Microsoft, but by Sun and IBM, who refused to make the needed minor changes in ODF and to reprogram OOo so it wouldn't trash the needed foreign elements, attributes, and content. And we were blocked by IBM and Sun a second time after we let it be known that we could accomplish the same results using the RDF metadata support coming in ODF 1.2. IBM and Sun promptly amended the ODF 1.2 spec to allow the conformant destruction of the needed RDF metadata.

Or consider this: I pushed the ODF TC to make compatibility modes mandatory for implementations of ODF 1.2. I got evasion from Rob Weir rather than action. Now IBM beats up on Microsoft for what in essence is an argument that Microsoft should have implemented an ODF compatibility mode in Office. I hope you comprehend the irony and double standard involved.

IBM and Sun management are every bit as evil as Microsoft's. There are only two significant differences: [i] IBM and Sun have a much smaller market share; and [ii] they behave even more irresponsibly than Microsoft because they don't have antitrust regulators looking over their shoulders.

In sum, if you care about FOSS and ODF, I hope you might worry more about the foxes that are already in the FOSS chicken house eating their fill than you worry about the foxes in Redmond, Washington. It's the foxes already in the chicken house that are doing the real damage.

Jose_X

2010-02-22 18:58:28

Rob's experiment was to attempt to test Microsoft's various claim that OOXML is more robust to errors than is .doc.

To conduct the tests, Rob purposely corrupted various files in various ways and contrasted OOXML, .doc, and ODF. He looked at how MS Office and OpenOffice handled these corrupted files.

Overall, it appears Microsoft's claim might not hold up. Also ODF and OpenOffice did fairly well.

LinuxToday linked to this yesterday and there are some comments there that highlight some of the results. http://www.linuxtoday.com/infrastructure/2010022100435OSMSSD

What Windows Home Server and OOXML Have in Common: They Corrupt Data

Comments

Recent Techrights' Posts