On the 13 April 1999, a press release appeared headed "Mindcraft study shows Windows NT server outperforms Linux." The summary read: "Microsoft Windows NT server 2.5 times faster than Linux as a file server and 3.7 times faster as Web server." One thing the press release failed to mention was the following, found in the study itself: "Mindcraft Inc. conducted the performance tests described in this report between March 10 and March 13, 1999. Microsoft Corporation sponsored the testing reported herein."
In due course, more details emerged of how Mindcraft had been able to draw directly on support from Microsoft when tuning the system, but had not involved Red Hat, whose distribution was being used for the tests, in the same way. This meant that several important tweaks that would have improved the latter's performance were lacking. Indeed, it later turned out that the tests had actually been conducted in a Microsoft laboratory.
As a result of the firestorm that greeted these revelations, Mindcraft agreed to re-run the tests, this time drawing on help from some of the top coders in the GNU/Linux and Samba communities. But still no one was allowed to be present when the tests were run – hardly a satisfactory, or fair situation.
A third re-run of the test addressed these and other detailed concerns, although by this time, most people knew what the result would be: that Windows NT did, indeed, run faster than GNU/Linux under the various conditions that were used. This was because the benchmarks exposed various unsuspected bottlenecks in the open source system that limited its performance.
There were two results of this Mindcraft incident. The first was that it helped the free software coders to improve the software substantially, by addressing those bottlenecks, and to make it truly able to cope with enterprise-level tasks. The second was more subtle, but in retrospect, actually much more important.
By arranging for GNU/Linux, Apache and Samba to be benchmarked against Windows NT, Microsoft was officially admitting that these were real and direct rivals. After all, there is no point benchmarking things that are not in some sense comparable, and a company wouldn't bother trashing a competitor that represented no threat. This was a major shift from Microsoft's previous stance that GNU/Linux was not up to enterprise-level tasks, and that nobody was using it anyway.
It seems that Microsoft has forgotten this important lesson. For it has put together a three-minute video of customers explaining why they switched from OpenOffice.org to Microsoft Office.
The criticisms made in the video are not really the point – they are mostly about OpenOffice.org not being a 100% clone of Microsoft Office, and compatibility problems with Microsoft's proprietary formats. The key issue is
the exactly the same as it was for the Mindcraft benchmarks. You don't compare a rival's product with your own if it is not comparable. And you don't make this kind of attack video unless you are really, really worried about the growing success of a competitor.
Just as it did in 1999 for GNU/Linux, Apache and Samba, the company has now clearly announced that OpenOffice.org is a serious rival to Microsoft Office, and should be seriously considered by anyone using the latter.