April 15, 2008

What Intel giveth, Microsoft taketh away

The conventional wisdom surrounding the Windows/Intel (aka Wintel) duopoly since the early days of Windows 95 is that performance advances in hardware are quickly consumed by the ever-increasing complexity of the Windows/Office code base.

By Randall C. Kennedy, Competitive Systems Analysis

---

Microsoft Office 2007, which, when deployed on Windows Vista, consumes more than 12 times as much memory and nearly three times as much processing power as the version that graced PCs just seven short years ago, Office 2000.

Despite years of real-world experience with both sides of the duopoly, few organizations have taken the time to directly quantify what my colleagues and I at Intel used to call The Great Moore's Law Compensator (TGMLC). In fact, the hard numbers above represent what is perhaps the first-ever attempt to accurately measure the evolution of the Windows/Office platform in terms of real-world hardware system requirements and resource consumption. In this article I hope to further quantify the impact of TGMLC and to track its effects across four distinct generations of Microsoft's desktop computing software stack.

To accomplish my goal, I'll be employing a cross-version test script -- OfficeBench -- and executing it against different combinations of Windows and Office: Windows 2000 and Office 2000; Windows XP (SP1) and Office XP; Windows XP (SP2) and Office 2003; and Windows Vista and Office 2007. Tests were first conducted in a controlled virtual machine environment under VMware and then repeated on different generations of Intel desktop and mobile hardware to assess each stack's impact on hardware from the corresponding era.

What does this all mean for Windows IT shops? Should they upgrade to Vista and Office 2007? Or should they stick with Windows XP and Office XP or Office 2003? As I've argued in a previous article, "Death Match: Windows Vista vs. XP," most IT organizations will find they can safely skip a generation and avoid Vista altogether. In addition to sending a message to Microsoft that IT won't tolerate bloat-ware, it also buys you time to allow the hardware cycle to catch-up with what will hopefully remain a static software target, or at least a slower-moving one (through Windows 7) -- a way of putting TGMLC to work for you.

About OfficeBench: The OfficeBench test script is a version-independent benchmark tool that uses OLE automation to drive Microsoft Word, Excel, PowerPoint, and Internet Explorer through a series of common business productivity tasks. These include assembling and formatting a compound document and support workbooks and presentation materials, as well as gathering data through simulated browsing of a Web-based research database. (For more detail, see "Cross-generational Windows/Office performance: About OfficeBench.") OfficeBench is available for free download from the exo.performance.network Web site as part of the DMS Clarity Studio testing framework.

The Stone Age: Windows 2000/Office 2000

Back in 1999, when I was working as an advisor to Intel's Desktop Architecture Labs (DAL), I remember how thrilled we all were to get our hands on Windows 2000 and Office 2000 -- finally, a version of the Windows/Office stack that could leverage all of the desktop horsepower we were building into the next-generation Pentium 4 platform. I remember it was also the first time I had a fully scriptable version of the Office suite to work with (previous versions had supported OLE automation only in Word and Excel). Shortly thereafter, the first version of OfficeBench was born, and I began my odyssey of chronicling TGMLC through the years.

First off, let me characterize the state-of-the-art at the time. The Pentium 4 CPU was about to be unveiled and the standard configuration in our test labs was a single-CPU system with 128MB of RDRAM and an IDE hard disk. A joke by today's standards, this was considered a true power-user configuration suitable for heavy number-crunching or even lightweight engineering workstation applications. It was also only marginally faster than the previous-generation Pentium III, a fact that Intel tried hard to hide by cranking up the CPU clock to 1.5GHz and turning its competition with rival AMD into a drag race.

Sadly, I didn't have access to an original Pentium 4 system for this article. My engineering test bed was long ago scrapped for parts, and I doubt that many of these old i840 chip-set-based boxes are still in use outside of the third world. However, I could at least evaluate the software stack itself. Through the magic of virtualization, we can see that, even with only 128MB of RAM, a Windows 2000-based configuration had plenty of room to perform.

During OfficeBench testing, the entire suite consumed only 9MB of RAM, while the overall OS footprint never exceeded 132MB of RAM, roughly half of the available memory. Clearly this was a lean, mean version of Windows/Office. It chewed through the test script a full 17 percent faster than its nearest competitor, Windows XP (SP1) and Office XP.

The Bronze Age: Windows XP/Office XP

The introduction of Windows XP in 2001 marked the first mainstream (not just for business users) version of Windows to incorporate the Windows NT kernel. In addition to better plug-and-play support and other improvements, XP sported a revamped user interface with true-colour icons and lots of shiny, beveled effects. Not wanting to look out of style, and smelling another sell-up opportunity, the Office group rushed out Microsoft Office XP (aka Office 10), which was nothing more than a slightly tweaked version of Office 2000 with some UI updates.

Hardware had evolved a bit in the two years since the Windows 2000 launch. For starters, Intel had all but abandoned its ill-fated partnership with Rambus. New Intel designs featured the more widely supported DDR-SDRAM, while CPU frequencies were edging above 2GHz.

Intel also upped the L2 cache size of the Pentium 4 core from 256KB to 512KB (the Northwood redesign) in an attempt to fill the chip's stall-prone 20-stage integer pipeline. Default RAM configurations were now routinely in the 256MB range, while disk drives sported ATA-100 interfaces.

Windows XP, especially in the pre-SP2 timeframe, wasn't all that more resource intensive than Windows 2000. It wasn't until later, as Microsoft piled on the security fixes and users started running anti-virus and anti-spyware tools by default, that XP began to put on significant weight. Also, the relatively modest nature of the changes from Office 2000 to Office XP translated into only a minimal increase in system requirements.

For example, overall working set size for the entire Office XP suite during OfficeBench testing under VMware was only 10MB, just 1MB higher than Office 2000, while CPU utilization actually fell 1 percent across the three applications (Word, Excel, and PowerPoint). This did not, however, translate into equivalent performance. As I noted before, Office XP on Windows XP took 17 percent longer than Office 2000 on Windows 2000 to complete the same OfficeBench test script. View the overall test results. View more detailed test results at xpnet.com.

Newsletters

Your daily injection of CW news and expert analysis: we’ll bring the news to you.