The importance of high-performance computing was starkly revealed this week by some numbers released by IDC: One in four of all microprocessors shipped today are being installed in HPC systems.
The market has grown dramatically: In 2004, about 16% of processors or 1.65 million, were shipped for HPC systems; last year, it was 3.35 million processors, or just over 26%, reported market research company IDC.
These numbers explain, in shorthand, why Microsoft this week said it is planning a new development effort in parallel computing to make it easier to program for multi-core and large systems. The numbers explain why Hewlett-Packard has put a blade system on wheels for office use that it calls a "supercomputer in a box" and why Intel is pitching its new Xeon Penryn family processor to HPC users.
But if you want to know why HPC matters to these vendors, someone who can explain is Eric Morales, an engineer at golf club maker Ping. Ping installed a Cray XD1 supercomputer three years ago and cut processing time for design simulations from 13 hours to 20 minutes. It turns out that this system was just the appetizer.
"I think we've done as much as we can with what we have, but I feel that we need to expand," said Morales. "I think there is more that we can do."
Morales wants to apply HPC's ability to show, in colourful and exacting detail, what happens to a golf club when it makes contact with a ball. Price and performance improvements help make possible HPC's expanded role in golf-gear manufacturing.
Morales said that while he was walking around the show floor at the SC07 supercomputing conference, this week, he saw systems for $20,000 (£9,759) with the processing power that he paid $100,000 (£48,814) for several years ago. HPC systems "keep getting more and more advanced, but they are also coming down in price," he said.
The worldwide technical HPC server market is expected to hit $11bn (£5.3bn) this year. By 2011, it will be more than $15bn (£7.3bn), an annual growth rate of nearly 9%, according to IDC.
HPC systems run applications, such as fluid dynamics, that need lots of compute power. They use applications designed to run in parallel, which means they can use multiple processor cores and clusters simultaneously. With the continuing development of multi-core chips, the need for applications that can run in parallel is increasing.
That's why Microsoft this week announced a new initiative, along with its updated cluster server, HPC Server 2008, to build out a development environment for parallel applications, as well as make it easier for its customers integrate these applications in their existing environments.
"We're consistently seeing the HPC market growing beyond the traditional focus," said Kyril Faenov, general manager of HPC at Microsoft, to "commercial adoption of HPC systems at a broad level."
But HPC is heavily dominated by Linux and Unix systems. Microsoft began attacking this market in earnest in 2005. Bill Gates spoke at this supercomputing show two years ago to underscore the company's interest in this market.
With the broadening market, Faenov said, computer makers need to make HPC easier to manage and more accessible to a broader range of users and developers. "Parallelism has gone mainstream," he said. Consequently, Microsoft is establishing what it calls its Parallel Computing Initiative, explained Faenov.
Microsoft is building a parallel framework, which will be an extension of .Net. It will have libraries, compiler extension and tools that allow parallel applications to be developed much more rapidly, said Faenov. "We're making .Net an excellent environment for parallel programming," he said.
Faenov said he hopes to preview some of the fruits of this effort in about six months.
Purushotham Bangalore, an assistant professor in the School of Natural Sciences and Mathematics at the University of Alabama at Birmingham, also attended the supercomputing conference. He said, Microsoft, because of its client-side focus, is many years behind in HPC computing and is "just catching up."
Bangalore did not discount Microsoft's ability to influence developers. "They have a way of redefining the world," he said.
Part of Microsoft's strategy also includes working closely with systems companies. HP this week announced a cluster system built on its BladeSystem c3000. It has eight blades and fully loaded can deliver nearly a trillion floating-point operations per second (teraflop) of compute power from a standard wall outlet. The system is 2 square feet and doesn't need to be located in a data centre. One version of the system that HP will soon release is shaped like a tower and has wheels.
The system is priced between $25,000 (£12,212) and $50,000 and can be shipped with HPC applications used in materials and fluid engineering design and analysis.
The server can be built with Intel's new Xeon 5400 quad-core processor, part of the Penryn family built from the 45-nanometer process. Intel announced the chip this week and said it was being targeted at the HPC market, among others.
David Scott, Intel's technical director for high-performance computing, said the chip has been optimised for HPC, with a larger memory cache and better interconnects. He said users can expect an increase in application performance of 40% or more for some applications.
There is another significant trend to point out in all of this, and that concerns workstations. They have been the mainstay for applications such as computer-aided design, but low-cost servers and the availability of HPC applications might change this.
Earl Joseph, an analyst at IDC who studies the HPC market, said that the lowest end of the HPC market has been growing the fastest, and that IDC expects large numbers of scientists and engineers to move from workstations to servers. These smaller servers are about the same price that workstations were a few years ago, he said.