In Foster's vision, these fabrics combine virtualisation, SOA, commodity hardware and open-source software.
One example of these worlds converging is Amazon.com's Elastic Compute Cloud (EC2), a service in beta that provides computer power on demand over the Internet. EC2 allows software and web developers to set up virtual servers, instead of having to buy computers or hire a computer-hosting company. Developers can quickly set up a virtual computer and in minutes or hours add or subtract capacity based on their needs.
Much like Amazon's S3 storage service, EC2 allows developers to experiment and get running with a new service almost instantly, with no capital costs. EC2 costs 10 cents per hour to run a custom server, plus data and bandwidth charges.
Although Amazon doesn't consider its implementation a grid environment, some testers are running compute grids on EC2. Werner Vogels, vice president and chief technology officer of Amazon.com, describes the situation this way. EC2 is a "resource management layer that can run under a grid environment," he said. "Ultimately, we provide the resources for grid computing to exist on top of our service."
Examples of application grids that use EC2's resources include Randall Render Rockets using EC2 as a grid to provide image rendering, enabling customers to have digital images and short movies rendered. Others have built environments on EC2 for digital file encoding, to translate MP2 to MP3 files, for example. "We're using virtualisation technology to make EC2 instances appear like full-blown servers, using dynamic allocation to assign virtual images to replicate physical servers," Vogels explained.
Amazon currently uses EC2 internally to perform efficient resource management. This means that excess capacity on idle servers is used as required by applications and user demand throughout each day. This compares to non-grid environments, in which one application depends on one server or server farm to supply compute power, requiring copious amounts of overhead to cover any potential bursts in demand.
But it's the business opportunity provided by the on-demand compute resources that appeals to a whole range of developers, from web 2.0 developers to those who specialise in advanced enterprise computing development, Vogels says.
Still, despite grid's criticality to underlying applications, Vogels says that in the past few years, the grid concept has been over-hyped, and grid computing's limitations have been exposed. Grid "still requires middleware and resource management capabilities" to make it usable for on-demand services, he explains.
In fact, the biggest challenge to the success of grid computing's evolution is the current lack of standardisation, says Foster. "Different vendors are all trying to compete, creating different methods or features that likely won't integrate easily without standardisation," he notes. Also, would-be resource sharers must also navigate layers of technology to create a grid architecture.
But Foster, considered a father of the grid computing concept, believes the need for grid computing may create sufficient momentum to overcome the challenges. "The grid vision ... of resource federation and resource-sharing across enterprises is something a lot of people are eager to see happen," Foster says.