Forrester Research last year asked enterprises in a survey how many of them had built an internal cloud, and about 5% said they had, according to analyst James Staten. But when asked to define the internal cloud, IT executives typically replied "my VMware environment," Staten says.
In reality, adoption of internal clouds as defined by Forrester is less than two percent of enterprises, and vendors are just beginning to provide the proper tools necessary to build them, he says.
"The big challenge we see is most enterprises are not organisationally ready to deploy an internal cloud," Staten says. "You have to be ready to share resources among business units. Most enterprises are not."
Additionally, customers need extensive experience with virtualisation and automation technologies, and must be comfortable with letting users provision their own services through a self-service portal.
Cloud networks deployed by an enterprise for its own users are often called "private clouds", but that phrase has been co-opted by a few vendors to describe certain forms of external hosting services. Therefore, Forrester and other analysts have decided to use the phrase "internal cloud" to describe cloud networks that exist entirely within a customer's own IT infrastructure.
In a report titled "Deliver Cloud Benefits Inside your Walls," Forrester's Staten writes: "Architecturally, an internal cloud isn't that different from a virtualized scale-out infrastructure in today's enterprise. Both are composed of a collection of x86 servers topped with either a grid engine or a virtual infrastructure based on hypervisors."
But internal clouds have several key elements that go beyond virtualised infrastructure. For example, an internal cloud lets developers deploy applications to the cloud via a self-service portal, without any involvement from a server administrator, Forrester says.
Additionally, internal clouds have an automated workload distribution engine (such as those found in grid networks) to determine the best placement of new workloads and optimise the pool of virtualised computing resources to make room for more applications.
Moreover, internal clouds are multi-tenant, sharing resources across business units and divisions within a company that may not share computing equipment today, Staten says. "To account for the use of the virtual pool, internal cloud infrastructures usually provide a method of metering and tracking resource use that feeds chargeback or direct billing for the resources consumed," he writes.
In her research, Yankee Group analyst Agatha Poon defines three key elements of an internal cloud: the network, process and corporate culture.
On the network front, reliability and security become more important than ever in a cloud environment because of the reliance on a consolidated pool of processing and storage, Poon says. In addition to strict access controls, the network must be robust enough to survive a performance hit that could be caused by virtualisation.
"Running virtual machines on a single server and accessing them via Gigabit Ethernet could overwhelm the network, leading to degraded performance," Poon writes in a report titled "Rebuilding Corporate Data Centers as Private Clouds." "The situation is exacerbated as enterprise users expect to access business applications anywhere, anytime, resulting in applications being dynamically rerouted on the fly to meet specific requests."
In terms of process, automation is key for dynamically scaling IT resources and enabling quick provisioning and deprovisioning of computing instances and applications.
"Automation is a key feature used in the cloud computing environment to orchestrate the interplay between the physical and virtual components required to build an internal cloud," Poon writes. "As the number of virtual machines per physical server continues to swell, it becomes very cumbersome for enterprise IT to manually manage processes such as installing and configuring the OS and doing patching and upgrades for ongoing support."
Adequate monitoring tools and policies are also needed to guarantee service availability and performance, and meet regulatory demands. Key vendors in this area include IBM, VMware, Neustar and AccelOps, according to Poon.
Enterprises also need to tackle the corporate culture in order to effectively deploy an internal cloud. Users often resist changes to the status quo, but that does not mean IT should avoid innovation.
"When a company decides to build an internal cloud to share a pool of computing resources for the deployment of user-specific applications, it should provide users with a familiar interface for accessing resources so that little or no training is required to simplify the transition," the Yankee Group report states.
While numerous vendors have tackled one or more aspects of the cloud-building process, several say their platforms alone are robust enough to build a private cloud. VMware has dubbed the latest release of its virtualization software a "cloud operating system," while cloud building software packages are also available from Platform Computing, 3tera, Eucalytpus Systems and other vendors.
VMware's vSphere isn't a complete cloud platform yet, lacking self-service deployment, automated provisioning and billing, but VMware can be expected to bolster these aspects in the future, Staten says.
Platform Computing's ISF software aggregates servers, storage, networking tools and hypervisors to create a shared pool of physical and virtual resources. It is perhaps the most comprehensive cloud building software, according to Staten, noting that it includes a workload distribution engine; an infrastructure aggregation layer; a self-service portal for IT administrators; metering and monitoring; and robust APIs for integration with third-party tools.
SAS, a business intelligence software vendor, is piloting Platform ISF internally to create a self-service portal for developers allowing them to "quickly and reliably provision and deploy compute and application resources," says Cheryl Doninger, research and development director for the enterprise computing infrastructure at SAS.
Pooling together server, storage and networking resources will eliminate inefficiencies in SAS's previous method of deploying custom computing setups to individual R&D groups.
SAS went with Platform ISF because the company needs to support multiple hypervisors and wanted its cloud to include both physical and virtual servers, possibilities not supported by VMware's vSphere. "Even though virtualization technology is getting better, there is still a performance consideration when you move an application from a physical deployment to a virtual deployment," Doninger says.
Eucalyptus Systems, an open source company, provides a Linux-based platform that installs on existing hardware and is designed to let an internal data center operate like the Amazon Elastic Compute Cloud, says Eucalyptus CTO and co-founder Rich Wolski. Virtualisation alone does not give users control over their own resources, Wolski explains, so products like Eucalyptus enable the self-service interface and sharing of resources.
"Users can configure their particular piece of the cloud in exactly the way they want, just like they do with Amazon," Wolski says. "The cloud enables the user to have a great deal more control over the piece of infrastructure they are entitled to use. Without a cloud, the administrator has to control everything."
Private cloud adoption may be low today, but the market is expected to heat up significantly over the next few years. In addition to software-only products such as Platform ISF and Eucalyptus, companies such as IBM and HP are selling hardware/software bundles that work in similar ways, albeit without the benefit of reusing existing hardware.
The analyst firm Gartner predicts that IT organisations will invest more in private cloud services than in external cloud providers through 2012.
"Private cloud services will be a stepping-stone to future public cloud services," Gartner says. "For many large organisations, private cloud services will continue to be required for many years, as public cloud offerings mature."