SalesForce outages show SaaS customers dependence on providers' DR plans

Software as a service (SaaS) applications are generally more difficult to prepare disaster recovery plans for than infrastructure as a service (IaaS) offerings, analyst and consultants say.


Software as a service (SaaS) applications are generally more difficult to prepare disaster recovery plans for compared to infrastructure as a service (IaaS) offerings, analyst and consultants say. And given the recent example of two outages in as many weeks, it's a topic they say customers don't generally pay enough attention to.

SalesForce is the poster child of the SaaS cloud, with its CRM and platform as a service (PaaS) offering named Heroku and On the morning of 28 June a newly discovered software bug caused what the company called a "rare dual failure" in both its storage tier and secondary active standby storage tier, resulting in shared memory corruption and some customers not being able to access their SalesForce systems for as long as five hours. On 10 July 10, manual upgrades at a West Coast data centre the company rents space in resulted in a power failure, knocking out service to some customers for two days.

Outages on the IaaS side, from providers such as Amazon Web Services, are occasional and there are a variety of tips for how customers can prepare for them.

But John Morency, a disaster recovery analyst at Gartner, says there are fundamental differences between SaaS and IaaS applications that makes apps like SalesForce much harder to create a disaster-recovery plan around compared to Amazon.

"In SaaS, the provider is the keeper of the entire application stack," he says. "Management, backup, recovery, you're really at their mercy."

In a SaaS environment, he says, the provider - in this case - manages almost the entire system for its customers, from the infrastructure all the way to the application. That means customers have less ability to tinker with the application and the underlying infrastructure it runs on to make it more highly available. In an IaaS environment, the provider is just supplying the underlying compute or storage hardware while the customer is responsible, in most cases, for the applications running on the infrastructure.

So what are SaaS customers to do?

A basic rule of thumb is for customers to do their homework, Morency says. Know what your SaaS provider's service-level agreement (SLA) ensures for uptime and make determinations of how tolerant your organization would be of downtime. "Before making contractual commitments to any SaaS provider, ensure that you have a complete understanding of the recovery management procedures and managed service levels," he says.

If downtime is not an option, there is a growing industry around recovery as a service (Raas), which are cloud-based systems that provide backups of virtual machines, data and applications. There are more than 50 providers in this space now, Morency says, ranging from big-name IaaS providers such as Amazon, Microsoft, Terremark and Sunguard, to other players such as Bluelock, and HP, along with others. These providers work with customers to create customised backup solutions, mostly on the IaaS side, but sometimes on the SaaS side as well, Morency says.

Robert Mahowlad, director of SaaS research at IDC, says it's difficult to insulate your SaaS environments from downtime compared to what IaaS users can do.

SalesForce, he says, is basically a large database of indices holding customer data. Using the CRM is basically the equivalent of making queries within the database and the application spitting the results back out. Unless a SalesForce customer backs up all of their data on their own premise, within SalesForce or with another provider, when SalesForce is down, access to that customer database can also be down. "Amazon has backup options within the same region, within various regions of the country, perhaps even with multiple cloud providers," he says. "You could conceivably have four or five redundancy levels on the IaaS side. On the application side, that's a little more difficult to do."

There are some steps that experienced SalesForce users can take, however, says Eric Hartye, an independent SalesForce consultant who helps businesses design applications running in SalesForce's PaaS offering. SalesForce offers developer sandboxes, which are meant to be a place for coders to test the application and for organizations to train employees on how to use it, without impacting the "production" data, which is the live version of the database.

In a downtime event, Hartye says the sandbox could theoretically be used by a customer to access their SalesForce customer records and data, if the sandbox has been configured to be a full copy of the production workload.

The Enterprise Edition of SalesForce comes with one developer sandbox with some memory limitations, while the unlimited version of the CRM application comes with multiple sandboxes. Hartye says he hasn't seen the sandbox approach used, but in practice it could work. Overall, he says customers aren't typically thinking enough about disaster recovery. "DR doesn't make money," he says. Normally customers just want to get up and running, and they don't think as much about what happens when the system goes down.

A SalesForce spokesperson said in an email it did not have any comments regarding the outage, other than pointing users to, where the company makes public statements related to the outages, and it explains in more detail the circumstances and next steps related to the two recent incidents.

"Recommended For You"

Oracle's Larry Ellison throws down the cloud gauntlet What's the difference between IaaS, SaaS and PaaS?