In April, IBM acquired Diligent Technologies , which specialises in data de-duplication for large enterprise storage systems. The company didn't reveal how much the acquisition cost, but it was a key move in a market that could grow to US$1 billion in annual revenue by 2009, according to research company The 451 Group.
De-duplication systems find identical bits of data in a storage system, treat them as redundant, and eliminate them. So if there are several nearly identical copies of a document, all will be deleted except one copy and the differences that are unique to the other copies.
The Diligent deal was an early move in a year full of de-duplication activity. In June, Hewlett-Packard introduced a suite of de-duplication systems for small and medium-sized businesses and added some features to its HP StorageWorks backup line. And in November, EMC, Quantum and Dell said they would use a common software architecture for data de-duplication products. Dell will enter the de-duplication business next year. It is already a major reseller of EMC gear, under a partnership that in December was extended until 2013.
Data de-duplication can reduce the amount of storage capacity an enterprise requires by as much as two thirds, said ESG's Whitehouse. It has been available before, but this year companies started to integrate it with storage arrays or sell it in appliances, bringing the technology closer to a turnkey solution, she said. They also established data de-duplication as a technology customers could trust, at least for archived material.
"If you eliminate a block of data that somehow negates the value of that data when you recover it ... that's a really scary prospect for some companies," Whitehouse said.
So far, most enterprises are only using it for secondary storage, or the archived information that's backed up for safekeeping, she said. The next step will be to embrace de-duplication for primary storage, the data that applications are using in real time. Users will start to trust the technology enough for that next year, she said. In July, NetApp enhanced its V-Series storage virtualisation products so they can perform de-duplication on primary storage systems from third parties such as EMC, Hitachi and HP.
In late February, enterprise storage giant EMC bought Pi, a provider of software and online services for consumers to keep track of personal information stored locally or online. The deal, which followed the company's 2007 buyout of online backup provider Mozy, was one sign of growing interest in cloud storage.
Handing off personal or corporate data to a third party's hard drives and accessing it via the Internet can be a less expensive alternative to provisioning all that capacity in your data centre or home network. It may be used in conjunction with cloud-based applications, but also just for archiving or disaster recovery, Illuminata's Webster said. In many cases, the cloud-storage service can be set up as a target when data is being backed up. The information can be sent to the cloud only or to the cloud and a dedicated tape backup system simultaneously, he said.
With the economy weakening, cloud storage will be big next year, Webster believes. Paying for additional capacity on a monthly basis moves that expense out of the IT department's capital budget and into its operational budget, which tends to be easier to fund when times are tough, he said. It's also relatively quick because nothing needs to be purchased or installed, he added.
A related option, managed services, may also take off in the coming year, Webster said. While keeping their own storage systems in-house, enterprises can pay a vendor such as Brocade or IBM to manage it for them remotely. The vendor can monitor alerts through an appliance at the customer's site and respond if needed. If IT staff needs to be cut back, this may be one way to maintain service levels to the rest of the company, Webster said.