Simplify the workplace with IT data centre consolidation (part 2)

Data centre consolidation is all the rage again, but today's payoff is more about agility than mere cost savings. Here's how a range of organisations have managed to shrink IT -- and stay in one piece.


This is the second part of a two-part article. The first part was published yesterday, here

Overcoming objections

Even with broad involvement, however, there’s often resistance, particularly when it comes to moving servers and applications to a central location. “Everyone wants to hug their servers before they go home,” Charron says.

There’s also fear of major disruption, particularly in the case of application consolidation. “We decided to consolidate at the height of the re-finance boom, when people were very busy,” First American Title’s Godec says. “The idea went over like a lead balloon.” First American Title’s experienced escrow officers had prided themselves on their efficiency in getting documents produced in a timely manner. “Now they would no longer do that work. I think they felt outsourced.”

Resistance escalated during the roll-out. “They were screaming bloody murder.” It didn’t help that the more experienced employees were having a harder time adjusting to the new system than those on staff for two weeks, who were more productive sooner. “Two weeks later, though, they were all thrilled,” says Godec, who adds that some quit but many have come back. Godec emphasises that a big success factor in application consolidation is focusing heavily on the human element. “Make sure you over-train and are right there with the end-users during the change.”

Resistance can also come from IT, but it’s often not as serious as one might expect. “There was the usual fear of job loss, and we worried that we’d lose key skills too early,” says John Carrow, vice president of strategic client development and former CIO at Unisys, which has been consolidating in waves since the late ’90s. “We thought about retention programs, but it turned out we didn’t need them. The techie work culture got caught up in the technology and understood the learning opportunities it created.”

Nonetheless, consolidating soon after a company merger can exacerbate tensions, as it did at Advantage Sales and Marketing, a leading marketer of consumer packaged goods. “It was really tough,” says William Hiatt, the company’s national technology director. “We were the big, bad evil corporation coming in and changing everything. A lot of IT left so we had to hire new people even while we were cutting to reduce the budget.”

Many companies make a concerted attempt to redeploy IT workers. At KMD, a Danish software developer for local governments that used VMware’s ESX Server to consolidate three data centres to one, many of the same staff continued managing servers and applications remotely in virtual teams after the consolidation. “We needed to make sure each of the three departments was well represented on the technical side,” says Alan Madsen, a systems engineer at KMD.

But cuts are often inevitable. “If you’re going from 5,000 servers to 2,500, you’re obviously not going to need all those system administrators,” IDC’s Bailey says.

Taking baby steps

Just about everyone cautions against being overly ambitious. Tackle consolidation in small to medium chunks, they say, and use what you learned for the next wave. “We worked with Dell to migrate one ERP application from Unix to a Dell cluster running Linux. Once that pilot was successful, we decided to take out a blank sheet and really try to re-architect,” TRW’s Drouin says.

Yet TRW left PeopleSoft running on HP-UX. “We had thousands of users running those applications,” Drouin says. “Changing those platforms in the middle of a transition was just too big a bite for that first stage.”

Advantage Sales and Marketing’s first step was messaging. “E-mail was the low-hanging fruit,” Hiatt says. “It was so antiquated, so we started by building a unified messaging system and active directory domain, which was well received. Then we started focusing on virtualisation and consolidating database servers.”

Unisys started by consolidating regionally in the late ’90s. The second wave was to consolidate to a single global data centre in Minnesota. Only then did the company start focusing on applications, such as consolidating to Oracle ERP for financials and Siebel for CRM. Even then, it retained two instances, one for the U.S. and one for international users. “As it turned out, it was a smart move because we need the redundancy for business continuity,” Unisys’ Carrow says. Finally, the company started consolidating messaging under Exchange.

“It’s one thing to talk about consolidating hardware and another to talk about databases and applications, which are much more complex and more visible,” IDC’s Bailey says. “If you take on too much, you get cost overruns and a lot of pressure to just get it out, so the project ends up falling short.”

AT&T’s reputation rests so heavily on faultless reliability, that it often starts consolidating by simply replicating all existing systems at the new location. “Collapsing the footprint is much more easily achievable than collapsing servers and applications, and it has a big return on investment in terms of management, efficiency, and real estate savings,” says Dave Roberts, senior technical director of consolidation, planning, and relocation at AT&T.

First American’s Jafa advises not to get too obsessed with standardisation for standardisation’s sake. “You want to serve the customer need for things like scalability and reliability, not force them to suddenly change their cost model or negatively impact them with a limited selection of standards.”

Plan ... then plan some more

After the strategy has been decided, execution requires very careful, detailed planning. “You really need to discover all the details, all the relationships among systems and networks that you’re going to move around, and do lots of planning for everything that might happen,” Jafa says. “And you need to really focus on the lessons learned with each consolidation and use them to develop standard approaches, processes, and implementation plans for the next one.”

Business units should also be involved in the logistics planning. “We’d turn over many of the plans to the business units. The ultimate go/no-go decision was always made by the business, not by IT,” Jafa says. Similarly the business units were the ones that decided whether things were functioning the way they should.

Because it had small windows for each actual move, TRW built in specific drop-dead milestones. “If it got to be Saturday at 10 pm and we weren’t where we expected to be, we had to roll everything back, no questions asked,” says Anil Goindi, senior IS manager at TRW. Dry runs in a test environment were also essential.

For maximum reliability, AT&T replicates data to the target location and updates both systems in parallel for a full six months before going live at the new location. “We’ve learned that data movement is absolutely key to success,” Roberts says.

Getting help from vendors is also a big ingredient for success. “You have to tag along with the tech experts for each system,” First American’s Jafa says. “We got a lot of help and used a lot of best practices from our primary partners Fujitsu, HP, and IBM.”

Consolidation is hard, and some disruption is inevitable, but when it’s done right, the benefits are real. First American Title estimates $5 million in annual IT savings, with a total of $100 million in annual savings and revenue due to accelerated business processes. “We’ve integrated our lenders’ systems into our centralised system so we can have all the lenders’ instructions in before closing. Before we had to kill time feeding the family cookies in the waiting room while the escrow officer was still scrambling to acquire, key, and print the most up-to-date, final information. None of our competitors can do this.”

Statistics Canada estimates savings of $2.9 million in the first four years, and despite massive storage growth from a recent census, it has had to hire only one more full-time employee. Unisys estimates shaving $50 million out of the run-rate cost of IT operations, while TRW and First American were able to use saved space and decommissioned servers to build a first-rate disaster-recovery infrastructure.

Perhaps the most dramatic benefit, however, is agility. “After we realised how much more agile we were, and how successfully we had made all the changes, we became much more creative and confident about what we could do,” TRW’s Drouin says.

This is the second part of a two-part article. The first part was published yesterday, here

Find your next job with computerworld UK jobs