For outsourcing customers, benchmarking is billed as the great equaliser. Whether an IT leader is thinking about outsourcing, insourcing IT work performed by a external provider or renegotiating an existing contract, a third-party benchmark can provide pricing data that purports to put the buyer on a level playing field with the giants of the outsourcing industry. In fact, most outsourcing contracts contain a benchmarking clause that grants customers the right to benchmark their deals under certain conditions.
The drawback to third-party benchmarking is that it's expensive and time-consuming: It can take several months and cost some outsourcing customers hundreds of thousands or even millions of dollars.
Outsourcing consultancy Alsbridge wants to change the pricing model for benchmarking. CEO Ben Trowbridge says historically, only the biggest outsourcing customers performed benchmarking because it was so expensive. "We think if you radically change the price, customers will do it more often," he says.
To that end, Alsbridge is pushing its ProBenchmark service, which it says can provide an accurate pricing analysis of IT services in two to three weeks at a price that's 70 percent lower than the traditional, labor-intensive benchmarking provided by its competitors. At the heart of the offering is Alsbridge's ProBenchmark software, which it acquired when it bought boutique benchmarking provider Nautilus Advisors in 2008.
The ProBenchmark tool works by taking input on the outsourcing customer's IT environment (its service level requirements, number of seats, whether offshore outsourcing is permitted, for example), and analyzes it using proprietary algorithms. The software then computes an average market price.
ProBenchmark vs. Traditional Benchmarking Services
Alsbridge's approach represents a shift in the way the handful of vendors, such as Gartner and Compass Management Consulting, that dominate the $400 million IT services benchmarking industry, do business.
"Typically the benchmarking guys will collect as much data from the customer as they can. The thinking is that if we collect everything there will be something in there that we need," says Howard Davies, who joined Alsbridge as director of benchmarking operations in 2009 from Compass.
In reality, notes Davies, benchmarkers only need 20 percent of the data they collect. The problem is that they don't know which 20 percent. So they take all of the data they collected and model it manually in spreadsheets.
"We've turned that [process] upside-down," adds Davies. "We start with only the data that we need, the things that really drive pricing. And the modeling is transparent so customers can have confidence in the answer."
Most custom benchmarking calculations are done behind the scenes. Providers rarely reveal their methods. "I suspect that is because a lot of what the public buys as 'normalization' [of pricing data] is actually 'fabrication,'" says Phil Fersht, founder of outsourcing analyst firm Horses for Sources.
While Alsbridge executives describe their tool as transparent, the user can see how adjustments to input affect output, their back-end algorithms, for which they're attempting to secure a patent, are just as secret as a traditional methods.
The Roots of ProBenchmark
Adam Strichman began to work on building a better mousetrap for the benchmarking industry in 2004, before he sold his six-person Nautilus Advisors to Alsbridge in 2008. "At that time, the [benchmarking] method was essentially a process which built a custom benchmark each time it had to be done," Strichman says. "A dirty benchmarking secret is that you always get a different benchmark result depending on who the consultant is who's doing it."
Strichman and his team developed a calculation engine driven by market data and standard price curves and adjustments based on outsourcing prices. (Strichman left Alsbridge shortly after the Nautilus acquisition for personal reasons. "Benchmarking is a tough business," he says. "Hard to do, hard on you.")
Since then, Alsbridge has refined the ProBenchmark tool and updates it quarterly with new pricing data from its customers. It sells subscriptions to the tool and uses it as the foundation for much of its consulting and third-party benchmarking work. The consultancy still provides traditional custom benchmarking as well.
Despite a history of aversion to the traditional benchmarking process , vendors are the primary customers of the ProBenchmark tool. Two thirds of the top twenty providers are subscribers, says Davies.
By contrast, outsourcing customers are unlikely to use the tool directly. "Anyone can log on and use it, but there is a big learning curve, so only power users really get the benefit from using it on their own, those clients who need lots of scenarios like vendors," says Strichman. "Smaller users would likely never subscribe."
What ProBenchmark Customers Say
Linda Kinder, vice president and CIO of Kellwood Company, engaged Alsbridge to use the ProBenchmark tool to determine if the company should insource the applications work it had been outsourcing to a major IT service provider. "The consultants used the tool and reviewed outputs with our team," explains Keller, who ultimately used the pricing data to build a business case for insourcing. "We chose this method in the interest of time. We wanted a very rapid turnaround for this phase of our project."
Matt Appel, senior vice president and CFO for the Zale Corporation, hired Alsbridge for similar reasons, to asses an infrastructure outsourcing deal that dated back to the 90s. "Alsbridge used the tool to produce a market price comparison that served as the basis for our renegotiation. The provider accepted the benchmark data as factual, as it was based on a current large database of information," says Appel, who ultimately renegotiated lower prices with the vendor, extended the contract term and expanded its scope. "We don't have any particular expertise in using the benchmark tool so we felt it was beneficial to have Alsbridge run it."
Max Staines, North American president of consultancy and IT benchmarking provider Compass Management Consulting, says real benchmarking requires more than punching numbers into an automated tool. "You get what you pay for, in terms of the quality of the data and the usefulness of the analysis. A truly effective benchmark digs deep to identify the root causes and issues underlying performance gaps, and allows you to take specific actions to address those gaps," Staines says. "What's more expensive: a $100,000 study that drives millions of dollars of annual savings, or a $40,000 study that ends up changing nothing?"
Staines admits that the benchmarking market, like the outsourcing industry overal, has become increasingly competitive and price-driven, and providers must be better and cheaper to win business. "But there's no purchased tool or other silver bullet that makes that possible," Staines says. "It's just a matter of working smarter and focusing on continually refining and getting better at what we have been doing for over 30 years."
Contrary to Staines' statements, Fersht, of Horses for Sources, believes benchmarking tools are essentially databases with a good query function. As such, they are indeed built to spit out numbers. "Experts do some interpretation to justify the price, but at the end of the day, if you know what to input, it will spit out a value, he says. "As long as you are comparing simple IT infrastructure services, a monkey could produce a semi-useful result. That said, because outsourcing deals tend to be different, if you want a higher level of precision, you would need the experts to parse things out and make more detailed queries."
Strichman says there's a natural aversion to automation in industries driven by billable hours. "Alsbridge initially struggled with that model and didn't understand that you can't bill hourly for a software-based benchmarking model," he says. "All consultancies shy away from non-billable development work."
Whether ProBenchmark will shake up the benchmarking industry is unclear. "Everyone hates the benchmarkers. They are expensive, take a long time, and create arguments," Strichman says. "I think the tool goes a long way towards solving the first two [problems with benchmarking]."
The real revolution in benchmarking will come from true transparency, which could spell trouble for the benchmarkers themselves, says Fersht. "Social media and [standardization of] the outsourcing contract process ought to change [things]," Fersht says. "Marrying technology plus self-reporting from customers will eventually put the benchmarkers out of business. The only workable model in the future is a "wiki-benchmark," continuously updated by people who have an interest in its accuracy."
Find your next job with computerworld UK jobs