Do we still need to migrate data?

Are data integration technologies and master data management sufficient to see the end of large scale data migration projects?


Businesses have long been in search of ways to improve the management and use of the information they hold. Celona Technologies’ Tony Sceales considers the consequences of increased integration and the application of master data management solutions, and asks whether enterprises still need to migrate data?

For a very long time the received wisdom has been that if only enterprises could gain a 360° view of the information they hold – provided that information is also accurate and up-to-date – they could revolutionise the way they do business.

Their customers will be happier, their businesses will run more efficiently, new revenue opportunities will be discovered, new product designs will be improved and campaigns will be better targeted and managed. The problem is finding the road that leads to this converged simplicity.

Today’s IT landscape is complex and it seems as if whatever is done to simplify it, it just keeps getting more complicated and more integrated. And, while we know that IT proliferation and complexity means that operational costs rise, there is always a reason to buy more, better or faster technology.

According to Bloor Research, the average large enterprise implements 4.5 major applications each year. And that’s just the big ones. Add to that all the applications we’ve inherited plus the smaller applications we’re implementing, plus the applications that are self-developed and could even be outside IT’s control, and the total rockets. Now add in the effects of merger and acquisition (M&A) which creates a moving pavement for many enterprises so far as IT rationalisation is concerned.

For all these reasons the proliferation of solutions and the data silos they use shows little sign of abating. Yet the requirement to tie these together so that the enterprise can function effectively is also strong.

In fact, ‘integration’ is arguably one of the most over-used words in IT: we integrate solutions, we integrate channels, we integrate data, we integrate within the enterprise and we integrate with the outside world. We are often found discussing the merits of new ways of integrating – like Web services – or finding new things that need integrating (as with master data management). Our appetite for it is undiminished.

Yet the fallout from increased use of integration is complexity, interdependence and higher operational costs. This has resulted in a situation where most of our IT budget is spent simply maintaining legacy, with less and less money available for innovation (See The Burden of Legacy by Toby Sucharov and Philip Rice). IT may wish to reduce the number of redundant or duplicated applications and databases, but often the rate at which applications are being switched on exceeds the speed at which IT can turn them off.

Quite often IT resorts to making a tactical choice to integrate, rather than to migrate or consolidate applications and data. The reason for such a choice is that migrating applications has historically been difficult, time-consuming and risky. Integration has offered an easier, cheaper, faster and less intrusive method of achieving corporate goals – at least in the short term.

The drivers to achieve high-quality, complete, consistent and consolidated datasets are frequently the desire to improve processes and performance (stimulated for example by a CRM, BI or ERP initiative), the need to comply with new regulation (eg SOX) or the wish to implement service-oriented architectures (SOA) or software as a service (SaaS).

Often it’s only part-way through one of these initiatives that enterprises realise just how challenging their data issues are.

Recently, attention has focused particularly on the improvement of so-called master data, which comprises some of the most valuable data enterprises hold: information about people (customers, employees etc), assets, products and places (office locations, geographic divisions and so on).

There is little new under heaven and earth, and so master data management (MDM) is an extension of previous concepts such as customer data integration (CDI) and product information management (PIM).

Managing master data is challenging for many large enterprises, and it is questionable whether simply integrating datasets is the best way forward. Take, for example, the situation that arises after a round of M&A activity. The enterprise acquires a complete set of master databases from the acquired company, each of which has dependent applications.

"Recommended For You"

Global pulp and paper firm Suzano deploys SAP Hana in multi-million IT project HP launches rival to Cisco certifications