The telecoms industry is barely recognisable from the single product, monopolistic, stateowned utility it once was. Today's dynamic, competitive, multi-product, multi-trillion dollar industry is increasingly mobile and continually changing in response to consumer demand and innovation. As liberalisation continues to spread across the globe and competition hots up, rapid technological change has meant telecoms services are ever more firmly embedded in our lives. These services now underpin both our business and personal interactions, with changes in the telecoms business both mirroring and driving changes in the way we live and in the way we conduct business.
What is becoming increasingly apparent, however, is that we are not simply witnessing an industry transformation - which suggests a change from one state to another - but rather a process of evolution comprising a series of small and large changes of various kinds that stretch from the 1980s, through the exciting state of flux that is the present and on into an unknown future. Telecoms is a rapidly changing, growing and vibrant industry. Amidst all the uncertainty there is only one certainty: change is here to stay.
Publication date: 03/2007The Application Transformation ImperativeA Celona White Paperwww.celona.comCelona Technologies Limited63 Gee Street,London EC1V 3RS,United Kingdom.T: +44(0)20 7566 9100F: +44(0)20 7566 9164Email: firstname.lastname@example.orgThe telecoms industry is transforming, with new data-centric networks, a proliferation of services and increasing competition. The need for speed and the inevitability of change are just two of the factors that telecoms service providers have to grapple with. As a result, telecoms companies themselves are transforming - from technology-centric, network operators into customer-centric service providers.Untitled DocumentPage 1The Application Transformation ImperativeForewordIf you have picked up this white paper and are about to commence, or are in the middle of, a data migration between two or more systems and system architectures then good luck. If the success rate of application migrations as analysed by the Standish Group is at all accurate you have little more than a 25% chance of being on a successful project. The clear message from across the IT industry is that the problem of application data migration is not local to one or two companies, or even to a particular market vertical, but rather it's a problem common to all types of enterprise. Here in telco-land, I regularly come across architects who say that they can't remember a single successful migration. Take Marvin Whizztech of multi-service operator Convergicoms. He's a CIO in hot water. A while back, he commenced a major systems migration programme intent on cutting costs by reducing the application count and overhead whilst improving flexibility by installing a new convergent architecture. He got the CFO Boris Beancounter to sign off the project on the basis that it would enable rapid launch of new products and reduce the cash-to-order cycle, thereby maintaining the competitiveness of Convergicoms and fending off competition from cut-price rival Coms-U-Like. It's now two years down the line and although Marvin has managed to turn off several small applications, all the major systems remain firmly in place, with little sign of significant progress. The first attempt at migrating from some of these systems has already failed; while other migrations are either late starting or are unable to create a 'beach-head' on the target applications. For most projects there is no ability to predict any real progress. The consultants are now recommending turning off the billing system for a few weeks to facilitate the migration.Some of you may be feeling uncomfortable recognition of Marvin's predicament. That's not surprising. What is surprising is that, given the scale of these types of problems, so little attention has been given to solving them and to creating products specifically designed to help. Of course, many software tools have been modified or applied to the problem - data warehouse load tools most commonly - but there has been a complete lack of out-and-out application migration focused products to choose from.Having a specifically developed tool providing technical support for application migration is an important part of the solution, but it is not the entire solution. In addition, business and IT must work together, using practised methods, in order for the tool to deliver. But what's also clear, from work in application migration methodology (undertaken by major SIs and independent experts), is that method is not enough on its own either. To be successful you need both tool and method.The upside to the grim statistics quoted earlier, however, is that the expectation of a successful outcome is so low that if you are able to pull it off then you'll be a hero. So, if you're one of the 25%, congratulations! You've made it. You now have a job for life as the expert on application migrations. At Celona Technologies, our business is assisting telecoms service providers to deliver successful application migrations. We have developed a third generation data migration tool that has already helped tier 1 service providers deliver projects not on time, but ahead of time. We would like to help you become one of the 25% of winners, rather than one of the also-rans. Find out more about us and what we do from our website www.celona.com or email us at email@example.com.Paul HollingsworthCelona Technologies March 2007Untitled DocumentPage 2The Application Transformation ImperativeTop tips for successful application transformationAbstractThe telecoms industry is transforming, with new data-centric networks, a proliferation of services and increasing competition. The need for speed and the inevitability of change are just two of the factors that telecoms service providers have to grapple with. As a result, telecoms companies themselves are transforming - from technology-centric, network operators into customer-centric service providers. This paper considers the factors driving these changes and what this commercial transformation means for internal business processes and the IT systems that support them. New requirements and a new customer- and service-led paradigm is affecting IT investment patterns within telcos seeking to benefit from the new multi-trillion dollar industry that is currently being created. However, while many large traditional operators are currently overhauling their back-end architectures in order to align themselves to the needs of the new telecoms market, there is a major stumbling block in their quest to reinvent themselves - telecoms data.While it is generally recognised that telecoms data used well can offer great competitive advantages, in traditional operators valuable data is all too often trapped in multiple data silos, inaccessible and unusable. Unlocking its potential is, however, far from risk free using traditional methods. But the entry into the market of increasing numbers of competitors, who are unhampered by legacy data issues and adept at using data to their advantage, mean that large operators have little choice but to act now.This paper looks at the three generations of data migration methods, analysing the risks presented by traditional methods and showing how new next-generation data migration approaches can unlock the benefits of data while also mitigating the risks.The change drivers: why transformation is necessaryThe telecoms industry is barely recognisable from the single product, monopolistic, state-owned utility it once was. Today's dynamic, competitive, multi-product, multi-trillion dollar1 industry is increasingly mobile and continually changing in response to consumer demand and innovation. As liberalisation continues to spread across the globe and competition hots up, rapid technological change has meant telecoms services are ever more firmly embedded in our lives. These services now underpin both our business and personal interactions, with changes in the telecoms business both mirroring and driving changes in the way we live and in the way we conduct business. What is becoming increasingly apparent, however, is that we are not simply witnessing an industry transformation - which suggests a change from one state to another - but rather a process of evolution comprising a series of small and large changes of various kinds that stretch from the 1980s, through the exciting state of flux that is the present and on into an unknown future. Telecoms is a rapidly changing, growing and vibrant industry. Amidst all the uncertainty there is only one certainty: change is here to stay.1. For example, the Telecommunications Industry Association (TIA) estimates that the global telecoms industry was worth around USD3 trillion in 2006 and growing strongly by an estimated 11.2%. Europe is the largest market in the world, comprising approximately one-third (USD1 trillion) of the total global market, followed by the US which alone is worth USD923 billion. Business and technical staff need to work together - technical staff need to ensure that the project is driven by the needs of the business. Also make sure that the team follow well tried strategies and bring in migration experts to save you relearning painful truths.1Top tips for successful application transformationProjects need to be 'lights on' - that is, service providers need to continue 'business as usual' and cannot afford to stop doing business or put things on hold during the transformation project. It is important to ensure that the migration delivery software is able to support this requirement.2Untitled DocumentPage 3The Application Transformation ImperativeChange in the telecoms market is being driven by a range of factors, as shown in Figure 1. These factors do not necessarily work in isolation from one another, but are often interdependent. For example, regulators have acted to reduce prices for consumers and to introduce competition; while increased competition has, in part, stimulated innovation. Similarly, globalisation has acted to increase competition and also to stimulate customer mobility.Figure 1: Change drivers in the telecoms marketAll of the factors outlined combine to create a market that is constantly evolving. The survivors and beneficiaries of this market will be those companies that can embrace change and respond positively to the challenges thrown up. This is what BT calls the requirement for 'continuous innovation'. In other words, surviving change involves embracing change.Business imperatives result in new technical strategiesThe change drivers that we have outlined result in six main business imperatives for telecoms service providers. These imperatives are the things that telecoms service providers need to do in order to stay in business and prosper, and include:need to cut costs and improve revenue managementthe need to innovate and get products to market more quicklythe need to retain customersthe need to increase ARPUsthe need to comply with regulation and legislationthe need to use business information more effectively (data transparency).Source: Celona Technologies, 2007Top tips for successful application transformationReduce risk by taking control - by using progressive migration methods (that is, moving the data in small controlled steps that you determine), you don't have to take the 'leap of faith' required by many traditional approaches to data migration. Starting with small, controlled steps, you gain knowledge and understanding of the migration data, and speed up over time. Overall this appraoch will result in a faster migration.3Untitled DocumentPage 4The Application Transformation ImperativeThe initial reaction of the industry to change drivers such as competition and investor pressure was to embark on a programme of cost cutting in response to price competition. However, once the first round of obvious costs were trimmed away, subsequent efficiency gains became more challenging to achieve, so service providers looked, for example, at how they could automate more effectively and streamline systems in order to reduce cost. "The top-performing one-third of ICCs will save an average of 30 percent in data interface development time and costs, 20 percent in maintenance costs as well as achieving 25 percent reuse of integration components&. The remaining two-thirds of ICCs will fall short of those benefits because of insufficient sponsorship and other organizational execution problems&." GartnerHowever, cost-cutting alone is not enough. In order to achieve business goals such as retaining customers, increasing ARPUs and so on, service providers need to innovate. It is also vital for them to be able to react more quickly to change and competition - which is often described as becoming 'more agile'. In the new service-based paradigm we are entering, this translates into the ability to launch new services more efficiently and at the right time - and preferably more quickly than competitors.Although these ideas seem relatively simple on a business level, achieving them at an operational level is far more complex. Legacy systems and architectures, for example, were not designed for this environment and many service providers are therefore struggling to deliver the essential fast time-to-market for new services, and fast provisioning times to customers. At the same time, legacy IT infrastructure is often very expensive for service providers to maintain. In many companies, the vast majority of the IT budget is spent simply maintaining legacy infrastructure rather than on innovating to improve competitiveness. In addition, established service providers do not have the luxury of greenfield projects. They have to continue doing business while working on system transformation projects. They also have large sums and effort invested in back-end solutions and are often not willing to simply throw this away and start again. A further problem is that back-end systems and architectures are often so complex, and so poorly understood, that making significant changes to them is highly risky. One of the thorniest pieces of this complex puzzle, and a significant barrier to change, is the myriad of problems surrounding the data silos that sit beneath existing solutions. Using business information more effectively is also a vital issue. This is about making data the company holds more accessible and therefore more usable. It is important for compliance issues, but also aids better business analytics to help make the business function better and compete better. However, a consistent and complete view of data is essential to achieve this goal.A lot of major systems overhauls are already underwayThe business imperatives that service providers are seeking to deliver against, have stimulated them to undertake strategic overhauls of their back-end systems and architectures to reduce costs and align them better to new commercial requirements. Three types of consolidation can be identified:New architecture - moving legacy to a new B/OSS stackRemoving replica applications - reducing the number of instances of the same applicationsTop tips for successful application transformationReduce dependencies on external groups - spend time selecting your project team, and use business staff to approve process and design. Traditionally this isn't easy to achieve because in large 'waterfall' projects the team needs input at each stage. However, if you follow a progressive migration approach you'll get external buy-in quickly on the first few cycles of migration.4Untitled DocumentPage 5The Application Transformation ImperativeRe-platforming - replacing existing platform infrastructure or technologies.Only the former of these requires both the business and IT department to work together to transform systems, data and process. Examples of rationalisation projects that are currently being undertaken include: BT has an ongoing project aimed at reducing supporting systems from several thousand to less than 100, and realising cost savings of 150 million by October 2007 Deutsche Telekom announced further cuts in admin and IT costs in September 2006, aimed at reducing OPEX by 5 billion over four years Telstra is also undergoing a massive consolidation of its back-end systems aimed at cutting out more than 1,000 existing systems.As can be seen from these examples, large established service providers are suffering from system proliferation resulting from years of tactical investment. This means that back-end architectures are often needlessly complicated and expensive to maintain, with key processes fragmented between best-of-breed systems, and multiple systems performing the same function for different services. "Service provider's network environments are often overly complex, creating a barrier to the efficient and cost-effective roll-out of new services." David Milham, BTHowever, when service providers contemplate consolidating systems and processes, they are faced with significant data migration challenges, which are complications arising from the requirements to move information from a legacy environment and transform it to fit in the new system environment. They can't even outsource the problem to make it go away, as this still requires them to deal with fundamental data quality issues for example.Data may be a competitive weapon, but it's also a twin-edged swordThe dual nature of telecoms data as both an enabler and a disabler, an opportunity and a problem, was highlighted by industry analyst Teresa Cottam in a recent Analysys report."Data, not networks, is now the basis of a service provider's entire business. Without accurate data, service providers cannot bill for services supplied, assure their revenues or services, design or target new services properly, or make sensible investment decisions. The advent of the VNO effectively decouples the network from the business of telecoms provision. Even in markets where this model has yet to be introduced, data is what turns a collection of network equipment into a business. And telecoms service providers are awash with valuable data that has great potential to transform their businesses&[but] while data is the key to unlocking next-generation revenues, operators possess not a single skeleton key, but a bunch of keys of varying sizes, types and ages. Merely buying another new system will not solve their problems." Teresa Cottam, World Telecoms BSS and OSS Markets, Analysys Research, 2006This is the dilemma that many service providers face. They cannot deliver against their business goals unless they solve both their systems and architecture challenges, and Top tips for successful application transformationReduce dependencies on other migrations - most large service providers have multiple system migration projects running in parallel. A data migration project for one system should not require a corresponding migration of data in another system, as interdependencies will significantly reduce the chance of success. 5Top tips for successful application transformationReduce human error and learning curves - people make mistakes, and many tend to repeat the same mistakes. People also take time to get up to speed. Therefore it is wise to use fewer staff in specialised functions and enable them to repeat their activities often in a migration which will help them become more proficient.6Untitled DocumentPage 6The Application Transformation Imperativeone of the biggest barriers to effective systems renewal are the problems surrounding data conversion, management and migration.These problems are not unique to telecoms. It is estimated that around two-thirds of Fortune 1000/Global 2000 companies are engaged in some form of data conversion project at any given time. These projects include creating new data warehouses and datamarts, migrating data from legacy systems to packaged applications, data consolidation exercises and data quality improvement projects. So in theory, at least, you would think that with all this practice companies would be highly proficient at data conversion. Well unfortunately that's all it is - theory and wishful thinking. In practice, the stats on failed projects make dire reading. For example, according to the Standish Group, in 2006 more than half of all technology projects in the US were considered failures, costing businesses between 80-100 billion annually. That figure goes even higher if you add in 'shelfware' - that is, projects that are completed but nobody actually needed them or uses them.Garbage in, garbage outAnalysing failed IT projects reveals that one of the main causes of failure are problems due to source data. Traditional approaches to data migration create nearly as many problems as they resolve. Common problems encountered include data not loading properly, poor data quality and compounded inaccuracies - resulting in ineffective projects, time and cost overruns, and project cancellations. It is all too common to woefully underestimate at the start of a project the complexity and risk presented by data migration projects in large companies. Business managers often have a very simplified view of IT systems and how they interact, and make misleading assumptions such as that two systems maintaining similar data must be performing similar tasks, therefore mapping data from one to another should be straightforward. This is hardly ever the case and, unfortunately, it often only becomes obvious that it is not the case when the project is already failing. The Standish Group say in the 2004 Chaos Report that 71% of IT projects fail or overrun. Eighty-four per cent overrun on time and fifty-six per cent overrun on cost. Another common fatal error is to make assumptions or overestimates about the true quality of underlying data. Data may be spread across multiple datastores, with problems stemming from the fact that legacy systems might have their own data terminology, data types, metadata, codes and master data. There may be many different 'flavours' of the same data and many data elements of the same 'flavour'. The data might not be in the right format, data quality might be poor or data might even be missing or incomplete. 'Data model accuracy' is also a problem. In other words, there might not be a clear understanding of how the business is using individual data fields. In such cases, the business might be using an attribute for something other than that documented (for example, a 'location' field might be actually being used for customer segmentation). This causes problems during migration because it means the data will get mapped to the wrong place. This is particularly problematic in the telecoms market because, all too often, how the fields are actually being used is undocumented and with staff turnover over a period of time this can become a substantial problem that is only discovered during the migration itself.Top tips for successful application transformationTry to create a 'migration factory' - migrate in many steps, use software transformation products to help reduce the project development effort and to provide information on the project's progress, separate the technical activities from the business activities, and identify data groups and specialised test and reconciliation users.7Top tips for successful application transformation8Auditability - ensure that you can recover to a known state and that each step can be reconciled. Each step should be recoverable; if a mistake is identified then the business should be able to return to a consistent state simply. Changes in business data must able to be fully reconciled across all the partaking systems.Untitled DocumentPage 7The Application Transformation ImperativeIn addition, data profiling tools while good at capturing the technical and statistical analysis of data, are not good at capturing the semantic meaning. This means that while data may be technically and statistically correct, it is semantically wrong. For example a negative tariff might have been used by the business to flag a special case of service. To effectively translate data semantics requires a merger of both business and technical definitions. Either of which might be undocumented.Data migration tools are becoming more sophisticatedData migration was originally delivered using manual methods, involving expensive and skilled people. While such an approach may still be valid for very simple environments and datasets, with little impact upon the business, it is not an approach that scales in terms of volume or complexity. It is also, by its nature, prone to human error.This led to a second generation approach (see Figure 2), which involved building automated tools to reduce manual effort and error. However, bespoke tools often lacked flexibility and were still very expensive and time consuming to build. This led to the development of a market for third-party data migration and integration tools. Figure 2: Data migration is evolvingThis current generation (2.5G) of data migration tools use methods such as ETL (extract-transform-load), often with a data cleanse between the 'transform' step and the 'load' step. Third generation data migration products use data federation (or EII) to provide real-time integrated views across multiple datastores. The market has, until recently, been highly fragmented with different types of vendors providing different styles of data migration tool or different parts of the data integration puzzle. The market is now consolidating to create more fully functional packages. The problem is that while traditional 2G and 2.5G tools may be very effective at performing bulk data warehouse loads, or database transfers (that is, the sort of migration seen when upgrading from one version of a system to another), they are not as effective at tackling the complex application-level migrations that are characteristic of systems transformation projects currently underway in major tier 1 service providers. Top tips for successful application transformation9Ensure adaptiveness - because of the inevitability of change, it is important not to fix the physical, transformation, business logic and migration process. Ensure that these can flex during the migration.Source: Celona Technologies, 2007Untitled DocumentPage 8The Application Transformation ImperativeUsing traditional tools still creates fundamental problems such as:a high level of riskapplication integrity constraints business processes split across systems (in both source and target environments) lack of flexibility and ability to cope with changedependency between systems and processes. The third generation of data migration tools addresses these issues. Such tools support faster data migration at lower risk, natively support service-oriented principles and data federation. Built using open principles, they help service providers buffer themselves against inevitable change and are re-usable (thereby protecting their investment). One way in which re-use is facilitated, for example, is by separating the physical data transformation (schematic transformation) from the business data semantics, from the transformation process, contrasting current generation approaches which combine all three layers into a single script/graph layer.Although 2G and 2.5G tools can accommodate the move to service-oriented architectures, they are not built to natively support it, which means 'workarounds' have to be designed to accommodate this. These 'workarounds' (such as XML 'wrappers') introduce performance problems and mean, for example, that updates take weeks to implement rather than hours.Top tips for successful application transformation10Don't panic: trial and trial again - make sure that each move is well understood. Trial the generic migration moves and if necessary trial specific challenging data moves. Are data issues costing you dearly and standing in the way of business innovation?According to Larry English, author of Improving Data Warehouse and Business Information Quality, poor information quality costs enterprises 10-20 per cent of total revenue. So ask yourself how good is your data?Is it complete? Or is data missing or unusable?Is it accurate?Is it consistent? Or are there data conflicts?Is it stored in a standard format? Or do you have problems capturing or using data stored in unusable non-standard formats?Do you suffer from data duplication?Does your data have integrity? Or are you suffering from missing relationship links?Have you fully established the relationships between the data and the capabilities, customer and services it describes?Untitled DocumentPage 9The Application Transformation ImperativeA migration success story: BT FeaturenetDespite the horrendous statistics on failed data migration projects, there are some that go well and a few that go even better than planned. An example of such a project is provided by BT's highly successful Featurenet migration.Featurenet is BT Retail's fully managed service supplying centrex-based services, VPNs and value-added services to around 500 large corporates and enterprises. More than one million directly connected users are supported, who together generate between three and five million charged events daily. The service is highly tailored to the needs of individual customers, meaning that each customer effectively has its own dial plan which has to be mapped on to the public network plan and tariffs. There are a large number of complex, multi-layered tariffs, combined with multi-tiered and multi-layered discount schemes. Featurenet services were supported by a bespoke legacy solution for billing, tariff and dial plan management. BT Retail wanted to migrate these customers to Convergys's Geneva system, but wanted a "completely seamless transition" to the new system.BT Retail achieved a successful migration in just six months and a full 13 months ahead of schedule, using the Evolve tool from Celona Technologies. Evolve's application transformation capabilities allowed a much higher level of control, resulting in a lower risk, stepped migration that was driven by the business. BT Retail has since credited the successful project with creating more than 148 million in new revenues, thanks to its ability to launch innovative new services to Featurenet customers. "It's hard to imagine how we could have achieved this highly strategic evolution without Celona's next-generation products and highly skilled team," reflects BT's Billing Development Manager Carl Wilson. Celona Evolve doesn't use ETL, but a unique synchronise-reconcile-switch methodology that delivers greater control at lower risk.Questions they'd rather you didn't askWill your data migration project deliver on time and at budgeted cost?Are you comfortable with the risk profile you are assuming?What happens if it doesn't work?How much is it costing your organisation for every day of delay?How flexible is your migration architecture, will it be able to respond to inevitable change?