Getting the winning habit - how businesses can build for big data success

Omar Agha compares businesses migrating from traditional data management to big data implementations to football teams getting ready to join the premier league. The potential rewards are great, but it's a challenging transition where preparation is key.


For the football teams currently getting ready to join the Premier League, these are exciting but anxious times. The potential rewards are great but making the transition from the Championship will be challenging, and preparation will be key.  Businesses migrating from traditional data management to big data implementations will feel similar emotions to those teams - trepidation mixed with determination to make the most of the opportunity they have worked so hard to achieve. Like the teams, they’ll need to plan meticulously if they want to ensure it’s a long-term success.

Not everything will be new of course. The basics of traditional and big data management approaches are similar. Both are essentially about migrating data from point A to point B. However, when businesses move to embrace big data, they often encounter new challenges. 

New tools are needed and there are new skills to learn. Over time, businesses will increasingly need to deliver data in real-time on demand, often to achieve a range of business goals from enhanced customer engagement to gaining greater insight into customer sentiment or tapping into incremental revenue streams.

It won’t always be straightforward. The volume of enterprise data is increasing exponentially. Estimates indicate it doubles every 18 months. The variety is growing too, with new data sources, many unstructured, coming on stream continuously. Finally, with the advance of social media and the Internet of Things, data is being distributed faster than ever and businesses need to respond in line with that increasing speed. 

Taken together, these trends are driving the compelling need for organisations to migrate to big data implementations. Traditional approaches to data management increasingly struggle to manage in this new digital world without driving costs sky-high or taking too long to reach viable results.

The emergence of big data necessitates businesses moving to a completely new architecture based on new technologies from the MapReduce programming language to Apache Spark and Apache Storm Big Data real-time in-memory streaming capabilities to the latest high-powered analytics solutions.

There is much for businesses to do in terms of learning new technical languages and building new skills and in terms of governance, funding and technology integration. Once again, it’s a real step up.   Success will not happen overnight.Businesses need to appreciate this and set realistic expectations and goals - just as in football, managers whose teams are new to the top flight, need to take a pragmatic approach and not be too dispirited if they fail to match Chelsea at the first attempt.

Talend has identified five key stages to ensuring big data readiness, the exploratory phase, the initial concept, the project deployment, enterprise-wide adoption and then finally optimisation. The following outlines the key goals they will need to accomplish at each stage of their journey in order to ultimately achieve big data success.

In the initial exploratory phase, the focus should be on driving awareness of the opportunities across the business. Organisations therefore first need to become familiar with big data technology and the vendor landscape; second, find a suitable use case e.g. handling increasing data volumes and third, provide guidance to management on next steps.

The second phase is around the design and development of a proof of concept. The overarching aim should be IT cost reduction but the key landmark goals along the way will typically include building more experience in big data across the business, not least in order to better understand project complexity and risks; evaluating the impact of big data on the current information architecture and starting to track and quantify costs, schedules and functionality. 

The next stage moves the project on from theory to practical reality. The project deployment phase specifically targets improved performance. Key goals include achieving greater business insight; establishing and measuring ROI and KPI metrics; and developing data governance policies and practices for big data.

Enterprise-wide adoption drives broader business transformation. It is here that businesses should look to ensure that business units and IT can respond faster to market conditions; that processes are measured and controlled and ultimately become repeatable.  The final level of readiness is business optimisation. To achieve this, organisations should look to use the insight they have gained to pursue new opportunities and/or to pivot the existing business.

Clearly pragmatic execution and detailed planning is what will ensure you achieve big data success. Don’t do this and you may not get funding or support for a second project. It’s a bit like getting relegated at the end of your first season in the Premier League.

 by Omar Agha, VP of Sales, Talend

"Recommended For You"

Making sense of Big Data Swansea City turns to IT in bid to become 'a Premier League business'