British Airways’ BI lead: ‘If you don’t adopt Hadoop at least in part, you won’t exist in a few years’ time’

British Airways have produced a blue-print for Hadoop that could be replicated throughout the business for data-as-a-service and innovative streaming and social media mining.

Share

Enterprise business intelligence departments will ‘not exist in a few years’ time’ if they do not adopt Hadoop in at least one instance, British Airways’ data exploitation manager warned during the Hadoop Summit in Brussels today.

Alan Spanos, who leads the airline’s data analytics team, advised the Hadoop community and prospective Hadoop users on how to build a business case to use the technology.

“This stuff may seem cool but if you can't get your finance team to buy into it you will never deploy this software," he said. 

“In business intelligence, if you don’t adopt this technology to do at least part of your job role, you will not exist in a few years time. You can only go so far with traditional technology. It still has a place within your architecture, but quite frankly, this is where you need to be."

British Airways has deployed its first instance of Hadoop as a data archive for legal cases that were primarily stored, to a high cost, on its enterprise data warehouse (EDW) platform.

Since deploying Hortonworks 2.2 HDP, Spanos said his department has returned on its investment within a year, and is able to deliver 75 percent more free space for new projects, which translates to cost reductions to the airline’s finance team.

The space is available on the EDW platform as well as new commodity servers that were bought in with extra space. Innovative uses of Hadoop include using the technology for moving ETL or streaming data, however British Airways, which is primarily a ‘virtualise everything’ firm is initially rolling Hadoop out to introduce its capabilities to the wider business.Upon this platform, other business uses can be prototyped, validated and built so it has ensured that the architecture is scalable.

Jay Aubby, data warehousing architect at British Airways, added: “We are archiving but we will do exploratory work and the overall BA vision is to do more stuff on hadoop and leverage data-as-a-service going forward extensively."

The airline use Puppet (an open source configuration management tool) and YARN (the ancestor to MapReduce). It hopes to build a core layer of applications including Kerberos for security authentication, Ambari, Falcon and Zookeeper for deployment and management as well as Oozie and Control-M for scheduling as part of its Hadoop architecture.

Aubby added that the team had opted for Hortonworks as a distributor as it appeared to be the “least confrontational”.

Hortonworks announced that IBM and Pivotal will all use a standardised version of Hadoop 2.6 in an attempt to further the work of the Open Data Platform consortium, which hopes to solve interoperability issues with businesses using different versions of the big data technology.

The announcement is a considerable landmark since the group formed in February this year, but has rustled feathers following the inclusion of the open source Ambari deployment and management tool within the certified kernel. 

During the Hadoop Summit, vendors IBM and Pivotal were quick to add that the Open Data Platform was not an attempt to muscle together and crush competitors like Cloudera, who offers a similar management tool, as well as MapR, but was in the interests of increasing Hadoop adoption in the wider business world.

"Recommended For You"

HortonWorks previews open source Hadoop platform Marks and Spencer uses Hadoop to improve understanding of multichannel customers