HSBC turns to Google Cloud for analytics and machine learning capabilities

HSBC istock whitemay

CIO Darryl West says that customer demand for digital services is resulting in huge data growth

Share

HSBC is using Google's public cloud to apply data analytics and machine learning tools to its vast data sets, targeting a range of applications from anti-money laundering to Monte Carlo simulations.

The pilot project was announced by HSBC CIO Darryl West at the Google Cloud Next conference today. It is part of a wider move to the cloud that has also seen the multinational bank partner with Amazon Web Services and Microsoft Azure.

The deal is a significant win for Google, which has faced criticism in its bid to take on the established cloud providers due to a lack of enterprise reference customers – outside of large tech firms such as SnapChat, Evernote, Niantic and Spotify. However, 150-year-old HSBC, one of the world's largest lenders with 27 million customers and $2.4 trillion in assets, certainly falls under this category.

Although the project is in its early stages – the bank has been working with Google for six months and is not yet production ready – West said that it will help the bank become a "simpler, better and faster organisation" and respond more quickly to customer demands.

West explained that the move to the cloud is being driven by a need to make sense of growing volumes of data. In 2014 this amounted to 56 petabytes and has since doubled to more than 100 petabytes.

"What is happening is our customers are adopting digital channels more aggressively," he told attendees. "We are collecting much more data about how our customers are interacting with us."

"Embedded in this data are massive insights," he said. "What we need to do as a bank is work with partners to enable us to understand what is happening with this data and draw out the insights so we can run a better business."

From legacy infrastructure to the public cloud

West said that the bank's route to the cloud is similar to many large enterprises. Like most lenders, HSBC has relied on legacy infrastructure such as mainframes, which, while reliable, do not have the database structures needed for machine learning and advanced analytics.

Three years ago the bank began to adopt open source analytics platform Hadoop to gain better insights into its data. West described this as a "tough road" to take.

"We have had to provision large infrastructure, physical infrastructure, build our data centres and hire talented new people who have an understanding of these new technologies," he said.

He added that the business came to the realisation that crunching data on the scale required would be difficult if it was to rely purely on its own data centre infrastructure.

"We thought about this very carefully in the last year and said to ourselves that at the core of our business we are a bank, but we also have a significant technology company embedded within the organisation," he said. "The question that I was asking the management teams [was] do we really want to compete with the cloud providers and people like Google, are we really going to try and do what they do as well as they do it?

"Our conclusion was that it was better for our business if we adopt a cloud-first strategy."

Cloud use cases

The bank began targeting use cases involving very large data sets that require very intense computing capabilities in short bursts, West said.

One is anti-money laundering. West said: "We are running analytics over this huge dataset with great compute capability to identify patterns in the data to bring out what looks like nefarious activity within our customer base.

"The patterns that we identify are then escalated to the agencies and we work with them to track down the bad guys. This is an application that clearly requires massive datasets, great computing capability, but also a machine learning capability to them identify the pattern in this huge data asset."

Another is risk simulations, providing insight into "billions of transactions".

West said: "We need to be able to run complex Monte Carlo simulations on a regular basis to better understand our trading positions and our risks."

Multicloud approach

HSBC is already using AWS for development and test purposes, and told press that the bank is also working with Microsoft, using its Azure public cloud. It is also using Oracle for software as a service applications

West said that the bank has been working with "all the big cloud providers" and will continue a "hybrid" strategy, identifying the strengths and weaknesses of each provider.

He said that one of the advantages of Google has been in appealing to its teams of data scientists.

"We spent a good amount of time working with all the different cloud providers and evaluating their products," he said. "If you look at the scarce talent we have around data analysts and data scientists, they need tools that allow them to do their work in a very productive way."

He added: "There is always a challenge to get data scientists to shift from their favourite platforms – they have really enjoyed that experience."

The bank is always working with Google on developing security solutions, he said. The next stage is to begin to move towards using the cloud in production.

"It is early in our relationship, we are still in pilot mode and we hope to get these use cases to production in the coming weeks and months but our experience so far has been very positive," he said.

Find your next job with computerworld UK jobs