Jim Anning, data lead at British Gas Connected Homes (BGCH) - a lean startup that sits within the 200-year-old energy company - speaks to ComputerworldUK about the techniques the team are using to connect its customers' homes.
The unit responsible for smart thermostat Hive is supported by a 16-strong data team, all with expert understanding of the company's computer network, which means BGCH won’t rely on products like QlikView or Tableau to analyse its big data, despite their growing popularity amongst enterprises.
Anning says: “Like a lot of people, we have come from using a rag-bag of tools. We are pretty much an open-source shop with an awful lot of [programming languages] R and Python - we are very interested in what will happen with R and Spark in the future.
“We are not using any of the self-serve products. I don’t think we have a need for that sort of self-service business intelligence slice and dice stuff. A lot of things we look into require a reasonable amount of domain knowledge.”
The Hive thermostat currently measures 300,000 customer’s energy every half hour - feeding the data back to the customer through a simple dashboard on their smartphone.
But the Connected Homes team wants to offer customers a truly real-time insight into their energy usage.
It plans to pull data off sensors in customers’ homes every 10 seconds so it can offer useful warnings, for example, in the case of unusual energy usage which may suggest there is a fault.
“This has the potential then to help people understand where their energy is going because with more granular data you can spot more things than you can with every half hour dataset," he says.
This “Live energy” report needs a robust database behind it that Anning’s engineers can work with to develop algorithms as data from the sensors scale.
It will use Apache Cassandra, a NoSQL distributed database, developed at Facebook in 2008, that can manage large amounts of data spread across many servers.
“Having that ability to scale is important, that’s one of the areas where, with [Apache] Cassandra, it becomes much easier to predict what compute power you need based on what your expectations are in terms of customer numbers.
“Other database technologies tend to have crunch points which you have to invest a lot in to get a lot of power all at once.”
Connected Homes will also begin to put sensors on customers’ boilers next year.
Creating lean-startups within organisations
Although Anning says there is no “cure” for firms that want to use an Internet of Things strategy, he believes creating innovative startup units, or separate units within an organisation, has proven successful for British Gas.
He says: “It’s ultimately about giving people freedom, and removing barriers from the parent company."
Despite a shortage of data scientists within the industry, Anning says he “doesn’t have massive problems recruiting people into my team because we are working on interesting stuff.
“It clearly works for British Gas to work that way.”
The unit sits - amongst other British Gas employees - within a separate office in Surrey. This way Connected Homes can work in an agile manner.
The data team and software developers sit next to each other so they can feedback on iterations.
Anning says: “We all sit together, we’re a pretty small outfit, it feels more startup-like than you would expect. There's a lot of to-ing and fro-ing.
“Every week progress is made and we are able to change our mind and steer round problems and difficulties really quickly.”
Challenges of an Internet of Things platform
The biggest challenge for Anning’s team turned out to be less about the technology, and more about a human touch to help customers install sensors correctly, he admits.
“Don’t underestimate the advantage of having 10,000 engineers to install this stuff - connecting it all up is quite a tough one to get right.”
Further, be prepared for holes in data if you are working with customer’s technology, Anning warns.
“Because things are coming from consumer devices you don’t always get all the data back. There are holes in the data because sometimes people switch things off and you have to be resilient to dealing with data that isn’t 100 percent there," he says.
Over at British Gas, CIO David Cooper has spoken about allowing IT staff to be creative to boost morale and retain talent, including handing out Raspberry Pi computers and investing in open-source architecture.
Find your next job with computerworld UK jobs