Organisations looking to make the most of their big data could learn a thing or two from the Met Office, Computerworld finds.
“We realised in the last five years we have become a little bit trendy, which is great,” IT director Charles Ewen tells ComputerworldUK.
But the organisation’s approach to big data is far from that of start-ups, which is unsurprising given its 150-year legacy.
“There are two types of big data,” Ewen says. “One is about discovering new information where you didn’t know it exists, taking public data sets and correlating it with another for new knowledge. A lot of start-ups and innovative new companies are using cutting edge technology [like Hadoop] for that.
“But then there is a whole world out there, with companies doing big data and not even realising it. That’s us - we routinely extract information where we know where to look.”
The Met Office’s big data is, strictly, a science programme “furthering the world’s understanding of climate and our atmosphere” or a process of gathering weather information so it can “be embedded in a whacking great computer model”.
The model that Ewen is referring to is something called the Unified Model (UM), a system on which the Met Office can predict rain or shine by calculating weather patterns.
Although “unified”, the model has a number of variations and configurations to do different jobs in varying areas and timeframes, as the organisation works with a host of different industries and weather organisations across the world.
For example, the organisation just signed a deal with London’s Heathrow Airport to deploy six dedicated meteorologists on-site, connected virtually to the Met Office’s Exeter-based IBM supercomputer. The project is just one small part of the services it is offering within aviation and further afield.
"Aviation is a great example of where big data will go in other domains"
The Met Office is helping airlines burn less fuel by enabling pilots to choose smoother flight paths. Instructed by the International Civil Aviation Organisation (ICAO), it will provide forecasts including wind speed and temperature charts for aircraft flying above 24,000 feet.
And it has many predictive-data projects in the pipeline, Doug Johnson, the Met Office’s deputy director of applied science and scientific consultancy, says.
"Traffic management across Europe is gearing up for a revolution as our skies reach flight capacity," he says.
“There will be a fundamental redesign in the way in which aircraft is controlled in Europe.”
This redesign will be enabled through flight path-based data.
Johnson explains: “Out of an entire world of data we have, we know a plane is only going to fly from A to B. It is only flying through a relatively small sub-section of that dataset. Why would you send all of the data available to that aeroplane when you know it is flying a certain trajectory? So instead you cut out that trajectory, in data terms.”
The Met Office’s supercomputer needs to be able to separate and direct this data, but further, it needs to work with a plane that is moving through time, as well as space.
“Plus, you are predicting as well, which adds another layer of complexity,” Ewen says. “And then what about diverts?”
All of this information will flow straight to the cockpit as opposed to a flight management system as the plane flies above.
“Aviation is a great example of where big data will go in other domains,” Ewen adds.
20 million unique visits a day
Aside from the big data world of scientific and numerical weather predictions, and core applications that support the business 24 hours a day, the Met Office has an innovative area within IT that focuses on the latest technologies, in which it is world-leading, according to Ewen.
“We have applications, rich internet applications, machine-to-machine interfaces, web services,” he says.
“The Met Office has the most visited website in government – we can see 20 million unique visits on a busy day and are regularly in the top 50 or even top five UK sites.”
Sell insight, not data
Organisations need to think “cleanly” about the information it is trying to communicate rather than the data itself, Ewen advises.
Using Heathrow as an example, Ewen and Douglas liken their remote meteorologists to data analysts. Rather than simply reporting the weather, they are learning about Heathrow as a business, and using that insight to deliver the most beneficial predictions at the right time to assist with flight scheduling, and essentially, passenger satisfaction.
“We are able to build this value chain, or proposition, or service that big data is a big part of, but ultimately we’re turning the data into actionable insights,” says Ewen.
“You could put this in a retail context and it would be about the management of supply chains or procurement chains…A mature market would be providing that insight as a service rather than throwing around huge volumes of data.”