Splunk is integrating packaged and custom machine learning algorithms into its major products to give IT pros and business users added capabilities like automated anomaly and pattern recognition, smarter alerting and predictive actions.
The machine learning updates will be launched into most of Splunk’s popular applications straight away, including Splunk Enterprise, Splunk IT Service Intelligence (ITSI), Splunk Enterprise Security (ES) and Splunk User Behaviour Analytics (UBA).
How does it work?
Splunk is bringing these capabilities to customers in two ways, adding machine learning algorithms to its domain specific tools like ITSI, ES and UBA, and also launching a free machine learning toolkit.
Splunk CEO Doug Merritt said the toolkit “allows customers to program algorithms which work natively with Splunk data so they can leverage machine learning for statistical analysis, outlier detection, predictive analytics, forecasting. The classic machine learning use cases.”
The toolkit provides some base algorithms but Splunk is taking an open source approach, allowing customers to “donate” algorithms around specific use cases, in the same way they operate their Splunk Base for apps.
The 2015 acquisitions of Metafor, which specialises in IT operations analytics, and Caspida have helped Splunk bring domain and machine learning expertise into the company so that they can bring these capabilities into products like ITSI and UBA, respectively.
“In our applications themselves we are embedding machine learning. That is more wrappered, you can’t necessarily expose it, it’s the secret sauce of some of the capabilities we drive, machine learning is woven through our solutions,” Merritt said.
The move mirrors that of Salesforce with its recently announced Einstein, which brings machine learning capabilities directly into its products so that customers can use the powerful AI capabilities without the need for a PhD in data science.
What are the updates?
For Splunk Cloud and Enterprise users the 6.5 update is looking to bring advanced analytics to non-technical users through the machine learning toolkit to do things like predictive analytics, outlier detection, clustering and forecasting. The toolkit is available to download immediately.
For users of Splunk’s IT monitoring software ITSI Merritt said the 2.4 release is “all about what you’ve been telling us: event management is a dark art and you are drowning in too many alerts.”
Senior vice president of IT markets Rick Fitz said: “With ITSI we took two particular algorithms which focused on two particular problems IT had. Very specifically around KPIs.
“Traditionally IT used to set static thresholds and you got an alert when something went wrong. It turns out when you start to manage thousands of things that is onerous on the operator and it very quickly becomes too noisy. So people just turn them off. So we created a way of doing dynamic thresholding, so learning over a period of time and adapting the thresholds.”
Splunk has also added machine learning-powered anomaly detection and is wrapping more contextual data into events so that teams can prioritise what to resolve.
For organisations that rely on Splunk’s security monitoring products Enterprise Security (ES) and User Behaviour Analytics (UBA) the latest versions look to bring a more complete view of their security landscape.
Splunk ES 4.5 has added automated retrieval, sharing and response in for what tends to be a multi-vendor environment. The feature, called Adaptive Response, promises to centrally automate retrieval, sharing and response to threats “by allowing users to grab and wrap context around risks but ensure remediation action is taken”, Merritt said.
UBA 3.0 has machine learning models for improved threat detection which learns to prioritise outcomes, and Splunk has added content updates so that the machine learning algorithms can be kept up to date with the latest threats. These updates to ES and UBA will be generally available from October 31.
What the customers say
Nick Bleech, head of information security at major UK building materials merchant Travis Perkins gave the announcement a double thumbs up, saying: “We are seeing this Hollywood scenario for security folks where you get a succession of small things occurring that start to build up that you would have ignored. So pattern recognition and anomaly detection becomes important.”
Gatwick Airport's, head of business systems, Chris Howell said he will be looking at the machine learning capabilities for standard IoT use cases around preventative maintenance for baggage belts and utilities, rather than predictive analytics around customer flows at this point.
Chris Kammermann from UK music recognition service Shazam says he is looking to create new applications using the machine learning capabilities, as well as use it for basic anomaly detection. He said: “I played with some of the machine learning capabilities already and we have developed an app with Splunk to predict the next number one hit and it works modestly well, so I would like to develop even more.”