White goods retailer AO.com generates more revenue online than any sector competitor, and the fourteenth most in the country overall, ahead of household names such as Debenhams and Waitrose. The vast majority of sales are made on the company’s customer-facing apps and websites. Three years ago AO.com adopted application and web monitoring analytics from New Relic to ensure they always function to their maximum capacity.
The software has helped slash the number of incidents affecting AO.com customers, and the time it takes to detect and resolve any problems that emerge. The ecommerce website is currently operating comfortably at 99.5 percent availability.
"We've dramatically reduced the frequency of instances that a customer is affected [by poor performance]," says AO.com software director Carl Phillips.
"We've also dramatically reduced what they call the MTTR - the mean time to resolve - so we're actually optimising how fast we can fix incidents."
AO.com founder John Roberts started selling appliances online in 2000 on the back of a £1 pub wager with a friend who bet he couldn’t. Today the electrical goods retailer stocks more than 4,000 products and sells the majority of them on its ecommerce platform.
Central to its success is a smooth customer experience, which won the company the Customer Service Initiative award at the 2014 Oracle Retail Week Awards. Maintaining this reputation while building new features into the platform every day was becoming a challenge as its home-grown application logging wasn’t scaling up-to-speed with the company's rapid growth.
"It had quite a high cost to maintain, and as our application state grew and our business grew it was quite difficult to keep up," says Phillips.
"We initially trialled New Relic just for the ecommerce web application and because it's such an easy install and you get so much out of the box for free, instantly it changed the game for us in terms of what we can see about what experience our customers were getting, what was going on our web farm, and how the application was performing under different load levels."
The successful trial convinced Phillips to extend the New Relic Application Performance Monitoring (APM) remit across the back office estate from all the customer service systems through to logistics. The previously time-consuming analysis of data on performance was replaced by near real-time insights into customer experience and the automatic identification of problems.
New Relic functions
Phillips and his IT team of around 140 people log into the website on a desktop, mobile or iOS app to find a traffic light system on the first page of the APM interface. They can filter through the different applications monitored individually on dashboards or in custom groupings such as the live estate.
The central measure of performance is known as the Apdex (Application Performance Index). This analyses the response time of web applications and services against a set threshold, and measures the proportion of satisfactory response times. If the response time from the server is benchmarked at 85 percent within 200 milliseconds, an alert will be sent if less than 85 percent of requests pass that threshold. If there is a major spike in error rate, the Apdex will drop even if the application performance is fine.
"If your Apdex falls beneath a certain tolerance, firstly the application will move from a green into an amber status, which says something's happening which might not have breached the alert threshold yet, but something abnormal is going on in your application,” says Phillips.
"If the error rate spikes over a certain percentage or the Apdex drops beneath a certain threshold on the dashboard, that application will turn red."
Alerts are sent to AO.com of any breaches of the threshold caused by issues such as performance degradation during peak times such as Black Friday and Boxing Day sales, which previously had the potential to completely take down the website.
"When Black Friday first became a thing in the UK, there was a potential that such a big event like that could take down the website," says Phillips. "But what we were able to do was load test the website out-of-hours and then use New Relic to look at what the correlation between load and performance was, and to optimise the site to best be able to handle that kind of load."
A complementary tool called Insights adds comprehensive business analytics to the data. Another called Infrastructure offers insights about public cloud infrastructure that AO.com intends to use to identify problems in any availability zones within Amazon Web Services.
AO.com also considered Dynatrace but it needed a major upfront commitment in cost and time. New Relic offered an out-of-the-box solution that was easy to learn and still provided abundant rich information. Customers pay per application license per host and can begin with one application as AO.com did with the ecommerce platform, before adding other services as required.
"We didn't need to send our people on specialist training," says Phillips. "Financially, we didn't need to bet the farm on it. It had overall a really low cost of adoption."