How Just Eat avoids costly weekend outages

The food delivery service uses AppDynamics monitoring to avoid outages at weekends and upsetting hungry customers


Popular food delivery service Just Eat has recently turned to AppDynamics to get a high-level view of its web and mobile apps as it looks to cut down on costly outages at peak times.

Just Eat has more than 10 million customers in the UK, and with the growing popularity of food delivery apps, as well as its recent acquisition of rival Hungry House, that figure is only set to rise.

© Just Eat
© Just Eat

Just Eat is, amazingly, more than 12 years old. In a busy market where upstarts like UberEats and Deliveroo are aggressively expanding, outages at peak times can cause a huge amount of damage. These apps pride themselves on the convenience factor, meaning customers often respond to issues with an outsized level of outrage.

Read next: Why Deliveroo continues to suffer from server outages

Take Just Eat's reported outages on New Year's Day, where a tweet from the company reporting technical issues across its website and app received more than 200 responses, some verging on the dramatic side. "You're ruining my life!" one user tweeted.

The ideal situation for Just Eat then is to get ahead of issues before they hit customers, and a solid monitoring function is a key part of that.

Adopting AppDynamics at Just Eat

This is where AppDynamics comes in. Just Eat adopted the application performance management software - which was acquired by Cisco in January 2017 for $3.7 billion - in the spring of last year and got it in front of the relevant users within three months.

Just Eat's infrastructure consists of approximately 400 microservices running across thousands of AWS instances in the cloud, with an open source monitoring and logging service, which has been tweaked in-house.

This rolls up to an alerting system, also open source, which allows engineers to reduce downtime by spotting issues before they affect customers.

AppDynamics then "fits on top of that stack," Just Eat's director of technology Richard Haigh told Computerworld UK.

"We wanted a high-level view of all of those systems and understand which areas things go wrong," he said. The aim was to have a "joined-up view across the whole estate to improve triage".

Read next: EasyJet looks to avoid disruption with AppDynamics adoption

Where AppDynamics really stood out from rival solutions for the Just Eat team was its ability to pinpoint 'architectural dependencies' or in other words, pinch points.

This is important within a tech function that spans , according to Haigh, a complex estate with "hundreds of monthly releases" and "engineering teams all working at their own cadences".

"So visibility is important and that dependency mapping allows us to spot things that might have made sense a year ago," he explained. "We now have data for that."

Previously staff relied on a range of monitoring systems and dashboards, without a 'single pane of glass' view. Now they can see the "most important systems in as good as real time, where they communicate and how they perform in one screen," Haigh said.

So through AppDynamics the operations team gets a performance baseline, whether that's for a Tuesday night in August or a Sunday night in December, so it can spot any latency or potential performance issues ahead of time and drill down directly from the tool.

Read next: Best application performance management software

For example, Just Eat doesn't use autoscaling for its AWS infrastructure, instead opting for scheduled scaling "as demand can be cyclical," Haigh explained.

This means that any unprecedented spike in demand or a software release that's using up a bit more capacity than expected can be spotted early, allowing engineers to scale the infrastructure before users get cut off.

AppDynamics is now part of the toolset of the service operations team, who use it as one of their key dashboards. Developers get a view of their systems to assure what they have deployed is working as it should be as well as being able to test in pre-production - and lastly the tech managers use it to look at high level or business-focused dashboards.

What next?

Haigh did say he would like to see more automation within AppDynamics in the future.

The dream scenario, Haigh explained, would be: "To come in and see we have had an issue that could have affected customers and an automated system has mitigated it and supplied a clear list of actions we can take the next morning.

"So if I don't have to wake up a member of staff in the middle of the night, and no customers have been affected and we are left with a clear list of actions."

Read next: Application incidents down 30 percent at Barclays following AppDynamics adoption

This is where John Rakowski, director of technology strategy at AppDynamics jumped on the line to talk about the company's recent efforts to make the system more automated and intelligent.

The Cisco acquisition of Perspica in November will play a role here too. The San Jose-based company specialises in machine learning and data processing technology and has joined the AppDynamics team.

"Our path is looking at systems of intelligence and automated response to get enterprise customers to that automated response point quicker," Rakowski said.

"Recommended For You"

How Deliveroo uses data to deliver food whatever the weather AppDynamics now provides intel for business managers