How ServiceNow deploys MariaDB to handle 25 billion queries per hour

servicenowtim1600
Tim Yim on stage at M18 in New York

Computerworld UK sat down with ServiceNow’s director of operations Tim Yim to discuss how the open source database continues to underpin its rapidly scaling platform

Share

ServiceNow has turned to the open source database MariaDB to process 25 billion queries per hour, and is working closely with the vendor to support its growth plans.

The IT service desk software-as-a-service (SaaS) vendor is specifically running MariaDB TX, which is the enterprise version of its open source database, packaging up a range of tools and services for managing, backing up and monitoring core MariaDB technology, and is paid for on a subscription basis.

ServiceNow runs MariaDB at a pretty unprecedented scale: nearly 85,000 databases globally to be exact. That's 84,876 instances of MariaDB, running 176,686,129 InnoDB tables and processing those 25 billion queries per hour.

Previously it was running on an open source version of MySQL, but started to push it to its limits three years ago. Fortunately moving to MariaDB was a "drop-in replacement," said Tim Yim, director of operations at ServiceNow, using the open source vendor's favoured phrase when it comes to moving customers off of its Oracle-owned rival.

To further complicate matters, ServiceNow has each customer on its own database or multiple databases. It also allows customers to personalise queries and run custom workflows on its platform too, so "the query pattern changes per hour," Yim explained, making it tricky to tune.

The key performance indicator for ServiceNow, like many technology companies, is availability. This is one reason it runs so many instances across eight pairs of data centres globally, so that it can limit failover to a single customer. Since moving to MariaDB the platform runs at 99.996 percent availability, according to Yim's slides.

Read next: Crossrail opts for ServiceNow for its short-term IT helpdesk needs

Speaking to Computerworld UK during MariaDB's M18 user conference in New York this week, Yim repeatedly said that where MariaDB stood out was "less about the technology" and "more about the people".

He explained: "It all started three years ago and we had the guys come out and do onsite training with DBA [database administrator] staff and systems administrators and SRE [site reliability engineers]. From there we really saw that the folks at MariaDB are on the right track.

"We really progressed into stronger relationships and started sharing our problem statements and our roadmap of where we needed to go, and once we shared those statements the MariaDB folks were very open and enthusiastic to fix those."

For example, "one of the sponsored developments we have had is the instant add column in version 10.3 and another is instant alter which will be in 10.4. That is to help our customers upgrade their platforms even faster," Yim said.

In other words: with great scale comes unprecedented access. ServiceNow has a MariaDB enterprise architect visit its San Diego headquarters once a quarter to discuss the technical challenges it wants to overcome with them.

The two companies are on the record as collaborating on new features such as real-time data definition language (DDL) and plans to enhance encryption for cloud computing.

Read next: MariaDB launches innovation labs

Lastly, Yim said he is very interested in the sort of machine learning capabilities that MariaDB CEO Michael Howard spoke about earlier in the conference.

Yim said: "I think any DBA would be interested in a database that is more intelligent about responding to his workload and it sounds like the MariaDB folks are on the right track there in terms of performance tuning, scale up/scale down and visibility into what the database is doing. Having machine learning backing up all of that logic seems like the right direction."

Read next: Oracle unveils 'self-driving' autonomous database cloud