The BBC has improved the performance of its popular iPlayer video on-demand service after moving from its legacy relational database system onto a NoSQL platform.
Speaking at the 'Forrester Forum for Technology Management Leaders' in London today, the BBC’s senior product manager for editorial metadata, Allan Donald, said that switching to new database technologies as part of the recent iPlayer upgrade has offered significant benefits over traditional SQL systems.
“The new iPlayer that we launched a couple of months ago is now completely built on the NoSQL MarkLogic stack – so if you are using iPlayer you are seeing a NoSQL-based system,” he said, adding that it allowed content to be uploaded much more quickly compared to its old database.
“Our legacy system took up to 30 minutes or an hour sometimes [to publish a video clip]. We compared the MarkLogic NoSQL technology to some SQL vendors, and what we got in 20 seconds on SQL took us 200 milliseconds in NoSQL – orders of magnitude faster. So we said let’s move iPlayer to this."
The broadcaster announced the relaunch of iPlayer in March, with the aim of enhancing personalisation features and automatically adapting the look and functionality depending on the device used to access it.
Donald said that one of the drivers for using NoSQL technologies was that, as more viewers began to watch the iPlayer on mobile devices, the service had struggled with the demands of supporting different platforms - from smartphones to tablets and even gaming consoles.
“One of the problems was that we would publish an episode and then the database had to spend a good amount of time working out working out if it is available on Xbox, can you watch it on a handset. It spent so long trying to figure out the answers to those questions that it ran out of time to answer iPlayer questions.”
The BBC had first started using NoSQL technology during the 2012 London Olympics. This eased the significant challenge of quickly delivering content relating to the many different event types across thousands of web pages, as well as making the data available across numerous mobile platforms. Overall, around 2PB of data was accessed by viewers each day during the event, he said.
Having become more familiar with using the NoSQL tools, the BBC is now planning to move more of its programme services onto a central database platform.
“This has fundamentally changed the way we think about storing our data and bringing our products to our data,” Donald said.
“When we started this process, we had one big central programmes information database, and that was very structured and rigid. Then when we wanted to launch another product we would launch another database that read from that programmes database. So the iPlayer has its own database that covers its programmes, and then the news clips has its own database, and so on. We really had a 'set' of data.”
He added: “What we learned with the NoSQL is that we can have a one data store and then shape the data to the needs of the product when it required it. So we are moving much more towards to a model of having one central database that applications come to, rather than taking the data out to there. That has been a big change for us.”