eBay reveals the secrets of grid computing

IT professionals in the UK will get the chance to hear the details of eBay's remarkable grid computing infrastructure at the Open Grid Forum in May. If you cannot make the event, read how the online auction site stitched together 12,000 servers


Do you use eBay? Most of us have at least bought something off it at some point, and many have tried selling – the scale of its operations is phenomenal.

With 222m registered users at the last count, 610m listings each quarter, and 6m new items added every day, it takes a lot of computing power to keep eBay operating as fast and reliably as users have come to expect.

So what’s keeping all this running?

There are over 12,000 servers working hard, using software that eBay has had to create for itself because no vendor is offering anything that can cope.

When the company was created, it tried using off-the-shelf software but thisquickly "broke", unable to cope with the volume, says eBay distinguished research scientist Paul Strong. It is now keen to encourage vendors to improve their offerings, to help other companies getting into grid – and take some development work off its own shoulders.

“Building management tools isn’t eBay’s core business. Ultimately we want things to be developed by vendors, so that we can get on with building an e-commerce platform,” he says.

Three levels of infrastructure

There are three main aspects to eBay’s grid infrastructure.

The auction platform, “a very large piece of Java code” runs on around 7,000 servers in the Western USA. “We have about 12,000 to 15,000 instances of the site, running on about half that number of servers, because we direct the traffic from different parts of the world to the same machines,” Strong says.

Those servers take care of all of the code for the transactions that people associate with eBay. “We consider that to be a grid, a highly distributed transactional system,” he says.

"Recommended For You"

IT 'should pay' the power bill - eBay datacentre boss Microsoft wins Dell and eBay with Azure