Getting to grips with Google’s open-source container project

google margimurphy
Google's cloud platform developers created Kubernetes ©MargiMurphy

eBay is using Kubernetes in a bid to create an entirely programmable datacentre

Share

Google decided to put its container project Kubernetes [ku-ber-net-ease] out in the wild because "there is power in open source", its co-founder tells ComputerworldUK.

The project, which aims to simplify containers for organisations looking for faster app launch and scale-out, began as a “grand experiment”, Kubernetes' co-founder Craig McLuckie reveals. During the build, he discovered that if Google built a cloud platform in the open, “it would be better across any measurable dimension.”

Google has a habit of creating technology that is quickly emulated by an open source community. Take MapReduce, its proprietary offline database technology and inspiration for Hadoop, as well as Big Table, which inspired the NoSQL HBase project.

With this in mind, the search giant opened Kubernetes' code, hoping that developers and businesses that want to learn from Google's expertise - it launches 2 billion containers per week to support apps like Google Search and Maps - will purchase Google Cloud on the side.

The Kubernetes logo 

“We have to accept that many of our customers want to live in a multi-cloud world. If they don’t have access to new and innovative technology in every place they want it would ultimately slow us down and gate our options for success,” he says.

While furthering community and nurturing talent is a universal benefit, McLuckie and his team know the open-source sauce is a powerful marketing tool - regardless of a project’s quality.

“There is power in open source. We have seen this over, and over again. I’ve watched little companies become fantastically popular and not necessarily by virtue of what they bring, but the power of the community,” he says. “As we brought a product to market, we realised that it is a far better way to build software.”

Kubernetes and Docker

The introduction of container manager Docker sent ripples through the tech industry last year. Docker makes containers - something historically used by big cloud providers - easy to use. This gives businesses, including eBay, Baidu, the BBC, Groupon and Spotify, near real-time app startup, faster scalability, and more efficient servers than virtualisation technology.  

Docker logo

Spotify, for example, began using Docker when it could no longer manage its infrastructure one server at a time, due to its rising popularity. It dabbled with management tool Puppet to configure its 5,000 servers, but found the process time consuming. Avoiding costly virtualisation, Spotify trialled Docker before it had even left beta (transporting metadata like album or track titles) and is continuing to deploy containers.  

Following its first release just over a year ago, analysts and developers predicted containers would be as revolutionary as virtual machines (VM) and Google, IBM, HP, Red Hat, VMware, CenturyLink, Amazon Web Services, and Microsoft swiftly signed up to offer Docker through their cloud services.

Kubernetes is an orchestration tool for Docker. It allows infrastructure managers to administer a cluster of Linux containers as a single system. In simple terms, this allows apps to work on user’s devices, everywhere.

Like most open source projects, Kubernetes needed a name that wouldn't roll off the tongue. After struggling with it several times, McLuckie helps me out. “I don’t really know how to say it, we just made it up.” I later learn that Kubernetes is Greek for pilot, which alludes to the technology’s transportive nature. It’s sweet that McLuckie pretends not to be able to pronounce it. 

eBay developers write and deploy 10,000 apps a day

eBay is an early adopter. It has deployed Kubernetes to help support the 800 million products, 25 million active sellers and 157 million active buyers that interact on its ecommerce site daily.

The online marketplace’s developers write over 10,000 apps that are deployed hundreds of times a day in a completely automated, monitored environment. It uses Hadoop to process buyers' instincts and is one of the largest OpenStack deployments in the world.

Now, it hopes to make everything in its datacentre programmable, and will use Kubernetes to get to a level of abstraction where employees can use self service through truly open APIs. It hopes to break ground by running thousands of nodes for heterogeneous workloads (this includes frontend as well as Elastisearch) on a single set of architecture.

Vendor-led open source

Without financial or business incentives, open source projects struggle to take off. But it’s a fine line to tread. McLuckie praises Red Hat for making open source cloud computing platform OpenStack, “something way better than it ever was,” and wants to similarly incentivise developers to use Google’s own hosted version of Kubernetes once it is widely adopted.

He says: “We are not concerned with the idea of competing on the quality of our infrastructure. We just want to create an incentive for businesses to try us. A lot of early Google Container Engine customers were using Kubernetes on Amazon and then they decided to try our hosted version.

"They liked it and stayed with us. It’s working well as a driver for adoption...There is just a pretty strong commercial incentive in acknowledging that threat of vendor lock-in is a buying decision for customers.”

Google announced that its Container Engine left beta last week. It can run Docker containers, private registry as well as Kubernetes as an orchestration tool. 

Find your next job with computerworld UK jobs