The Facebook Approach to Open Source

Share

Here's an interesting sign of the times. Facebook is opening up a big wodge of code:

Here at Facebook, we're constantly facing scaling challanges because of our enormous growth. One particular problem we encountered a couple of years ago was collection of data from our servers. We were collecting a few billion messages a day (which seemed like a lot at the time) for everything from access logs to performance statistics to actions that went to News Feed. We used a variety of different technologies for the different use cases, and all of them were bursting at the seams. We decided to build a unified system (called Scribe) to handle all of these cases, and do it in a way that would scale with Facebook's growth. The system we built turned out to be enormously useful, handling over 100 use cases and tens of billions of messages a day. It has also been battle tested by just about anything that can go wrong, so I encourage you to take a look at the newly opened Scribe source and see if it might be useful for you.

What's interesting about this is that it is no token release to win brownie points; Scribe is core code:

Scribe is implemented as a thrift service using the non-blocking C++ server. The installation at Facebook runs on thousands of machines and reliably delivers tens of billions of messages a day. If you use the site, you've used Scribe.

Thrift, too, is open source code from Facebook:

Thrift is a software framework for scalable cross-language services development. It combines a powerful software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, and Ruby.

Indeed, Scribe and Thrift are just two of many open source projects that Facebook has initiated. This is not so surprising, since free software permeates the Facebook engineering culture:

Facebook has been developed from the ground up using open source software, and we are proud to give back to the open source community through various open source projects.

But in one other respect, there's something big going on here. The fact that Facebook tosses out its code as open source without making a big ballyhoo about it, and without even bothering to justify the move shows just how mundane – in the best possible sense – free software has become for a modern software company. The benefits of opening up your main code are so obvious, they don't even need to be enumerated anymore: it's just the way the Facebook generation of coders does things.

"Recommended For You"

Another Billion-Dollar Open Source Company: Instagram The big promise of Big Data