Preface
Apache Mesos began in 2009 as a research project at the University of California at Berkeley, led by Benjamin Hindman. Ben and his team wanted to improve datacenter efficiency by allowing multiple applications to share a single computing cluster, just like multiple applications can share the processor, memory, and hard drive in your laptop or workstation. But they wanted to do this across the many servers that make up a modern datacenter. After an initial implementation of 10,000 lines of C++ code, they published the paper Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center in 2010.
Not long after, Ben joined Twitter and used Mesos to better scale its infrastructure, largely bringing an end to the era of the “fail whale” that became infamous as Twitter was rising in popularity and its servers couldn’t handle the demand from users. Although Twitter doesn’t publicly disclose the number of servers in its expansive infrastructure, online sources and firsthand knowledge from presentations put this somewhere in the ballpark of 10,000 Mesos nodes per cluster.
In December 2010, the Mesos project entered the Apache Incubator, an arm of the Apache Software Foundation that provides a means for projects to gain the full support of the ASF’s efforts. The Apache Mesos project graduated from the incubator in June 2013 and is now a top-level project.