Exploring the background technology of Facebook, the world's largest PHP Site

At FacebookF8 this year's FacebookF8 developer conference, 51CTO provided you with the latest Open Graph strategy and semantic search technology. Today, let's take a look at the software behind Facebook and see how Facebook, as one of the most visited websites in the world today, ensures stable and reliable operation of 0.5 billion million users. Facebo: Exploring the background technology of Facebook, the world's largest PHP Site

At this year's Facebook F8 developer conference, 51CTO will show you about its latest Open Graph strategy and semantic search. Today, let's take a look at the software behind Facebook and see how Facebook, as one of the most visited websites in the world today, ensures stable and reliable operation of 0.5 billion million users.

Scalability challenges of Facebook

Before we discuss the details, we have some software scale that Facebook has already developed:

◆ Facebook has 570000000000 million page views per month (according to Google Ad Planner)

◆ Facebook has more photos than all other image websites (including websites such as Flickr)

◆ More than 3 billion photos are uploaded each month.

◆ Facebook's system service processes 1.2 million photos per second. This does not include photos processed in CDN.

◆ Over 2.5 billion pieces of content (status updates, comments, etc.) are shared every month

◆ Facebook has more than 30,000 servers (this number was from last year)

Software on which Facebook expansion depends

Facebook is, to some extent, still a LAMP site, but it is much larger than normal LAMP to incorporate other elements and many services and modify the current practice.

For example:

◆ Facebook still uses PHP, but it has already created a compiler for it so that it can be divided into local code to open the Web server, thus improving performance.

◆ Facebook uses Linux, but he has made special improvements to network throughput.

◆ Facebook uses MySQL, but mainly serves as a Key-value persistent storage. Jions and server logic operations are performed on the Web server. Because it is easier to execute.

There are also self-compiled systems, such as Haystack, a highly scalable object storage system, used to store Facebook photos. There is also Scribe, a log system that can run on Facebook's massive log system.

Let's introduce the software used by the world's largest social network website.

Memcached

Memcached is one of the most famous software on the Internet. This is a distributed memory cache system used as the cache layer between the Web server and the MySQL server (because the database access is slow ). Over the years, Facebook has proposed some methods to optimize Memcached and peripheral software. Such as compressing network stack.

Facebook caches 10 TB of data on thousands of Memcached servers every moment. It may be the world's largest Memcached cluster.

HipHop for PHP

As a scripting language, PHP runs slowly compared with local programs. HipHop can convert PHP to C ++ code and then compile it to achieve better performance. Facebook relies heavily on PHP, which makes it more efficient for Web servers to run.

A small team of engineers spent 18 months developing HipHop on Facebook (only three people at the beginning) and it is now available.

Haystack

Haystack is Facebook's high-performance Photo Storage/Retrieval System (strictly speaking, it is an object storage, so it does not have to store photos ). It has a lot of work to do; there are more than 2 billion uploaded photos, and each one is saved in four different resolutions, so there are more than 80 billion photos.

It not only processes hundreds of millions of images, but also plays a critical role in operation. As we mentioned earlier, Facebook's service has about 1.2 million photos per second. this number does not include those on CDN. This is an amazing number. For details about Facebook's image storage, refer to the Facebook image storage architecture technology full analysis report by 51CTO.

BigPipe

BigPipe is a dynamic Web service system developed by Facebook. Facebook uses it to process each page by section (called pagelets) for optimal performance.

For example, the chat window is separated, and the news Feed is also separated. These pagelets can be used when a page is displayed, which is obtained when the page is displayed. Even if some projects are closed or mid-range, users can obtain some webpages.

Cassandra

Cassandra is a distributed storage system with no single point of failure. This is an important part of the NoSQL movement and has published the source code (it even becomes an Apache project ). Facebook uses it in the search function.

In addition to Facebook, some people also use it, such as Digg. But recently Twitter gave up Cassandra. For more information about Cassandra, refer to heading for freedom in 51CTO's topic? Cassandra database application guide.

Scribe

Scribe is a flexible log system, and Facebook is widely used inside it. It can process large-scale log records on Facebook and automatically process new log record categories. Facebook has hundreds of log categories ).

Hadoop and Hive

Hadoop is an open-source map-reduce implementation that allows it to perform operations on big data. Facebook uses this for data analysis (and we all know that Facebook already has a large amount of data ). Hive is originated from Facebook, making SQL queries for Hadoop more likely, but it is easier for non-programmers to use.

Hadoop and Hive are open-source (Apache projects) and have many followers, such as Yahoo and Twitter.

Thrift

Facebook uses several different languages and services. PHP is finally used for front-end, Erlang is used for chat, Java and C ++ are also used in multiple places, maybe there are other languages. Thrift is an internally developed cross-language framework that connects languages so that they can work together to enable interaction between them. This makes it easier for Facebook to continue its cross-language development.

Facebook has made Thrift open-source. More languages have been added to Thrift.

Varnish

Varnish is an HTTP accelerator that acts as a load balancer and caches content, which can then be delivered at lightning speed.

Facebook uses arnish to process photos and personal data images, processing billions of requests per day. Like other things, Varnish is open-source.

Other things that keep Facebook running smoothly

The software we have mentioned forms a Facebook system and helps run on a large scale. However, processing such a large system is a complicated task, so we will list some other things that keep Facebook running smoothly.

Gradual release and dark start

Facebook has a so-called keeper system that allows them to run two different systems for different users. This allows Facebook to gradually release new features, such as A/B testing, only for Facebook employees.

Keeper can also enable "dark start" for Facebook, which activates some features before users use some features (it is called a dark start because users are not aware of it ). This will serve as a stress test in the real world, helping expose some functional barriers and other problems before the official launch. Dark start is usually two weeks before the official start.

Profiling live broadcast system

Facebook carefully monitors its system. Interestingly, it is also responsible for monitoring the performance of every PHP function in the production environment. Checks the configuration and running status of each PHP environment. Use the open-source tool XHProf.

Gradually use the disabled feature to improve performance

If Facebook encounters performance problems during runtime, one way is to gradually disable less important features to enhance the performance of Facebook's many core features.

Things we did not mention

We didn't mention hardware-related things, but this is also an important part of improving scalability. For example, like other large websites, Facebook uses CDN to process static content. Facebook also has the huge data center, which can help him expand more services.

Facebook's open source plot

Not only does Facebook use (and help) open-source software such as Linux, Memcached, MySQL and Hadoop, but in many other cases, it also contributes a lot of internal development software.

Facebook is also open-source Tornado, a high-performance network server framework developed by the FriendFeed team. You can find the Open Source software list on Facebook's Open Source page.