Server architecture of Flickr

Previously, Cal Henderson, an architect of Flickr, gave a comprehensive description of its architecture in the ppt "Flickr: Web Services.

 

John allspaw from the Flickr O & M team has two lamp slides: hardware layouts for lamp installations and capacity planning for lamp. However, it should be a summary of some experiences in the evolution and O & M of the Flickr architecture, some server architectures are also disclosed.

 

John allspaw also emphasizes the importance of measurement. He also has a written test of benchmark, which is consistent with our experience, whether it is for open-source software such as Cassandra benchmark, or the performance tests of self-developed processes are significantly different from those of post-launch operations, making capacity planning almost useless. It is basically necessary to perform more reliable planning based on the actual application load after the phased release.

 

 

 

 

The magic lamp Federation at Flickr: doing billions of queries per day from the DBA Dathan Vance pattishall of Flickr

It means how they do shard. Of course, some small tips mentioned in it are also quite useful. For example, swapiness set to 0, we had our own server processes blocked by swapiness. The ticket server mentioned in the above section isTicket servers: distributed unique primary keys on the cheapI have explained in detail.

 

 

Mikhail panchenko, a former employee of Flickr, presented the evolution of the architecture in the title of the Flickr architecture at the strange loop 2010 Conference (THe evolution of the Flickr ArchitectureOn infoq, you can watch the video. Here you can download the slides.

 

That's right. It's a former employee who has already left Flickr, but is talking about the architecture of Flickr.

 

After reading the video, I didn't talk much about the specific implementation and design of the technical architecture. Most of the time, I criticized the PHP code written in Flickr and his views on various popular technologies.

 

This includes mogodb downtime events earlier than foursqure, and he does not think it is a mogodb issue.

 

 

"This isn' t a MongoDB problem.

It's an "It's nosql, so I don't have

To think about it "problem ."

 

 

Flickr insisted on using MySQL. He said this:

 

 

"As far as I can tell,

Amount of effort spent

Making various datasets fit

Nosql databases is

Equivalent to the time it

Takes to get good at MySQL"

 

 

The three have to he mentioned are very real:

"

 

• You have to know what it is you need and

What your limits are

• You have to monitor for those limits

• You have to have a plan for what you're

Gonna do to continue avoiding those limits

 

"

They also used redis. In its offline tasks system, they spoke highly of redis.

 

 

Cal Henderson's other magic light,Expo08nyc_moving_pictures. PPS, which is about how to implement video in Flickr (the summary of several video formats is also profound ).