Youku.com Architecture Experience

This qcon (Beijing) conference website architecture case study this track, although there are not many topics, but the curriculum design considerations are relatively wide coverage. As a video website representative, Youku brings together a technology sharing that includes many practical experiences. Qiu Dan (Vice President of Youku network development and core architect) may be busy with the company and will not arrive at the venue until noon the next day. Just kidding me, 'How do you think so many people are small meetings '...

Cache

Cache Gold principle:Bringing data closer to "> CPU.

CPU--> CPU Level 1 cache --> Level 2 Cache --> memory --> hard disk --> lan --> Wan

I talked about Youku's internal project, which is for large file caching. Currently, Squid's write () User process space is consumed in open-source software. Lighttpd 1.5 AIO (asynchronous I/O) reads files to the user memory, resulting in low efficiency. Youku does not use memory for caching (avoid Memory copying and avoid Memory locks ). It is worth noting that the caching technology is prone to misuse and has some side effects. For example, if you receive a notification from your eldest brother that you want to remove a video, it is troublesome to cache it.

Database

Youku has made many attempts to database sharding, And the implementation effect should be good. DB read/write splitting has rich experience.

SSD is enabled to improve the I/O capability of the database. Six SSDS for raid. I sent a message on Twitter that Youku uses SSD. Many of my friends think it is used to store video files. Here I need to clarify that it is only for partial use.

Network throughput Optimization

This is a section I strongly want to add. Network Optimization, video websites certainly do well. The keyword in this section is "event-driven". A profound sentence is "epoll drives today's Web". Indeed, nowadays, epoll is the selling point for many popular Web components.

For more information, see the c10k problem (I have always wanted to translate this page, so I can't spare time) and libevent. If they are working on the Internet and encounter scalability problems, these two information points still cannot be avoided.

The last example is for the memcached agent, which can be compared with the memcached processing in the Facebook architecture.

At the end of the speech, someone asked Youku what is special about the video cache? The answer is that a large video may be divided into multiple small files, so that the buffer performance is better-(parallel)... in fact, accessing Youku is indeed a little faster than Tudou.

From: http://www.dbanotes.net/arch/youku_arch.html