On the technical architecture of Instagram service

The current mobile Internet photo sharing and light blog services are extremely prosperous. Class Instagram photo-sharing services, foreign services include Instagram, Color, Path, http://www.aliyun.com/zixun/aggregation/38187.html ">picplz, Foodspotting, such as the domestic Instagram including push map, Pushpin, with the beat, Ding Chai, Leshing Mody, Bubble Pat and so on. and Foreign light blog services, including Tumblr, Zpad, posterous and so on, the domestic frivolous blog services include point, push him and so on.

In addition to their interest in the products and business models of these services, the technical architecture of the backend is also of interest. But even a Web site like highscalability.com does not seem to have much to say about the technical architecture of such new services.

This kind of service mainly involves the massive photograph processing, the client and the server side communication, with other service synchronization, and so on, simple picture system deployment architecture diagram.

From a technical architecture standpoint, these services need to address some of the typical technical challenges:

1, with other SNS community service synchronization is the use of client synchronization or server-side synchronization?

Since the basic authentication interface is gradually eliminated, such as Twitter, Sina Weibo, and most of the services are basically using the OAuth interface, the need for the client to actively initiate authorization operations, can not be initiated by the server side.

Interfaces other than the OAuth interface if there are problems with client synchronization:
1, if you want to sync more SNS community, the client should be for different SNS community synchronization, inefficient, especially to occupy a large bandwidth flow
2, if the SNS community interface slightly change, the need for client upgrades, very troublesome
3, for Twitter to be G.F.W blocked account, the client needs to flip the wall to be able to sync
Conclusion: The client synchronization method is not adopted. The client passes the related request to the server and completes the synchronization operation by the server side.
2, because the class Instagram service, the picture is the main content form. Therefore, the need to focus on the image server architecture, especially the situation of massive pictures. The more realistic scheme can refer to Picture Server selection scheme, the ideal scheme can refer to the haystack of Facebook.

In addition, due to the large number of thumbnail processing, the Gearman distributed computing framework can be used to +graphicsmagick the thumbnail image processing.

3, due to the related SNS Community Interface Service synchronization, in order to ensure the performance of the system, the synchronization service should be separated from the core service, and the core service, after receiving the client request, passes the data that needs to be synchronized to the synchronization server through Message Queuing, and the synchronization server asynchronously completes the synchronization of the related interface services.

4, because it is content-oriented services, you can use Redis and other NoSQL to store OAuth Access Token, microblogging, user registration information, such as the need for persistent data

5, the Wall problem

These services generally provide the ability to synchronize with the existing SNS community services. The technology is relatively easy, only need one to handle the interface provided by each service provider, even if the existing interface is not perfect, it can be done through the GRASP interface message simulation.

But if you need to upload a user's picture with Twitter, Facebook and other foreign services account synchronization, because these services are the wall off, if the server itself in the domestic, you can put a synchronous proxy server abroad to synchronize with the domestic server, and then the server to complete the synchronization with foreign services. If the server is placed abroad, it is relatively worry, but also to consider the service is a bit well-known after the possibility of the wall off the service itself.

6, client and server-side communication related technology implementation

Communication protocol data compression transmission between client and server;

Client for such as photo preprocessing (such as the appropriate reduction of resolution), multithreading concurrent fragmentation transmission, breakpoint renewal processing;

The client locally stores, caches, edits the data and the server side synchronously to process the problem under the off-line state;

Client concurrent Request picture server, Business Server (different domain name), improve the efficiency of concurrent processing

7, search engine services in addition to the search for text content, but also involves geographical information search, real-time search problems, and Lucene and SOLR for this support relative to Sphinx and other search engines better, so the use of SOLR or Lucene.

If the service itself is to provide interface services, it may consider the Pubsubhubbub protocol.

Source: http://www.yeeach.com