Large-scale Internet technology architecture 1-Architecture Overview

The above coordinates point to the Silicon Valley, recently began to study the Internet distributed architecture, the cusp, on the tall, the special and the Geek friends share, mutual encouragement.

Internet Architecture

In recent years, the rapid development of the Internet, the big data era, booming years, we as a technical geek, need to keep pace, trends.

1 large website features

Large Web sites, whether e-commerce or social networking sites, often have the following features, such as high concurrency, low latency, massive data, scalability, HA 7x24, and data to illustrate: Google daily UV number 300 million, daily PV up to 35+ billion; Facebook week upload images more than 1 billion Taobao Double 11, the first minute UV up to 10 million. And these are our traditional applications can not be imagined, can not be a candle.

2 Architecture Evolution

Remember an American boss said, "Project is evolutionary not revolutionary!". The same is true of the Internet domain. With the development of business, concurrency, data storage, etc., the system began to evolve, divide and conquer. In fact, the computer world, including hardware, network, software is to introduce new layers, objects, services, so divide and conquer to simplify the problem of solving, and the real world is not so. Of course, divide and conquer, split solves some problems, but introduces another complex/interactive problem. Of course, divide and conquer, split solves some problems, but introduces another complex/interactive problem.

1. A server: Once remember the internet a few years ago, or in fact, now the small site is a classic lamp combination is done, that is, Linux, Apache, MySQL, PHP, and even application + file + database all in a server.

2. Three servers: With the business development, concurrency, data storage, etc. can not be met, start splitting; first, detach the application from the data; split into three servers: Application server, file server, and database server. Where the application server needs to be computed and therefore requires a more powerful CPU; The database server requires a fast disk and data cache, and the file server requires a larger hard disk.

3. Introduction of Caching: Continue to improve, introduce caching mechanisms to improve access to solve hot business. Current structure:

4. Cluster: To step 3, everything looks good. With the volume of traffic, traffic continues to soar, is the so-called happy annoyance, application server can not afford, direct downtime, leading to all business paralysis, is the so-called single point of problem. We introduce clusters to solve single point problems and provide scalability. Let's look at the current architecture a little bit:

Note that at this point we need to introduce a load balancer to coordinate which application server the user accesses, if the rich owner can choose F5, or LVS, nginx, etc. At this point, it all looks so perfect together.

5. Database read and write separation: Subsequently, the site's access bottleneck continues to exist under high concurrency. The reason, although the introduction of the cache to solve most of the data read access, so that there will be some read operations such as cache hit, or cache expiration, such as the need to access the database, of course, all write operations need to directly access the database; Continue to divide and conquer. At present, the main database has provided the master-slave hot standby function, automatic backup synchronization; We can take advantage of this feature. Read all walk from the database, write the master database. This cleverly solves the separation of read and write.

6. Introducing reverse Proxy and CDN: Continue to increase access speed and provide a better user experience. It is emphasized here that the website access speed/latency is positively correlated with the user churn rate. At present, the main mainstream means are CDN and reverse proxy; both principles are based on the caching mechanism. The difference is that the CDN is deployed in the network provider room, and the reverse proxy is not familiar in its own website room.

As you can see, the introduction of CDN and reverse proxies is expected to return data to the user as soon as possible. Popular Science:CDN (Content Delivery Network) is an ancient thing that has appeared at the beginning of the development of the Internet. A group of MIT elites have found that if people anywhere can quickly open their website, they need to publish their web pages to the nearest place of the audience, just as they do in churches around the world. So, they implemented this release with a simple caching-mirroring approach. The first network of the church was Yahoo. The CDN uses DNS delegation to guide the end visitor to find the ideal cache or mirror site, which is a service based on domain name. Reverse Proxy : The reverse proxy method refers to a proxy server that accepts connection requests on the Internet, then forwards the request to a server on the internal network and returns the results from the server to the client requesting the connection on the Internet, Reverse. At this point the proxy server appears as a reverse proxy server externally. At present the most widely used to be nginx, from the Russian people Igor Sysoev of the handwriting.

7. Distributed File/Database system: Continue to operate, of course, the change is because of the time, business development, our system with the evolution, it is not to evolve. The file system is relatively simple and introduces a distributed file server. The database is more difficult to split. After splitting the database, you need to introduce a unified data access module.

8. NoSQL and search engines: the famous NoSQL finally comes in, the relationship data again how to split, with the increase in data volume, still face performance bottlenecks, such as multi-table association, full table scan and other thorny issues. NoSQL says stress-free, such as hbase, originating from the Internet, is naturally born to the Internet and inherently distributed and scalable.

9. Business spin-offs: Well, the technical architecture you can think of is already in use, already 8 axe. Had to operate business, haha. In fact, the business includes business lines, business processes, if you can plan the business well, including the process, can solve a lot of problems, including performance, such as time-sharing sales promotion. Back to business split, usually according to business scenarios, such as e-commerce business including home, shops, orders, transactions, shopping carts and other product lines to split. After splitting, how to series together. There are usually several ways to apply between applications, such as through hyperlinks, Message Queuing, or by associating a share with the same database.

10. Distributed services: Out of the big recruit, is so-called split-long will be combined. The demolition of so many, so fine, inevitably caused by the complexity of the number of levels of ascension, deployment, maintenance more and more difficult, and there must be a lot of common functions, such as user management, landing management, product management, transaction payment and other public services. Well, consolidate services and invoke business services through distributed services.

In this case, the rack has been able to solve the vast majority of Internet problems, of course, it is easy to see the picture, the real implementation of the architecture requires a lot of manpower, material resources, financial resources. Ali was also a year after decades of evolution.

3. Internet Architecture

Let's take a look at a complete distributed internet Architecture diagram:

You know. It can be seen that the innovative business has created innovative technologies and is a perfect combination of business and technology. Here, do not put the cart before the horse, some in the direction of business has not been clear, the blind construction of large-scale Internet architecture, is rich + burn money. There are also small startups that blindly mimic Facebook, BAT, for technology and technology.

In addition, the development of Internet technology here, has entered the cloud computing, container service phase, small and medium-sized enterprises do not need to re-build the wheel, directly with the large companies to provide cloud computing, container services can, thereby concentrating on business development.

Well, a simple chat, the children woke up, to start "escort". Note: Many ideas are referenced from the Web, as well as the technical books.

Public number: Technical Geek techbooster