Optimization-high concurrency Processing Solution

Main points of concurrent processing:
HTML static
Image server Separation
Database Cluster
Cache
Server Load balancer

Large websites, such as portal websites. In the face of a large number of user access and high concurrency requests, the basic solution focuses on the following aspects:
High-performance servers
High-performance Databases
EfficientProgramming Language
High-performance WEB containers

The solutions provided above also mean a greater investment to a certain extent, and these solutions have bottlenecks and do not have good scalability, I will talk about some of my experiences from the perspectives of low cost, high performance, and high scalability.

• HTML static
  As we all know, the most efficient and least consumed HTML pages are purely static html pages, so we try our best to make the pages on our website adopt static pages, this simplest method is actually the most effective method. However, for websites with a large amount of content and frequent updates, we cannot manually implement them all, so we have a common information publishing system CMS, news channels such as the portals we often visit, and even other channels, are managed and implemented through the information publishing system, the information publishing system can automatically generate static pages based on the simplest information input. It can also provide channel management, permission management, automatic crawling, and other functions. For a large website, having an efficient and manageable CMS is essential.

  In addition to portal and information publishing websites, websites with high interaction requirementsCommunityFor type websites, static as much as possible is also a necessary means to improve performance, will post in the community,ArticleReal-time static, and re-static when there are updates is also a lot of use strategy, such as the mop hodgedge is the use of such a strategy, Netease community and so on.

  At the same time, HTML static is also a method used by some caching policies. For applications that frequently use database queries but have little content updates in the system, you can consider using HTML static, for example, the public setting information of the Forum in the forum. Currently, mainstream forums can manage the information in the background and store it in the database. In fact, this information is largely used by the front-endProgramCalling, but the update frequency is very small. You can consider static content during background updates to avoid a large number of database access requests.

• Image server Separation
  As we all know, for Web servers, images, whether Apache, IIS or other containers, consume the most resources. Therefore, it is necessary to separate images from pages, this is basically a policy adopted by large websites. They all have independent image servers and even many image servers. This architecture can reduce the pressure on the server system that provides page access requests, and ensure that the system will not crash due to the problem of image slices. On the application server and image server, different configurations can be optimized. For example, Apache can provide as few loadmodules as possible when configuring contenttype to ensure higher system consumption and execution efficiency.
• Database Cluster
  Large websites have complex applications, and these applications must use databases. In the face of a large number of accesses, database bottlenecks will soon become apparent. At this time, a database will soon fail to satisfy applications, therefore, we need to use a database cluster or database table hash.
  In terms of database clusters, many databases have their own solutions, and Oracle and Sybase all have good solutions. The commonly used MySQL Master/Slave is also a similar solution, you can refer to the corresponding solutions to implement the database.
  As the database cluster mentioned above is limited by the DB type used in terms of architecture, cost, and expansion, we need to consider improving the system architecture from the perspective of applications, database table hashing is a common and most effective solution. We install business and application or function modules in the application to separate the database. Different modules correspond to different databases or tables, then, a page or function can be hashed in a smaller database according to certain policies, such as user tables and user IDs, in this way, the system performance can be improved at a low cost and the scalability can be improved. Sohu's Forum uses this architecture to separate forum users, settings, posts, and other information from databases, then, hash databases and tables based on sections and IDs for posts and users. In the end, simple configuration in the configuration file allows the system to add a low-cost database at any time to supplement system performance.
• cache
  the word cache has been used in many aspects. The cache in website architecture and website development is also very important. Here we will first describe the two most basic caches. Advanced and distributed caching are described later.
  for architecture caching, anyone familiar with Apache can know that Apache provides its own cache module, or use the added squid module for caching, both methods can effectively improve Apache's access response capabilities.
  cache for website program development. Memory Cache provided on Linux is a common cache interface that can be used in Web development, for example, during Java Development, you can call memorycache to cache and share some data, and some large communities use this architecture. In addition, when using Web language development, various languages basically have their own cache modules and Methods. php has a pear cache module, and Java has more ,. net is not very familiar with, I believe there must be.
• Server Load balancer
  Server Load balancer is the ultimate solution for large websites to solve high-load access and a large number of concurrent requests.
  Server Load balancer has been developing for many years. There are many professional service providers and products to choose from. I personally have some solutions, including two architectures for your reference.
  Hardware layer-4 Switching
  The layer-4 Exchange uses the header information of the layer-3 and layer-4 information packets to identify business flows based on the Application interval and distribute the business flows of the entire interval segment to appropriate application servers for processing. The layer-4 switching function is like a virtual IP address pointing to a physical server. Its transmission services are subject to a variety of protocols, including HTTP, FTP, NFS, telnet, or other protocols. These services require complex load balancing based on physical serversAlgorithm. In the IP address world, the service type is determined by the TCP or UDP port address of the terminal. The application interval in the layer-4 switch is determined by the source and terminal IP addresses, TCP and UDP ports.
  In the field of hardware layer-4 switching products, there are some well-known products to choose from, such as Alteon and F5. These products are expensive, but value for money, it provides excellent performance and flexible management capabilities. Yahoo China used three or four Alteon servers on nearly 2000 servers.
• Images
  Images are often used by large websites to improve performance and data security. The image technology can solve the differences in user access speed caused by different network access providers and regions, for example, the difference between Chinanet and EduNet has prompted many websites to set up image sites in CERNET to regularly update or update data in real time. In terms of image details, I will not elaborate too deeply here. There are many professional solutions and product options. There are also low-cost software implementation ideas, such as rsync on Linux and other tools.

 

 

From: http://www.cnblogs.com/aaa6818162/archive/2009/03/19/1416395.html