MySQL supports the deployment of millions of High-concurrency websites

MySQL supports millions of trafficThe deployment method of highly concurrent websites is what we will introduce in this article. Once we understand this part, we can use MySQL to support websites with millions of High-concurrency traffic, next we will introduce this part.

1. MySQL Server Cluster

Because small companies have limited funds and costs, and few technical technicians have been involved, this method is naturally rejected by everyone.

Here is my personal understanding! Cluster creation not only imposes additional charges, but also brings great technical challenges. This is not realistic for our company's current situation. A cluster is nothing more than transferring the pressure on one server to two or more servers. I understand this. Maybe I understand it incorrectly. Please advise.

2. Divide and conquer

This method is similar to the cluster, but the Statistical Code is run on different servers. As the company has many low-configuration servers running tens of thousands to hundreds of thousands of IP addresses, there is still no problem, we can divide millions of traffic into dozens, hundreds, and hundreds of thousands.

Advantage: the current resources are fully utilized to solve the current problems.

Disadvantage: This method is not a long-term solution. Sooner or later, problems will occur. In addition, it is troublesome to collect statistics.

3. Modify the Statistical Code

Since we used to determine whether the IP address exists or not before inserting data, the processing of routes and so on virtually increases the pressure on the server, so everyone changes the Statistical Code to one and inserts it into the database, regardless of whether it is processed after November 21.

This method basically keeps the data of the day. However, due to the huge amount of data, the server is still killed when going back and forth, in addition, the indexes reserved during data structure design during insertion also greatly consume a lot of server resources.

It is slow to remove the index to the final processing, and the loss is worth the candle.

4. Modify the statistical method

The last method is very effective. What is that!

Here we will mainly introduce this method:

A. Keep the original data structure unchanged, and store all data in A certain structure into A file

Structure: It can be xml, json, or any regular data emission you want.

For example:

1) 221.2.70.52, http://www.baidu.com, windowxp \ r \ n

2) 221.2.70.52, http://www.baidu.com, windowxp \ r \ n

Write File: fopen, fwrite ?? No. I will introduce a very useful technique. You may know it, but I am not familiar with any cainiao like me. I am using error_log. Isn't this an error log? It's very convenient.

Format:

1 error_log ("content", 3, "/date. dat ");

I will not talk about his usage here. If you don't understand it, you can check the manual.

B. Data File naming

Why is the file name required? What is the difference between writing data to a file without any processing? Everything we do is useless.

First, the value obtained from date ('ymmdh') is 2008121112, in this way, the data is generated every hour without being too large.

IP address application: because many data operations are performed by one IP address, it is very convenient to put data of the same IP address in one file and process it later. Please refer to the introduction below. Here we take the IP address into three digits as part of the file name.

C. Data Import and Processing

The data of the two operations on the same day will be retained a lot. The next step is not very important. But here I will talk about my thoughts.

Warehouse receiving pre-processing:

As mentioned above, three identical IP addresses are put into one file for processing before they are put into the database. First, you can use the most stupid method to split the data into N pieces of data. Deleting duplicate data.

If an IP address browses multiple pages, PV can be calculated here, And the accessed page is processed to form new data.

Import method:

Here we will introduce the DATA import method: 1 $ SQL = "LOAD DATA INFILE '". $ file. "'into TABLE 'test' fields terminated by ', 'Lines terminated by' \ r \ n' (2 3 'time', 'md5' 4 5 )"; this statement can be said to be very fast when importing hundreds of thousands of data. I have not introduced the usage here. You can refer to the Manual if you do not understand it, therefore, the solution to the bottleneck of the database does not have to add equipment or change the database. You only need to change your mind to solve many problems.

The above is the deployment method of MySQL supporting millions of High-concurrency websites. This article will introduce it here. I hope this introduction will be helpful to you!