Architecture applications for high concurrency data acquisition (Redis applications)

The starting point of the problem:

recently, in order to develop the needs of the company, to expand the user's information collection, the amount of each user's acquisition estimated about 3 W. If the user volume increases, will be a large number of acquisition to 3 W times the growth, but also meet the daily business needs, especially the report data must be completed within the specified time.

Technical Barriers:

1. In the face of the increase in user volume, the record number of 3W times the growth, how to ensure that these records in a relatively fast time to enter the storage medium.

2. To cope with the increase in user volume, how to complete the acquisition within the specified time, increase the processing power of hardware equipment or use more servers to process requests.

3. Is the growth of the server capable of supporting existing expansion capabilities?

Schema implementation:

Technical items to use:

1. Service-oriented development thinking

2. Queue Service

3. Multitasking Concurrent execution

4. Distributed service Management

5. Applications for Redis

6. Data Sub-table (partition) Implementation and application

7. Redis asynchronous delay synchronization to the database

The architecture diagram is as follows:

Specific implementation:

1. The queue service can use HTTPSQS

2. Multitasking concurrent execution, PHP version can use Parallelcurl, can control the number of concurrent

3. You can use the PHP redis to achieve the operation of Redis

4. Data table sub-table or partition, you can write yourself, the principle is very simple, you need to guide the private vomited

Summarize:

The whole program is not very complex, in the processing of big data direction this piece, the basic principle is the same, the uncontrollable factors to grasp, so in the multi-tasking this piece must be controlled to a reasonable amount, in addition to the Redis cache also support distributed cluster, the increase of Redis server will not affect the upper code changes, Relatively speaking, the capacity for expansion is relatively good.

Architecture applications for high concurrency data acquisition (Redis applications)