Solution summary for big data and high concurrency

Now, the software architecture has become more and more complex, a lot of technology endless, dazzling, solve this problem, is to simplify the complex problem, the core is to grasp the essence.

The software is just beginning to realize the function, as the amount of information and the increase of users, big data and high concurrency become the software design must consider the problem, then big data and high concurrency nature is what?

The essence is very simple, one is slow, one is waiting. The two are interrelated, because the slow, so to wait, because so, so slow, solve the slow, also solved, and so on, resolved the slow.

The key is how to solve the slow and so on, the core one is short, one is less, the other is shunt.

Short refers to short paths. The typical MVC structure is to request->controller->model->dao->view and then return the page to the user. If you want to be short,

1, page Static-users can directly get the page, do not go so many processes, the comparison is applicable to the page is not updated frequently.

2, using Cache-the first fetch of data from the database is quasi-extracted, and then saved in the cache, you can extract the data directly from the cache. There is, however, a mechanism to maintain cache and database consistency.

3, the use of stored procedures-those processing a single request requires multiple access to the database operations, you can integrate operations into the storage process, so that only one database access can be.

4, bulk read-high concurrency, multiple requests can be merged into one at a time to reduce the number of accesses to the database

5, deferred modification-high concurrency, you can put multiple modification requests, first saved in the cache, and then periodically save the data in the cache to the database, the risk is that the loss of data in the cache can be lost,

6, using an index-index can be seen as a special cache, and using the index as much as possible requires the exact value of the indexed column in the WHERE clause.

Less refers to less data being queried.

1, sub-table-the original contents of the same table can be divided into multiple tables according to the region, category, a very simple idea, but to try to avoid the multi-table associated query.

2, separate active data-for example, login user business, a lot of registered users, but there are very few active users, you can save the active user dedicated to a table, query is to query the active table, not the words and then check the summary table, which is similar to the cache.

3, block-database level optimization, the program is transparent, query big data only to find the corresponding block on the line.

Shunt three kinds.

1, cluster-distributes concurrent requests to different servers, either as a business Server or as a database server.

2, distributed-distributed is the allocation of multiple business logic for a single request to multiple servers, so that many logic can be processed synchronously, typically using a particularly complex business request.

3,CDN-in the domain name resolution level of the diversion, for example, South China's user request distribution to South China's servers, central China's users request to allocate to central servers.

Temporarily summarize so many programs, with the progress of technology, there will be more programs appear, together grow and progress .....

Solution summary for big data and high concurrency