Database and distributed architecture strategies

I have been learning about the internet's Big Data High concurrency architecture and various optimization strategies.

In the initial stage of construction, you can consider the following two aspects:

 

1. database design strategies. In terms of database scalability, vertical and horizontal Data Division can be easily considered, namely:

 

Vertical partition/sharding: stores data in different formats to different databases, also called database shards.

Horizontal partition/sharding: stores data in the same data format to different databases. This article focuses on this, also called table sharding.

To understand and remember the two splitting methods, the brain can recall your database table. The horizontal field description is used, and the vertical field is used to record data. Therefore, vertical segmentation refers to dividing table fields into several pieces by vertical lines, that is, dividing a table into N tables with different fields. horizontal segmentation means horizontal dashes and dividing data records into several pieces, the data fields of each block are the same, but the record values are different.

 

 

1. Considering the large amount of data in the future. In database design, database sharding should be considered. For example, if the database is also news information, it should be regarded as political news or entertainment news.

Sports News, etc. Create their respective news tables to facilitate storage.

 

2. Considering the large traffic volume in the middle and late stages. Table sharding is considered during design. For example, for entertainment news, you can split the data into multiple tables based on the date, character, and other fields for storage to increase the query speed.

 

3. the above two cases can be used in combination. When the data volume is tens of millions and the concurrency is tens of thousands, consider creating a cache table. A temporary table is created between the actual table and the user. when accessing data, the temporary table is first accessed. If it does not exist, it is obtained from the actual table and then placed in the cache table, at the same time, cache table data is synchronized to the actual database through background threads, and the synchronization time can be performed according to system requirements.

 

2. program optimization strategies. To build a system, you must carefully consider the appropriate framework. If the company or individual has enough time and energy, you can develop the system framework on your own.

 

1. Framework Construction

1 ). the systems built by structs2 + spring3 + hibernate3 are updated every year, constantly changing and improving. This combination is best suited for building enterprise information systems, however, it is difficult to apply it to a web system with high Internet access and high concurrency. A variety of optimization strategies, such as static, cached data, and JS compression, need to be adopted.

 

2 ). according to the test information on several popular frameworks on the internet, spring MVC shows that spring controller is more efficient than structs, jdbctemplate is a simple encapsulation of Java JDBC, and the efficiency of SQL operations is certainly higher than that of structs combined with hibernate.

 

3). The architecture access efficiency and development efficiency of spring + hibernate are both good and suitable for enterprise development and Internet development.

 

4). Spring + JPA (hibernate) this cooperation framework is currently very efficient and most suitable for Internet applications. I personally recommend it.

 

2. Static. The recommended frameworks are freemaker and velocity.

 

3. data cache.