Nosql architecture practice (I)-supplemented by nosql

[ArticleAuthor: Sun Li link: http://www.cnblogs.com/sunli/ updated by: 2011-2-21]

The previous two articles, "Why use nosql" and "relational database or nosql Database", roughly introduced why nosql should be used and when nosql should be used. I often feel confused when I see the introduction of nosql, but I don't know how to officially use it in my project. The major reason is that the thinking is fixed in MySQL. The most common question they ask is that nosql is used. How can I perform relational queries. Next, let's take a look at how to use nosql in our system.

How can we introduce nosql into our system architecture design? We need to analyze the business scenarios of our system and what types of data are suitable for storing in nosql databases, what type of data must be stored in a relational database. Clarify the role of the introduced nosql database to the system, what problems it can solve, and possible new problems. Below we will analyze several common nosql architectures.

(1) Using nosql as an image

It does not change the original architecture that uses MySQL as the storage, uses nosql as the auxiliary image storage, and uses the advantages of nosql to help improve performance.

Figure 1-nosql is an image (CodeCompletion Mode)

 // Sample pseudocode for writing data

  //  Data is the data object we want to store  
  Data. Title  =  "Title ";
Data. Name  =  "Name ";
Data. Time  =  "  2009  -  12  -  01    10  :  10  :  01  ";
Data. From  =  "  1  ";
ID  =  DB. insert (data );  //  Write to MySQL database  
  Nosql. Add (ID, data );  // Write data to a nosql database using the auto-increment ID generated by writing data to MYSQL as the primary key 
 
If you have data consistency requirements, you can use the following method:
 
 //  Sample pseudocode for writing data
  //  Data is the data object we want to store  
  Bool status  =  False  ;
DB. starttransaction ();  //  Start transaction  
  ID  = DB. insert (data );  //  Write to MySQL database  
  If  (ID  >  0  ){
Status  =  Nosql. Add (ID, data );  //  Write data to a nosql database using the auto-increment ID generated by writing data to MYSQL as the primary key  
  }
  If  (ID >  0     &&  Status  =  True  ){
DB. Commit ();  //  Commit transactions  
  }  Else  {
DB. rollback ();  //  Failed to roll back  
 } 
 
The above code may seem a little troublesome, but you only need to make a unified encapsulation on the DB class or ORM layer to achieve reuse. Other code does not need to be modified.
 
 
This architecture adds an auxiliary nosql storage layer to the original mysql-based database architecture, with a small amount of code and a low technical difficulty, but plays a significant role in scalability and performance. OnlyProgramAfter writing data to the MySQL database and writing data to the nosql database, MySQL and nosql will have the same image data. In some places where the data can be queried based on the primary key, use the efficient nosql database query, this saves MySQL queries and uses nosql's high performance to resist these queries.
 
 
 
 
Figure 2-nosql image (synchronous Mode)
 
 
Instead of program code, MySQL synchronizes data to nosql. This mode is a variant of the above, and is transparent to writing but more technically difficult. This mode is suitable for existing complex old systems. Modifying the code is not easy to implement and may cause new problems. It is also suitable for synchronizing data to multiple types of storage.
 
 
The implementation of MySQL-nosql synchronization can be implemented using the MySQL UDF function and MySQL BINLOG parsing. It can be implemented using existing open-source projects, such:
 

  
  
 
   
   
 
    
MySQL memcached udfs: Operate the memcached protocol from a UDF.
 
    
Mysql-UDF-http: an open-source mysql-UDF in China: Operate the HTTP protocol through UDF.
 
   
 
  
  
With these two MySQL UDF libraries, we can use MySQL to transparently process memcached or HTTP, as long as there is a nosql database compatible with memcached or HTTP, then we can use MySQL to synchronize data. Combined with lib_mysqludf_json, the UDF and MySQL trigger functions can be used to automatically synchronize data.
 
 (2) MySQL and nosql combinations 
 
MySQL only stores small fields to be queried, and nosql stores all data.
 
 
 
 
Figure 3-MySQL and nosql combinations
 
 //  Sample pseudocode for writing data

  //  Data is the data object we want to store  
  Data. Title  =  "Title ";
Data. Name  = "Name ";
Data. Time  =  "  2009  -  12  -  01     10  :  10  :  01  ";
Data. From  =  "  1 ";
Bool status  =  False  ;
DB. starttransaction ();  //  Start transaction  
  ID  =  DB. insert ("insert into table (from) values (data. From )");  //  Write Data to the MySQL database and write only the fields to be queried from  
  If  (ID  > 0  ){
Status  =  Nosql. Add (ID, data );  //  Write data to a nosql database using the auto-increment ID generated by writing data to MYSQL as the primary key  
  }
  If  (ID  >  0     &&  Status  =  True ){
DB. Commit ();  //  Commit transactions  
  }  Else  {
DB. rollback ();  //  Failed to roll back  
  } 
 
Store small fields, such as numbers and times, to be queried in MySQL, and create corresponding indexes based on the query, large text fields are stored in nosql. During the query, we first query the primary key of the data from MySQL, and then directly retrieve the corresponding data from nosql.
 
 
This architecture mode integrates the functions of MySQL and nosql and performs their respective duties, so that MySQL is responsible for processing relational storage that is good at, and nosql is used as the storage of data. It has the following advantages:
 

   
   

    
    
 
     
Saves MySQL I/O overhead. Because MySQL only stores small fields that need to be queried and is no longer responsible for storing large text fields, this saves the space overhead of MySQL storage and MySQL disk Io. We used this optimization to reduce a 40 Gb MySQL table to several hundred MB.
 
     
Improves the cache hit rate of MySQL queries. We know that the query cache is invalid at the table level. Once the MySQL table is updated, it will become invalid. After this field separation, if the updated field is not stored in MySQL, this does not affect query cache. Nosql caches are generally row-level, and only expire with the cache of updated records.
 
     
Improves MySQL master-slave synchronization efficiency. Because of the decrease in MySQL storage space, the synchronized data records are also reduced, and some data updates fall into nosql rather than MySQL, which also reduces the number of times MySQL data needs to be synchronized.
 
     
Improves the speed of Mysql Data backup and recovery. Because the data stored in the MySQL database is reduced, it is easy to see that the speed of data backup and recovery will be greatly improved.
 
     
It is easier to expand than before. Nosql is naturally easy to expand. After this optimization, MySQL performance is also improved.
 
    
 
   
   
 
For example, mobile phone phoenixnet is this architecture http://www.cnblogs.com/sunli/archive/2010/12/20/imcp.html
 
 Summary 
 
The architecture supplemented by nosql is centered on the idea of MySQL architecture. It only helps to increase nosql in the previous architecture to improve its performance and scalability. This architecture is easy to implement, but can achieve good results. If you want to introduce nosql in the project, or your MySQL-based system is currently experiencing bottlenecks, I hope this article will help you.
 
 
About the author
 
Sun Li is currently responsible for R & D of the underlying group at Phoenix. He once worked in Sohu and ku6. Years of Internet experience and program development, development of distributed search engines, high concurrency, big data website system architecture optimization, high availability, scalability, distributed system cache, database sharding) have rich experience in O & M monitoring and automatic O & M control. Phplock, author of phpbuffer, an open-source project. Recently, I developed a nosql database storage inetdb, which is a nosql database enthusiast. His Sina Weibo is: http://t.sina.com.cn/sunli1223
 
 
This article has been launched on the infoq Chinese site and is copyrighted. The original Article is "nosql architecture practice (1)-supplemented by nosql". If you need to reprint it, please attach this statement. Thank you.
 
 
Infoq Chinese site is an online independent high-end technical staffCommunity, For Java ,. net, Ruby, SOA, agility, architecture and other fields to provide timely and in-depth information, high-end technology conferences such as qcon, offline technology exchange activities qclub, free mini book download such as architect, etc..