NoSQL Architecture Practice (i)--A NoSQL supplement

The previous "Why Use NoSQL" and "relational database or NoSQL database" two articles generally explain why NoSQL is used, and when to use NoSQL. Often have friends encounter confusion, see the introduction of NoSQL, feel very good, but do not know how to formally use their own projects. The big reason is that thinking is fixed in MySQL, and the question they ask most is using NoSQL, how do I do relational queries. So next, let's look at how to use NoSQL in our system.

How to introduce nosql into our system architecture design requires analysis based on the business scenario of our system, what type of data is suitable for storing in a NoSQL database, and what type of data must be stored using relational databases. The explicit introduction of the NoSQL database gives the system the role it can solve, as well as the new problems it may bring. Here we analyze several common nosql architectures.

(i) NoSQL as an image

Does not change the original MySQL as a storage architecture, using NoSQL as a secondary image storage, with the advantages of NoSQL to help improve performance.

Figure 1-nosql for mirroring (code completion mode)

Example pseudo code for writing data//data the data object we want to store data.title= "title"; Data.name= "Name"; Data.time= "2009-12-01 10:10:01"; data.from= "1"; Id=db. Insert (data),//write to MySQL database Nosql.add (id,data);//write to the NoSQL database with the self-increment ID that is generated by MySQL

If you have data consistency requirements, you can use them as follows

Example pseudo code for writing data//data for the data object we want to store bool Status=false; Db.starttransaction ();//Start Transaction id=db. Insert (data),//write to MySQL database if (id>0) {     status=nosql.add (id,data);//write to MySQL generated by the self-increment ID primary key to the NoSQL database} if (id>0 && status==true) {     db.commit ();//COMMIT Transaction}else{     Db.rollback ();//unsuccessful, rollback}

The above code may seem a bit cumbersome, but only need to be in the DB class or ORM layer to make a uniform package, it can be reused, no other code to make any changes.

This architecture adds an additional layer of nosql storage to the original MySQL database-based architecture, with a small amount of code and little technical difficulty, but has a significant impact on scalability and performance. Only when the program is written to the MySQL database and written to a NoSQL database, so that MySQL and NoSQL have the same mirrored data, and in some places where you can query based on the primary key, use efficient NoSQL database queries, which saves MySQL queries, Use NoSQL high-performance to withstand these queries.

Figure 2-nosql for mirroring (synchronous mode)

This is not through the program code, but through MySQL to synchronize the data into NoSQL, this pattern is a variant of the above, is a write transparent but with a higher technical difficulty a mode. This model is suitable for existing complex old systems, which can be difficult to implement by modifying code and may cause new problems. It also applies to the need to synchronize data to multiple types of storage.

The implementation of MySQL-to-NoSQL synchronization can be implemented using the MySQL UDF function, MySQL binlog parsing. Can be implemented using existing open source projects, such as:

• MySQL memcached UDFs: memcached protocol from Operation via UDF.
• Domestic open-source mysql-udf-http: Operate the HTTP protocol via UDF.

With these two MySQL UDF libraries, we can process the memcached or HTTP protocol transparently through MySQL, so as long as there is a NoSQL database that is compatible with memcached or HTTP protocols, we can use MySQL to operate the data synchronously. Combined with Lib_mysqludf_json, the combination of UDF and MySQL trigger function can realize automatic synchronization of data.

(ii) MySQL and NoSQL portfolio

In MySQL, only the small, small segment that needs to be queried is stored, and NoSQL stores all the data.

Figure 3-mysql and the NoSQL combination

Example pseudo code for writing data//data the data object we want to store data.title= "title"; Data.name= "Name"; Data.time= "2009-12-01 10:10:01";d ata.from= "1"; bool Status=false; Db.starttransaction ();//Start Transaction id=db. Insert (INSERT into table (Data.from)),//write to MySQL database, write only from the field where the query is required if (id>0) {     status= Nosql.add (id,data);//To write to MySQL generated by the self-increment ID primary key written to the NoSQL database} if (id>0 && status==true) {     db.commit ();//COMMIT TRANSACTION} else{     db.rollback ();//unsuccessful, rollback}

The fields that need to be queried, usually numbers, time and other types of small pieces are stored in MySQL, according to the query to establish the appropriate index, other unwanted fields, including large text fields are stored in NoSQL. In the query, we first query the data from MySQL primary key, and then directly from the NoSQL to take out the corresponding data.

This architecture model integrates MySQL and NoSQL, and allows MySQL to specialize in relational storage, NoSQL as data storage. It has the following advantages:

• Save MySQL's io overhead. Because MySQL stores only the small pieces of text that need to be queried, it is no longer responsible for storing the large field, which saves the space overhead of MySQL storage and saves MySQL's disk IO. We used this optimization to shrink a 40G table of MySQL to hundreds of M.
• Improved MySQL Query cache cache hit ratio. We know that the query cache cache invalidation is table-level, the MySQL table once updated will be invalidated, after the separation of this field, the updated field if not stored in MySQL, then the query cache has no effect. NoSQL caches are usually row-level, and only the cache of updated records is invalidated.
• Improve MySQL master-slave synchronization efficiency. Due to the decrease in MySQL storage space, synchronized data records are also reduced, and some data updates fall on NoSQL rather than MySQL, which also reduces the number of times MySQL data needs to be synchronized.
• Increase the speed of MySQL data backup and recovery. Due to the decrease of data stored in MySQL database, it is easy to see that the speed of data backup and recovery will be greatly improved.
• Easier to scale than before. NoSQL is inherently easy to expand. With this optimization, MySQL performance has also been improved.

For example, mobile phone Phoenix is this architecture http://www.cnblogs.com/sunli/archive/2010/12/20/imcp.html

Summarize

A nosql-supplemented architecture is centered on the idea of MySQL architecture, and it's just that the previous architectures have assisted in adding nosql to improve performance and scalability. This architecture is relatively easy to implement, but it can achieve good results. Hopefully this article will help you if you're trying to introduce NoSQL into your project, or if your MySQL-based system is currently having a bottleneck associated with it.

Thank Propellerhead for the review of this article.

from:http://www.infoq.com/cn/news/2011/02/nosql-architecture-practice/

NoSQL Architecture Practice (i)--A NoSQL supplement