Mosonic: attempts to improve the distributed storage and cache of subsonic (2)

Link.

Cache money is actually used in vector cache. In the production environment, it not only has a higher hit rate than the object cache, but also brings a significant performance leap.

In the performance test of mosonic, the performance has been improved by 10 times.

The performance of vector cache is terrible, but it has strict requirements on the table structure and query type. The following lists:

• The table must use an auto-increment number (INT/long) ID as the primary key.
• The where clause in the query must be equal to or equal to the condition, for example, where user_id = 1.
• If multiple where conditions exist, the relationship must be and, for example, where user_id = 1 and id_deleted = 0.
• The query result can only be a data ID, for example, select ID from users where... it cannot be select user_name from users where...
  • It can also be select count (*) from users where...
  • Pagination is supported for query results
• The query result must be in the descending order of ID, that is, order by id desc.

Vector cache takes effect only when the preceding five conditions are fully met. Fortunately, this type of structure/query is the most common in Web 2.0 websites.

Taking a blog as an example, the list of blog articles, number of classified articles, comment display, and so on are basically consistent with the above query.

For example, to obtain a user with a level of 1, you need to use the following two queries:

 

• Select ID from users where level = 1
• Select * from users where ID in (....)

 

Both query cache money can be fully cached:

 

• Select * from users where level = 1

 

Then, the cache money will be completely invalid.

For queries of the two styles, refer to the statement written by Robin, the javaeye Boss: Why is the ORM performance better than ibatis?

==================

Because the query results must be IDs and in reverse order, vector cache can be automatically updated in real time instead of automatically expired.

Consider the following call:

 

1. Select count (ID) from photos where album_id = 1 order by id desc limit 1,100
2. Select ID from photos where album_id = 1 order by id desc limit 1,100
3. Insert into Photos (album_id) values (1)
4. Select count (ID) from photos where album_id = 1 order by id desc limit 1,100
5. Select ID from photos where album_id = 1 order by id desc limit 1,100

 

Displays the list, inserts data, and displays the list again. This is a typical call.

The query in step 1/2 will be cached (even if there is no cache, the cache will be automatically generated after the query, that is, the so-called direct read ).

After you obtain the auto-increment ID of the database, you can directly append the ID to the cache result of step 3rd.

In step 4/5, the query directly hits the cache updated when data is written in step 3rd. You do not need to query the database at all.

When the query and application scenarios are in line with the ideal situation, with the vector cache, database reads can become terrible.0 read.

The database only needs to bear the write pressure, and 100% of reads are automatically cached by memcache.

This is why the vector cache of cache money has brought about a leap in read performance.

All database queries are converted into memcache get; memcache's single-host Reading Capability and concurrent load capability are both one order of magnitude higher than traditional relational databases; and its shared nothing architecture, it can also expand horizontally.

In the case of high concurrency and multi-machine cache, it is expected that the read performance of cache money will be improved by more than 10 times.

====================

Twitter engineers are clever in implementing cache money, and they have achieved 100% read cache for a scenario with many restrictions; this "too many restrictions" is exactly the most typical scenario in Web 2.0 websites.

When I implemented vector cache in mosonic, I completely copied the implementation algorithm of cache money; that is, C #'s code volume is several times larger than Ruby's.

Next, we will continue to introduce mosonic's reference to the design of the friendfeed distributed database.