A variety of cache detailed _ruby topics commonly used in Web applications

Source: Internet
Author: User
Tags comments curl redis

This article takes Nginx,rails,mysql,redis as an example, switching to other Web servers, languages, databases, caching services are similar.
Below is a 3-storey schematic to facilitate subsequent references:

1. Client-Side Caching

A client often accesses the same resource, such as using a browser to access the home page of a Web site or viewing the same article, or accessing the same API using the app, and if that resource and any changes that he has previously visited, you can take advantage of the 304 not Modified response header in the HTTP specification (http:// www.w3.org/Protocols/rfc2616/rfc2616-sec10.html), using the client's cache directly, without having to generate the content again on the server side.
With the Fresh_when method built into rails, one line of code can be done:

Class Articlescontroller
 def show
  @article = Article.find (Params[:id])
  fresh_when:last_modified => @ ARTICLE.UPDATED_AT.UTC,: ETag => @article
 End

The next time the user visits, will be compared to the request header inside the if-modified-since and If-none-match, if the match, the direct return of 304, and no longer generate response body.

But this will encounter a problem, suppose our website navigation has the user information, a user has not landed the topic to visit for a while, then the landing later visits, will discover the page to appear still has not landed the state. Or in the app to visit an article, do a collection, the next time to enter this article, or show the status of the collection. The solution to this problem is simple, and the user-related variables are added to the ETag calculation:

  Fresh_when:etag => [@article. Cache_key, Current_user.id]
  fresh_when:etag => [@article. Cache_key, Current_ User_favorited]

In addition to a pit, if nginx turns on gzip and compresses the results of rails execution, the etag header of the rails output is eliminated, and Nginx's developers say that in accordance with the RFC specification, the PROXY_PASS approach must be handled in this way (since the content changes), But I personally think that is not necessary, so in a rude way, directly src/http/modules/ngx_http_gzip_filter_module.c this line of code in this document to comment out, and then recompile nginx:

  

Or you can choose not to change the Nginx source code, the gzip off, will be compressed using rack middleware to deal with:

  Config.middleware.use Rack::D eflater

In addition to specifying Fresh_when in controller, the rails framework uses Rack::etag middleware by default, which automatically adds ETag to response, but compared to ETag, Automatic ETag can save only the client time, server side or the same will execute all the code, with curl to compare.
Rack::etag automatically joins ETag:

Curl-v HTTP://LOCALHOST:3000/ARTICLES/1
< Etag: "bf328447bcb2b8706193a50962035619"
< x-runtime: 0.286958
curl-v http://localhost:3000/articles/1--header ' if-none-match: ' bf328447bcb2b8706193a50962035619 '
< x-runtime:0.293798
with Fresh_when:

 
curl-v http://localhost:3000/articles/1--header ' If-none-match: "bf328447bcb2b8706193a50962035619" '
< x-runtime:0.033884

2. Nginx Cache

Some resources may be invoked a lot, unrelated to the user state, and rarely changed, such as the list API on the news app, the AJAX Request category menu on the shopping site, you can consider using Nginx to do caching.
There are 2 main methods of implementation:
A. Dynamic request static file
After the rails request is completed, the result is saved as a static file, and subsequent requests are directly provided by Nginx with static file content, implemented with After_filter:

Class Categoriescontroller < Actioncontroller::base
 After_filter:generate_static_file,: Only => [: index]

 def index
  @categories = Category.all
 end

 def generate_static_file
  File.Open (Rails.root.join (' Public ', ' categories '), ' W ' "Do |f|
   F.write response.body end end

In addition, we need to delete this file in any category update to avoid the problem that the cache does not refresh:

Class Category < activerecord::base
 after_save:d elete_static_file
 After_destroy:d elete_static_file

 def Delete_static_file
  File.delete Rails.root.join (' Public ', ' categories ')
 end

Before Rails 4, the process of generating static file caching could be done with the built-in caches_page, Rails 4, and then a standalone gem actionpack-page_caching, compared to the manual code,

Class Categoriescontroller < Actioncontroller::base
 caches_page:index

 def update
  #
  ... Expire_page action: ' Index '
 end


If there is only one server, this method is simple and practical, but if there are multiple servers, there will be update classification can only refresh itself this server cache problem, you can use NFS to share static resource directory resolution, or in the 2nd kind:

B. Static to the centralized caching service
First we have to give nginx the ability to access the cache directly:

 Upstream Redis {
  server redis_server_ip:6379;
 }

 Upstream Ruby_backend {
  server unicorn_server_ip1 fail_timeout=0;
  Server unicorn_server_ip2 fail_timeout=0;
 }

 location/categories {
  set $redis _key $uri;
  Default_type  text/html;
  Redis_pass Redis;
  Error_page 404 = @httpapp;
 }

 Location @httpapp {
  proxy_pass http://ruby_backend;
 }

Nginx will first go to Redis with the requested URI as key, if not (404) to the unicorn for processing, and then rewrite the Generate_static_file and Delete_static_file methods:

 Redis_cache.set (' categories ', response.body)
 Redis_cache.del (' categories ')

In addition to centralized management, you can set the expiration time of the cache, for some of the updated data without timeliness requirements, you can not handle the refresh mechanism, simple fixed time to refresh once:

 Redis_cache.setex (' categories ', 3.hours.to_i, Response.body)

3. Full Page Caching

Nginx caching is very difficult to handle when processing a request with a parameter resource or a user's state, which can be used for full-page caching.
For example, the paging request list, we can add the page parameter to the Cache_path:

Class Categoriescontroller
 Caches_action:index,: expires_in => 1.day, Cache_path => proc {"Categories/index /#{params[:p age].to_i} "} End

For example, we only need to cache the RSS output for 8 hours:

Class Articlescontroller
 Caches_action:index,: expires_in => 8.hours,: If => proc {request.format.rss?}
End

Another example is that for non-landing users, we can cache the home page:

Class HomeController
 Caches_action:index,: expires_in => 3.hours,: If => proc {!user_signed_in?}
End

4. Fragment Caching

Fragment caching is the most adaptable if there are only a limited number of scenarios that the previous 2 caches can use.

Scenario 1: We need an ad code on each page to display different ads, and if you don't use fragment caching, each page will query the code for the ad and take time to generate the HTML code:

-If advert = Advert.where (: Name => request.controller_name + request.action_name,: Enable => true).
 D
  = advert.content

After adding the fragment cache, you can less go to this query:

-Cache "Adverts/#{request.controller_name}/#{request.action_name}",: expires_in => 1.day do
 -if advert = advert . WHERE (: Name => request.controller_name + request.action_name,: Enable => true). A-
  div.ad
   = Advert.content

Scenario 2: Read the article, the content of the article may not change for a long time, often changes may be the article comments, you can add fragment cache to the main part of the article:

-Cache "articles/#{@article. id}/#{@article. updated_at.to_i}" do
 div.article
  = @ Article.content.markdown2html

Save the generation of Markdown syntax conversion to HTML time, here with the last update time as a cache key, the content of the article if there is change, the cache automatically expires, the default ActiveRecord Cache_key method is also used Updated_at, You can also add more parameters, such as article on the number of comments on the counter cache, update the number of comments will not update the article time, you can add this counter also to a part of the key

Scenario 3: Building a complex page structure
Data structure more complex pages, in the generation of time to avoid a large number of queries and HTML rendering, with fragment caching, you can save this part of the time greatly to our website Travel page http://chanyouji.com/trips/109123 (please allow a small ad, With point traffic):
Need to get weather data, photo data, text data and so on, but also to generate Meta,keyword and other SEO data, and these content and other dynamic content crossover, fragment caching can be separated by multiple:

-Cache "trips/show/seo/#{@trip. Fragment_cache_key}",: expires_in => 1.day do
 title #{trip_name @trip}
 Meta Name= "description" content= "
 meta name=" keywords "content=" ... "Body
 div
  ...
-Cache "trips/show/viewer/#{@trip. Fragment_cache_key}",: expires_in => 1.day do
 -@trip. Eager_load_all

Tips, I added a Eager_load_all method in the Trip object, the cache did not hit the time, the query to avoid n+1 problems:

 def eager_load_all
  activerecord::associations::P reloader.new ([self], {: Trip_days => [: Weather_station_data, : Nodes => [: Entry,: Notes => [:p hoto,: Video,: Audio]]]. Run End
 

Tip 1: Fragment Caching with conditions
Unlike Caches_action, rails has its own fragment cache that does not support the condition, such as we want to not log in to the user to use fragment caching, and landing users do not use, write a very cumbersome, we can rewrite the helper on it:

 def cache_if (condition, name = {}, cache_options = {}, &block)
  if condition
   cache (name, cache_options, &b Lock)
  Else
   yield
  end

-cache_if!user_signed_in?, ' xxx ',: expires_in => 1.day do

Tip 2: Automatic Updates of associated objects
Often use object Update_at time stamp as cache key, you can add touch options on the associated object, automatically update the associated object timestamp, such as we can update or delete article comments, Automatic Updates:

Class Article
 has_many:comments
End

class Comment
 belongs_to:article.: Touch => True
End

5. Data query Caching

Generally speaking, Web application performance bottlenecks appear on DB io, do well in data query caching, reduce database query times, can greatly improve the overall response time.
The data query cache is divided into 2 types:
A. Caching within the same request cycle
Give an example that shows a list of articles, output the title of the article and the category of the article, the corresponding code is as follows

# Controller
 def index
  @articles = Article.first end

# view
-@articles.
 H1 = Article.name
 span = Article.category.name

10 Similar SQL queries will occur:

SELECT ' categories '. * from ' categories ' WHERE ' categories '. ' id ' =?

Rails has built-in query cache (https://github.com/rails/rails/blob/master/activerecord/lib/active_record/connection_ ADAPTERS/ABSTRACT/QUERY_CACHE.RB), in the same request cycle, if there is no update/delete/insert operation, the same SQL query will be cached, if the article categories are the same, There will only be 1 times to actually query the database.

If the article categories are different, there will be n+1 query problems (common performance bottlenecks), and rails recommends a workaround using eager Loading associations (http://guides.rubyonrails.org/active_ record_querying.html)

 def index
  @articles = article.includes (: category).
 

The query statement becomes

SELECT ' categories '. * from ' categories ' WHERE ' categories '. ' ID ' in (?,?,?...)

B. Caching across request cycles

Performance optimizations with request-cycle caching are limited, and many times we need to cache some commonly used data (such as user model) by caching across request cycles, using a unified query interface to fetch cache for active record. Using callback to expire cache is easy to implement, and there are some ready-made gems to use.

For example Identity_cache (Https://github.com/Shopify/identity_cache)

Class User < ActiveRecord::Base
 include Identitycache
end

class Article < ActiveRecord::Base
 Include Identitycache
 cached_belongs_to:user
End

# will hit cache

User.fetch (1)
Article.find (2) . user

The advantage of this gem is that code implementation is simple, cache settings are flexible, and easy to expand, the disadvantage is the need to use different Query method name (fetch), as well as additional relationship definitions.

If you want to seamlessly add caching functionality to an application without data caching, recommend @hooopo to do Second_level_cache (Https://github.com/hooopo/second_level_cache).

Class User < ActiveRecord::Base
 acts_as_cached (: Version => 1,: expires_in => 1.week)
End

#还是使用find方法, the cache is hit

User.find (1)
#无需额外用不一样的belongs_to定义
Article.find (2). User
The implementation principle extends the active record bottom Arel SQL AST processing (https://github.com/hooopo/second_level_cache/blob/master/lib/second_level_ CACHE/AREL/WHERES.RB)
Its advantage is seamless access, the disadvantage is that the expansion is more difficult, for only a small number of fields can not be cached query.

6. Database caching

In edit

These 6 kinds of caching, distributed in the client to the server side of the different locations, the time can be saved just from more to less in order.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.