Django adds last-modified and ETag

The rest API, which is made with the Django Rest framework, has an API that requires:

The app side requests this API, the server side reads the data from the database, returns the JSON. The amount of data returned is slightly larger, but may only be modified once a year, so you want to be able to transfer data only when the data is modified, so that the app side updates.

1. Last-modified and ETag

The HTTP response header last-modified and ETag both implement this requirement, and the detailed explanation for both is simple and straightforward: http://www.iwms.net/n2029c12.aspx

The original excerpt is as follows:

1) What is "last-modified"?
When the browser requests a URL for the first time, the server-side return status is 200, the content is the resource you requested, and there is a last-modified attribute that marks the last time the file was modified at the end of the service period, similar in format:

　　Last-modified:fri, 2006 18:53:33 GMT
When the client requests this URL for the second time, according to the HTTP protocol, the browser transmits the If-modified-since header to the server, asking if the file has been modified after that time:

If-modified-since:fri, 2006 18:53:33 GMT
If the server-side resource does not change, the HTTP 304 (not Changed.) Status code is automatically returned, and the content is empty, saving the amount of data transferred 。 When the server-side code changes or restarts the server, the resource is re-emitted, similar to when the first request is returned. This ensures that the resources are not duplicated to the client, and that the client is able to get the latest resources when the server changes.
2) What is an "Etag"? The
HTTP protocol specification defines the etag as the entity value of the requested variable (see-section 14.19). Another argument is that the ETag is a token that can be associated with a Web resource. A typical web resource can be a Web page, but it may also be a JSON or XML document.　　The server is solely responsible for judging what the token is and what it means, and transmitting it to the client in the HTTP response header, the following is the format returned by the server side: The
ETag: 50b1c1d4f775c61:df3
Client's Query update format:
if-none-match:w/"50B1C1D4F775C61:DF3"
if the etag does not change, it returns the status 304 and does not return, which is the same as last-modified. I test the etag is mainly in the breakpoint download is more useful.
3) How can last-modified and etags help improve performance? The
Smart Developer will use the HTTP header of the last-modified and ETAGS requests to take advantage of the cache of the client (such as the browser). Because the server first generates the LAST-MODIFIED/ETAG tag, the server can later use it to determine if the page has been modified. Essentially, the client requires the server to validate its (client) cache by passing the token back to the server. The
procedure is as follows:

1. The client requests a page (a).
2. The server returns page A and adds a last-modified/etag to a.
3. The client presents the page and caches the page along with Last-modified/etag.
4. The customer requests page A again and passes the Last-modified/etag that the server returned when the last request was sent to the server.
5. The server checks the last-modified or ETag and determines that the page has not been modified since the last client request, directly returning the response 304 and an empty response body.

2. Add last-modified and ETag in Django

There are three decorator in Django that can add last-modified and etag to the response, see: https://docs.djangoproject.com/en/1.8/topics/conditional-view-processing/

• Condition(etag_func=none, last_modified_func=none)
• ETag(etag_func)
• Last_modified (Last_modified_func)

Note that the use of the official website is for function-based view, if you are using class-based view, you need to call decorator in the URL configuration, for example:

URL (r'^rest/colours/$',    etag (get_colour_checksum) (views. Colourlist.as_view ()),    name='colour-list'),

To achieve the previous requirements, the idea is simple, get the data table last modified time or the data table to calculate a hash value on the line.

At first, I found a MySQL. How to get the table last updated time: http://stackoverflow.com/questions/307438/how-can-i-tell-when-a-mysql-table-was-last-updated

SELECT Update_time  from    information_schema.tables WHERE  = ' dbname '    and = ' tabname '

Try it on your own database, but return is null. A careful look at the comment, this method is useful on MyISAM, but it is useless on InnoDB. Had to find another way.

Later, I found a statement to view table checksum: http://dev.mysql.com/doc/refman/5.5/en/checksum-table.html

TABLE [, Tbl_name] []

Test useful! This is used as a method of generating an etag.

3. Adding last-modified and ETag via the cache framework

Because I used the Django Cache framework (https://docs.djangoproject.com/en/1.8/topics/cache/), the cache time for the request was set to 5 minutes. So there are three response headers that are automatically added as I set the 5-minute cache time:

• Cache-control:max-age=5m
• last-modified:2015 SEP 27 11:38:00
• expires:2015 SEP 27 11:43:00

But after 5 minutes, the cache will be refreshed and the last-modified time will be refreshed accordingly, which is not enough to meet the requirements I mentioned earlier. So how do we add the ETag?

By reading the Updatecachemiddleware source code, I found it in the processing response call function Patch_response_headers, the function of the source code is as follows:

defPatch_response_headers (Response, cache_timeout=None):"""Adds Some useful headers to the given HttpResponse Object:etag, last-modified, Expires and Cache-control    Each header was only added if it isn ' t already set. Cache_timeout is in seconds.    The Cache_middleware_seconds setting is used by default. """    ifCache_timeout isNone:cache_timeout=settings. Cache_middleware_secondsifCache_timeout <0:cache_timeout= 0#Can ' t have max-age negative    ifSettings. Use_etags and  notResponse.has_header ('ETag'):        ifHasattr (Response,'Render') andcallable (Response.render): Response.add_post_render_callback (_set_response_etag)Else: Response=_set_response_etag (response)if  notResponse.has_header ('last-modified'): response['last-modified'] =http_date ()if  notResponse.has_header ('Expires'): response['Expires'] = Http_date (time.time () +cache_timeout) Patch_cache_control (response, Max_age=cache_timeout)

As you can see from the code:

• If last-modified, expires is not set before, then this is assigned to them according to the settings of the cache.
• If settings. Use_etags is true, and the ETag is not set, the default method or custom method will be called to assign the ETag value.

Because the default method _set_response_etag is to use the response content to calculate the MD5 value assigned to the ETag, as long as the data is not changed, the ETag value is constant, can meet the requirements.

So what needs to be done is to add use_etag=truein the settings, it's OK.

4. Nginx Support for ETag

With the previous setup, I have already implemented the ETag additions on the development machine, and I can see that the etag is gone when I deploy the code to the production environment to test again. The production environment is Gunicorn+nginx, so it is natural to think that there is a problem with nginx.

Most of the web-based Nginx ETag related articles are saying how to add the ETag for static files, and my situation is not the same.

Spent a long time, finally found the reason:nginx 1.7.3 version of the ETag support is not good, open gzip will automatically delete the ETag . See: http://stackoverflow.com/questions/31125888/nginx-missing-etag-when-gzip-is-used

The solution is simple, upgrade Nginx to the latest version of the line, even the configuration is not modified. Upgrade method See: Http://nginx.org/en/linux_packages.html

Django adds last-modified and ETag