Django Large File Download

When Django provides file downloads, if the file is small, the solution is to generate all of the content that will be delivered in memory and then pass in the response object once:

def simple_file_download (Request): # do something...content = open ("Simplefile", "RB"). Read () return HttpResponse ( Content

If the file is very large, the simplest way is to use a static file server, such as Apache or Nginx server to process the download. However, sometimes we need to limit the user's permissions, or do not want to expose the user to the real address of the file, or the large content is temporarily generated (such as the temporary merging of multiple files), then you can not use the static file server.

As mentioned in the Django documentation, you can pass an iterator to HttpResponse and stream the data to the client.

To write your own iterator, you can use yield:

def read_file (filename, buf_size=8192):    with open (filename, "RB") as F:while True:    content = F.read (buf_size) C2/>if Content:yield content    else:breakdef big_file_download (Request):    filename = "FileName"    response = HttpResponse (read_file (filename))    return response

or using a builder expression, here is an example of a large CSV file download in a Django document:

import csvfrom django.utils.six.moves import rangefrom django.http Import Streaminghttpresponseclass Echo (object): "" "an object that implements just the write method of the File-like Interfa    Ce.        "" "Def write (self, Value):" "" Write the value by returning it, instead of storing in a buffer. ""    return valuedef Some_streaming_csv_view (Request): "" "A view that streams a large CSV file." " # Generate a sequence of rows. The range is based on the maximum number of # rows, can be handled by a, sheet in most spreadsheet # appli    Cations. rows = (["Row {0}". Format (IDX), str (IDX)] for IDX in range (65536)) Pseudo_buffer = Echo () writer = Csv.writer (pseudo _buffer) Response = Streaminghttpresponse ((Writer.writerow (row) for row in rows), C Ontent_type= "Text/csv") response[' content-disposition '] = ' attachment; Filename= "Somefilename.csv" ' Return response 

Python also provides a file wrapper that wraps the class file object into an iterator:

Class FileWrapper: "" "    Wrapper to convert file-like objects to Iterables" "    def __init__ (self, filelike, blksize= 8192):        self.filelike = filelike        self.blksize = blksize        if hasattr (filelike, ' close '):            self.close = Filelike.close    def __getitem__ (self,key):        data = Self.filelike.read (self.blksize)        if data:            return Data        raise Indexerror    def __iter__ (self):        return self    def next (self):        data = Self.filelike.read (self.blksize)        if data:            return data        raise Stopiteration

When used:

From django.core.servers.basehttp import filewrapperfrom django.http import httpresponseimport osdef file_download ( Request,filename):    wrapper = filewrapper (file (' filepath '))    response = HttpResponse (wrapper, content_type= ' Application/octet-stream ')    response[' content-length '] = os.path.getsize (path)    response[' Content-disposition '] = ' attachment; filename=%s '% filename    return response

Django also provides the Streaminghttpresponse class for processing in place of httpresponse convection data.

Compress to zip file download:

Import OS, tempfile, zipfile from django.http import httpresponse from django.core.servers.basehttp import filewrapper def send_zipfile (Request): "" "Create A ZIP file on disk and transmit it in chunks of 8KB, without loading the whole file into memory.                                             A Similar approach can be used for large dynamic PDF files. "" TEMP = tempfile. Temporaryfile () archive = ZipFile. ZipFile (temp, ' w ', ZipFile.                                     zip_deflated) for index in range: filename = __file__ # Select your files here. Archive.write (filename, ' file%d.txt '% index) archive.close () wrapper = FileWrapper (temp) RESP Onse = HttpResponse (wrapper, content_type= ' Application/zip ') response[' content-disposition '] = ' attachment; Filename=test.zip ' response[' content-length '] = Temp.tell () temp.Seek (0) return response 

However, using Django to handle large file downloads is not a good idea, and the best way to do this is to have Django do the right thing, and then let the static server handle the download.

This requires the use of the Sendfile mechanism: "The traditional Web server when processing file downloads, always read the contents of the file into the application memory, and then send the contents of the memory to the client browser." This approach will consume more server resources in response to today's heavy-duty web sites. Sendfile is a high-performance network IO mode supported by modern operating system, the Sendfile call of the operating system kernel can push the contents of the file directly into buffer of the network card, thus avoiding the overhead of the Web server reading and writing files and implementing the "0 copy" mode. "

The Apache server requires the Mod_xsendfile module to be implemented, and Nginx is implemented by a feature called X-accel-redirect.

Nginx configuration file:

# would serve/var/www/files/myfile.tar.gz# when passed Uri/protected_files/myfile.tar.gzlocation/protected_files { Internal;alias/var/www/files;}

Or

# would serve/var/www/protected_files/myfile.tar.gz# when passed uri/protected_files/myfile.tar.gzlocation/protected _files {internal;root/var/www;}

Note the difference between alias and Root.

In Django:

response[' x-accel-redirect ']= '/protected_files/%s '%filename

This way, when initiating a request to the Django view function, Django is responsible for determining the user's permissions or doing something else, and then forwarding the URL to/protected_files/filename request, the Nginx server is responsible for the file/ Downloads for Var/www/protected_files/filename:

@login_requireddef Document_view (Request, document_id): Book    = Book.objects.get (id=document_id)    response = HttpResponse ()    name=book.mybook.name.split ('/') [-1]    response[' content_type ']= ' Application/octet-stream '    response[' content-disposition "] =" attachment; FILENAME={0} ". Format (            name.encode (' Utf-8 '))    response[' content-length '] = os.path.getsize ( Book.myBook.path)    response[' x-accel-redirect '] = "/protected/{0}". Format (book.myBook.name)    return Response

　　

　

Django Large File Download