Tutorial on generating a CSV file with streaming responses in Python's Django framework

In Django, streaming response streaminghttpresponse is a good thing to produce a large file quickly and in a memory-saving manner.

One of the current projects for streaming response is EventSource, which is used to improve the user-generated sense of slowness when communicating across systems. This is not a detail.

Another is to generate a large CSV file.

When the Django process is in a web container such as Gunicorn or UWSGI, if the response is not returned for a certain amount of time, it will be terminated by the Web container, although we can bypass the problem by adding a longer web container timeout, but it is still a palliative. To solve this problem fundamentally, Python's generator, the streaminghttpresponse provided by the Django Framework, is a useful streaming response.

In the CSV, the Chinese processing is also very important, to ensure that Excel open CSV does not garbled anything. To save space, I pasted all the code together. The actual use is placed according to the plan of the project ha

On the code:

From __future__ import absolute_importimport csvimport codecsimport cstringioclass Echo (object): Def write (self, value): return valueclass unicodewriter: "" A CSV writer which would write rows to CSV file "F", which are encoded in the GI  VEN encoding. "" "Def __init__ (self, F, dialect=csv.excel, encoding=" Utf-8 ", **kwds): # Redirect output to a queue Self.queue = C Stringio.stringio () Self.writer = Csv.writer (Self.queue, Dialect=dialect, **kwds) Self.stream = f Self.encoder = Codecs.getincrementalencoder (Encoding) () def writerow (self, Row): Self.writer.writerow ([Handle_column (s) for s in row] # Fetch UTF-8 output from the queue ... data = Self.queue.getvalue () data = Data.decode ("Utf-8") # ... and R Eencode it into the target encoding data = Self.encoder.encode (data) # write to the target stream value = Self.st Ream.write (data) # empty queue self.queue.truncate (0) return value def writerows (self, rows): For row in rows : SelF.writerow (Row) 

From django.views.generic import viewfrom django.http.response import Streaminghttpresponseclass Exampleview (View):  headers=[' some ', ' table header ']  def get (self,request):    result = [[' First line ', ' Data 1 '],         [' Second row ', ' Data 2 ']]    echoer = Echo ()    writer = Unicodewriter (echoer)    def csv_itertor ():        yield codecs. Bom_utf8        yield Writer.writerow (self.headers) for        column in result:          yield writer.writerow (column)    Response = Streaminghttpresponse (      row for row in Csv_itertor ()),      content_type= "Text/csv;charset=utf-8")    response[' content-disposition '         = ' attachment;filename= ' example.csv '    return response

