How to generate a CSV file using a stream response in the Python Django framework

This article mainly introduces how to generate CSV files using stream responses in the Python Django framework. The author particularly talked about preventing Chinese Characters in CSV files from garbled characters, for more information about StreamingHttpResponse, see StreamingHttpResponse in Django. It can generate a large file in a fast and memory-saving manner.

Currently, Eventsource is used for stream response, which is used to improve the user's low speed during cross-system communication. This is not detailed.

Another is to generate a large csv file.

When the Django process is in a web Container such as gunicorn or uwsgi, if no response is returned after a certain period of time, it will be terminated by the web container, although we can extend the timeout time of web containers to bypass this problem, it is still a permanent cure. To fundamentally solve this problem, the StreamingHttpResponse stream response provided by the Python generator and Django framework is very helpful.

In csv, the processing of Chinese characters is also crucial. Make sure that the csv files in excel are not garbled .. To save space, I Will paste all the Code together .. Actual use is placed according to the project plan.

Code:

from __future__ import absolute_importimport csvimport codecsimport cStringIOclass Echo(object):  def write(self, value):    return valueclass UnicodeWriter:  """  A CSV writer which will write rows to CSV file "f",  which is encoded in the given encoding.  """  def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):    # Redirect output to a queue    self.queue = cStringIO.StringIO()    self.writer = csv.writer(self.queue, dialect=dialect, **kwds)    self.stream = f    self.encoder = codecs.getincrementalencoder(encoding)()  def writerow(self, row):    self.writer.writerow([handle_column(s) for s in row])    # Fetch UTF-8 output from the queue ...    data = self.queue.getvalue()    data = data.decode("utf-8")    # ... and reencode it into the target encoding    data = self.encoder.encode(data)    # write to the target stream    value = self.stream.write(data)    # empty queue    self.queue.truncate(0)    return value  def writerows(self, rows):    for row in rows:      self.writerow(row)

From django. views. generic import Viewfrom django. http. response import StreamingHttpResponseclass ExampleView (View): headers = ['date', 'header'] def get (self, request): result = [['first row ', 'Data 1'], ['second row', 'Data 2'] echoer = Echo () writer = UnicodeWriter (echoer) def csv_itertor (): yield codecs. BOM_UTF8 yield writer. writerow (self. headers) for column in result: yield writer. writerow (column) response = StreamingHttpResponse (row for row in csv_itertor (), content_type = "text/csv; charset = UTF-8 ") response ['content-disposition'] = 'attachment; filename = "example.csv" 'Return response