Nginx-upload-module implements resumable File Transfer

Nginx-upload-module implements resumable File Transfer
GuideEvery time we want to simply implement the file upload function without using other languages (such as PHP and Java), or to implement resumable upload of files. At this time, a module of Nginx, nginx-upload-module, can meet our needs.Module Installation

Download module:

cd /tmpwget https://codeload.github.com/vkholodkov/nginx-upload-module/zip/2.2unzip 2.2

Installation module:

.configure --add-module=/tmp/nginx-upload-module-2.2/

Multipart/form-data form upload example

Nginx. conf configuration:

server {  [...]         location /upload {                upload_pass @uploadHandler;                upload_store /usr/local/nginx/upload_temp 1;                upload_set_form_field $upload_field_name.path "$upload_tmp_path";        }              location @uploadHandler {                proxy_pass http://backend-host;        }    [...]    }

The upload location is defined in the server. This location is the upload interface and @ uploadHandler location. After the file is uploaded, The nginx module sends necessary information to this location, for example, the file upload path involves several commands:

Upload_pass @ uploadHandler: After the upload is complete, necessary data will be sent to @ uploadHandler;
Upload_store/usr/local/nginx/upload_temp 1: temporary directory for file upload;
Upload_set_form_field $ upload_field_name.path "$ upload_tmp_path": After the file is uploaded, the temporary file path is sent to the location specified by upload_pass.

Resumable upload example

Nginx. conf configuration

server {[...]        location /resumable_upload {               upload_resumable on;               upload_state_store /usr/local/nginx/upload_temp ;               upload_pass @drivers_upload_handler;               upload_store /usr/local/nginx/upload_temp;               upload_set_form_field $upload_field_name.path "$upload_tmp_path";            }              location @resumable_upload_handler {               proxy_pass http://localhost:8002;        }    [...]    }

Different from the previous multipart/form-data form upload example Configuration:
Upload_resumable on: enables resumable upload;
Upload_state_store/usr/local/nginx/upload_temp: Specifies the directory for storing resumable upload files.

Upload the first part of the file

POST/upload HTTP/1.1 Host: example. comContent-Length: 51201Content-Type: application/octet-streamContent-Disposition: attachment; filename = "big. TXT "X-Content-Range: bytes 0-51200/511920 Session-ID: 1111215056 <0-51200 byte file data>

Server Response for the first part of the uploaded file

HTTP/1.1 201 CreatedDate: Thu, 02 Sep 2010 12:54:40 GMTContent-Length: 14Connection: closeRange: 0-51200/511920     0-51200/511920

Upload the last part of the file

POST/upload HTTP/1.1 Host: example. comContent-Length: 201711content-type: application/octet-streamContent-Disposition: attachment; filename = "big. TXT "X-Content-Range: bytes 460809-511919/511920 Session-ID: 1111215056 <460809-425919 bytes file data>

Server Response to the last part of the uploaded file

HTTP/1.1 200 OKDate: Thu, 02 Sep 2010 12:54:43 GMTContent-Type: text/htmlConnection: closeContent-Length: 2270 <response content>

Request Header description

Request Header description Content-Disposition attachment, filename = "uploaded file name" Content-Type mime type of the file to be uploaded, such as application/octet-stream (note: cannot be multipart/form-data) X-Content-Range bytes of the file to be uploaded, for example, the first segment bytes 0-51200/511920, And the last segment bytes 460809-511919/511920 (note: the first byte Number of the file is 0, and the last byte number is n-1, where n is the file byte size.) X-Session-ID indicates the ID of the file to be uploaded, Which is randomly specified by the client. for resumable upload, the client must ensure that all parts of the same file are uploaded with the same Content-Length identifier.

Python upload demo

#! /Usr/bin/python #-*-coding: UTF-8-*-import OS. pathimport requestsimport hashlib # file to be uploaded path FILE_UPLOAD = "/tmp/testfile" # upload interface address UPLOAD_URL = "http: // host/drivers_upload "# number of bytes uploaded by a single segment SEGMENT_SIZE = 1048576 def upload (fp, file_pos, size, file_size): session_id = get_session_id () fp. seek (file_pos) payload = fp. read (size) content_range = "bytes {file_pos}-{pos_end}/{file_size }". format (file_pos = file_pos, pos_end = file_pos + size-1, file_size = file_size) headers = {'content-disposition': 'attachment; filename = "big. TXT "', 'content-type': 'application/octet-stream', 'x-Content-range': content_range, 'session-id': session_id, 'content-length': size} res = requests. post (UPLOAD_URL, data = payload, headers = headers) print (res. text) # obtain the session iddef get_session_id (): m = hashlib Based on the file name hash. md5 () file_name = OS. path. basename (FILE_UPLOAD) m. update (file_name) return m. hexdigest () def main (): file_pos = 0 file_size = OS. path. getsize (FILE_UPLOAD) fp = open (FILE_UPLOAD, "r") while True: if file_pos + SEGMENT_SIZE> = file_size: upload (fp, file_pos, file_size-file_pos, file_size) fp. close () break else: upload (fp, file_pos, SEGMENT_SIZE, file_size) file_pos = file_pos + SEGMENT_SIZE if _ name _ = "_ main _": main ()

From: https://www.centos.bz/2015/09/nginx-upload-module-multipart-form-data-resumable/

Address: http://www.linuxprobe.com/nginx-upload-module.html