[Favorites] net large File Upload knowledge sorting

. Net large File Upload knowledge

Recently, I am working on the epartner project, which involves file upload. I have uploaded files before, but they are all small files and cannot exceed 2 m . This request is uploaded 100 m The above. I couldn't find any information to study it. For web-based file uploads, FTP and HTTP protocols can be used. Although FTP is stable in transmission, security is a serious problem. In addition, the FTP server reads the user database to obtain permissions, this is not convenient for users. Only HTTP is left. There are three methods in http: Put, WebDAV, and rfc1867. The first two methods are not suitable for uploading large files, currently, we use form-based file uploads Based on rfc1867 standard HTML. I. Briefly introduce rfc1867 (form-based file upload in HTML) standards: 1. HTML form with file submission Function The existing HTML Specification defines eight possible values for the Type attribute of the input element: checkbox, hidden, image, password, radio, reset, submit, text. in addition, when the form uses the POST method, the form has the "Application/X-WWW-form-urlencoded" enctype attribute by default.   Rfc1867 made two changes to HTML: 1) added a file option for the Type attribute of the input element. 2) The input tag can have the accept attribute, which can specify the file type or file format list that can be uploaded.   In addition, this standard defines a new MIME type: multipart/form-data, and the action that should be taken when processing a form with enctype = "multipart/form-Data" and/or a tag containing <input type = "file">.   For example, when the author of an HTML form wants the user to upload one or more files, he can write as follows:   <Form enctype = "multipart/form-Data" Action = "_ URL _" method = post> File to process: <Input name = "userfile1" type = "file"> <Input type = "Submit" value = "Send File"> </Form>   The change to the html dtd is to add an option for the inputtype object. In addition, we recommend that you use a series of file types separated by commas as the accept attribute of the input tag.   ... (Other elements )...   <! Entity % inputtype "(Text | password | checkbox | Radio | submit | reset | Image | hidden | file) "> <! Element input-0 empty> <! ATTLIST Input Type % inputtype text Name CDATA # implied -- required for all but submit and reset Value CDATA # implied SRC % URI # implied -- for image inputs -- Checked (checked) # implied Size CDATA # implied -- like numbers, But delimited with comma, not space Maxlength number # implied Align (top | Middle | bottom) # implied Accept CDATA # implied -- List of content types >   ... (Other elements )...   2. file transmission delay In some cases, it is recommended that the server verify certain elements (such as user names and accounts) in the form data before preparing to accept the data. However, after some consideration, we think that if the server wants to do this, it is best to use a series of forms, the data elements that have been verified are returned to the client as "hidden" fields, or the elements that need to be verified are displayed first by arranging a form. In this way, servers that require complex applications can maintain the state of transaction processing on their own, while those simple applications can be implemented simply.   The HTTP protocol may need to know the total length of the content in the entire transaction. Even if there is no clear requirement, the HTTP client should provide the total length of all uploaded files, so that a busy server can determine whether the file content is too large to be completely processed, returns an error.CodeAnd close the connection, instead of waiting until all the data is accepted for judgment. Currently, some existing CGI applications need to know the total length of the content for all post transactions.   If the input tag contains a maxlength attribute, the client can regard this attribute value as the maximum number of bytes of the transferred file that the server can accept. In this case, the server prompts the client how much space it has on the server before the upload starts. However, it should be noted that this is just a prompt that the actual requirements of the server may change after the form is created and before the file is uploaded.   In any case, if the received file is too large, Any HTTP server may interrupt the transmission during file transmission.   3. Other solutions for binary data transmission Some people have suggested using a new MIME type "aggregate", such as aggregate/mixed or content-transfer-encoding "package" to describe binary data with uncertain length, instead of being expressed by dividing them into multiple parts. Although we do not oppose this, it requires additional design and standardization work to make everyone accept and understand "aggregate ". On the other hand, the "split into multiple parts" mechanism works well and can be implemented easily on the client sender and server receiver, and can work as efficiently as some other methods to comprehensively process binary data.   4. Example Assume that the server segment provides the following HTML: <Form action = "http://server.dom/cgi/handle" Enctype = "multipart/form-Data" Method = post> What is your name? <Input type = text name = submitter> What files are you sending? <Input type = file name = pics> </Form> In the "name" field, enter "Joe Blow". What files are you sending? ', Select A text file "file1.txt ". The customer segment may send back the following data: Content-Type: multipart/form-data, boundary = aab03x   -- Aab03x Content-Disposition: Form-data; name = "field1"   Joe Blow -- Aab03x Content-Disposition: Form-data; name = "pics"; filename = "file1.txt" Content-Type: text/plain   Contents of... file1.txt... -- Aab03x -- If you select another image file "file2.gif", the client may send the following data: Content-Type: multipart/form-data, boundary = aab03x   -- Aab03x Content-Disposition: Form-data; name = "field1"   Joe Blow -- Aab03x Content-Disposition: Form-data; name = "pics" Content-Type: multipart/mixed, boundary = bbc04y   -- Bbc04y Content-Disposition: attachment; filename = "file1.txt"   Content-Type: text/plain   Contents of... file1.txt... -- Bbc04y Content-Disposition: attachment; filename = "file2.gif" Content-Type: image/GIF Content-transfer-encoding: Binary   ... File2.gif content... -- Bbc04y -- -- Aab03x -- Ii. Two methods for processing file uploads using rfc1867: 1. Get the uploaded data at one time and analyze and process it. After reading n multiple codes, I found that there are currently no components.ProgramAnd some COM components use the request. binaryread method. Obtain the uploaded data at a time, and then analyze and process it. This is why the upload of large files is slow. If IIS times out, even if hundreds of MB of files are uploaded, the analysis takes a while. 2. Write to the hard disk while receiving files.   I have learned about commercial components outside China, including power-web, aspupload, activefile, abcupload, aspsmartupload, and Sa-fileup. Among them, the better is aspupload and SA-FILE, they claim to be able to handle 2G files (SA-FILE EE Edition does not even have file size restrictions), and the efficiency is also great, is the efficiency of programming language so much less efficient? I checked some information and thought they all operated the composition component stream directly. In this way, the file size is not restricted. However, it is not absolutely perfect for foreigners. After aspupload processes large files, the memory usage is astonishing. 1g the left and right are common. As for SA-FILE although it is good but difficult to find the crack. Then we found two. Net Upload components, lion. Web. uploadmodule and aspnetupload, which are also operation file streams. However, the upload speed and CPU usage are not as good as the commercial components of foreigners. A test was conducted to upload data in LAN. 1g file. Average aspupload upload speed is 4.4 m /s, CPU usage: 10-15, memory usage 700 m . SA-FILE is almost like this. And aspnetupload is the fastest 1.5 m /s. The average value is 700 K/s, And the CPU usage is 15-39. test environment: piII 800,256 m memory, 100 m LAN. I think the slow speed of aspnetupload may be caused by hard disk writing while receiving files. The low cost of resource occupation is to reduce the transmission speed. However, I have to admire the foreign program. The CPU usage is so low... Iii. ASP. NET File Upload Problems We have encountered one or more problems when uploading large files using ASP. NET. Setting a large value of maxrequestlength does not completely solve the problem, because ASP. net blocks the entire file until it is loaded into the memory and then processes it. In fact, if the file is large, we often see that Internet Explorer displays "the page cannot be displayed-cannot find server or DNS error". It seems that this error cannot be caught. Why? Because this is a client side error, application_error on the server side cannot be handled. Iv. ASP. NET large file upload Solution The solution is to use the implicit httpworkerrequest and its getpreloadedentitybody and readentitybody methods to read data from the pipe created by IIS for ASP. NET in blocks. Chris Hynes provides us with such a solution (using httpmodule) that allows you to upload large files and display the upload progress in real time. Lion. Web. uploadmodule and aspnetupload both use this solution.   Solution Principle: Httphandler is used to implement functions similar to ISAPI extention, process request information and send response ).   Solution highlights: 1. httphandler or httpmodule A. The request object is intercepted before the Asp.net process processes the request. B. Read and Write Data in Parts C. Track the upload progress in real time and update the meta information. 2. Use the implicit httpworkerrequest to process the file stream using its getpreloadedentitybody and readentitybody methods. Iserviceprovider provider = (iserviceprovider) httpcontext. Current; Httpworkerrequest wR = (httpworkerrequest) provider. getservice (typeof (httpworkerrequest )); Byte [] BS = Wr. getpreloadedentitybody (); .... If (! Wr. isentireentitybodyispreloaded ()) { Int n = 1024; Byte [] bs2 = new byte [N]; While (WR. readentitybody (bs2, n)> 0) { ..... } } 3. Custom multipart mime parser Automatically intercepts mime delimiters Write a file into blocks such as temporary files. Update Appliaction status in real time (receivingdata, error, complete)