ASP. NET large file upload topic (1)-Overview: difficulties in uploading large files

 

Body part:

Note: The following red font indicates that the content is referenced on Microsoft websites.

To create a video website, you must provide the video upload function. However, the upload control provided by ASP. NET can only be used to upload small files, which obviously cannot meet the requirements. Some friends may ask, "Why do you need to use that? It is not good to directly transfer data via FTP?" Yes, it is much easier to use ftp, but after uploading data via FTP, you cannot edit uploaded files online (such as format conversion and adding files to the database). All these cumbersome tasks that can be automatically completed by the website will be handed over to the Administrator, this will undoubtedly increase the employment cost for a commercial website. In addition, these repetitive low-brainpower jobs seem to be no different from physical work, which does not reduce the value of our IT staff from another aspect. Why does Ms limit the upload control's capabilities so small? On msdn and other Microsoft websites, we can see that the default value of the maxrequestlength parameter in the
However, this will cause trouble for us to upload large files. Some may find that since the size of the file to be uploaded is limited by the maxrequestlength parameter, it is okay to change the value to a greater value. Indeed, in this way, you can easily increase the file upload size limit, but in the "Upload process, Asp. net first loads the entire file into the memory, and then the user can save the file to the disk. "That is to say, if the size of the File Uploaded by the user is 100 MB, MB of the server memory will be used to store the files uploaded by the user; if 10 users upload files simultaneously, for the moment, we will not mention the parallelism problem. Then, 10 users will occupy 100 MB of memory. What if it is 1000, or even tens of thousands of users? In this case, the memory size is not enough for user uploads.

"Other factors also affect the maximum file size that can be uploaded. These factors include available memory, available hard disk space, processor speed, and current network traffic. For files uploaded with regular traffic, Microsoft recommends that you set the maximum file size to between 10 and 20 mb. If you seldom upload files, the maximum file size can be 100 MB. "

An enterprise's internal video website estimates that its daily traffic is 1000. According to Microsoft's suggestion, the size of the uploaded files should be controlled within 20 mb, however, such a size can easily lead to paralysis of the server. After comprehensive consideration, I will keep the size within 6 MB (why is it in this range? I will tell you in advance to make Article Can be consistent ). We will be wondering, "generally, a video must have at least 100 bytes in size ~ 200 m, not much smaller than 6 m, right ?" Yes, this problem is the key to our solution.

Before solving this problem, we must clear one thing. The file size "6 m" and "200 m" here refer to the server, or the client? The correct understanding is as follows: m is the size for client users. Generally, users are allowed to upload files of a maximum size of M (this m is based on the hard disk size of my server, you can master it flexibly, but it is generally better not to exceed 1 GB); 6 MB refers to the file size that the server can receive, in this way, the server's memory will not be swallowed up by uploading files.

Now we can turn the problem into: How can the server receive M videos uploaded by users in a size smaller than 6 m?

If you see it a little dizzy, let me give you an example of "the sale of Olympic tickets.

These days are the last phase of Olympics tickets. Many people wait several days before the ticket window to get tickets at the last stage. Let's assume that million people need tickets (just like uploading a m video ), the ideal situation is to open windows to sell tickets (the window here is equivalent to the file size that the server can receive, and is equivalent to setting maxrequestlength and other related elements very large ). But if you think about it, you will know that it is impossible to do so (for the reason, think about it ~). How can this problem be solved? Only the number of windows is limited to a certain number (for example, 20) according to certain rules, and then the person who wants to buy a ticket queues to buy a ticket.

I believe that through this example, some friends may have thought of how to solve our previous problems. The method is to send requests to the server group in some way.

This part is written first.

[Supplement] many of my friends have suggested the following: "ASP. NET 2.0 or later versions seem to have not all been put into the memory. First of all, I am very grateful for your support and find that I do not have enough in-depth understanding of these contents. As a reminder, I have consulted some relevant materials, so I will make another supplement to avoid other people making the same mistakes. The following content is taken from: upload a file in an ASP. NET application.

For some server-side technologies such as spring framework or early ASP. NET 1.1 Program Processing will completely load the user's uploaded content into the memory, which indeed brings problems. However, the Protocol does not specify the method used by the server to process uploaded files. For example, in the current ASP. NET 2.0 already exists in a temporary file on the hard disk after the user uploads more data. This is completely transparent to developers, that is, developers can process data streams as before.

The threshold value (threshold) for enabling temporary files on the hard disk in ASP. NET 2.0 is configurable:

< System. Web >
< Httpruntime
Maxrequestlength = " Int32 "
Requestlengthdiskthreshold = " Int32 " />
</ System. Web >

Maxrequestlength does not have to be said, just got in touch with ASP. net friends will always find that the size of the uploaded file cannot exceed 4 MB, which is because the default maxrequestlength is 4096, which limits the size of each request to not exceed 40 96kb. This is intended to protect applications from malicious requests. When the request exceeds maxrequestlength, the ASP. NET handler will not process the request. Here and ASP. net throws an exception, which is why if the user uploads a file too large, it is not ASP. the error page specified in the. NET application (or the default one), because ASP. net has not processed this request.
Requestlengthdiskthreshold is the threshold value just mentioned. Its default value is 256. That is, when the request content exceeds kb, the hard disk is used as the cache. This threshold is theoretically irrelevant to whether the client is uploading content, as long as the request sent by the client is greater than this threshold. Therefore, in ASP. NET 2.0, the server's memory will not be exhausted due to abnormal client requests.

Next article