Output large txt files on a line-by-row basis on the web

In some scenarios, you need to present some log files on the Web that are some txt on the file server.

When a log file is large, downloading logs can cause the page to get stuck for a long time, stay in the loading state, parse the log after downloading the log and generate the DOM, and a large amount of DOM rendering can cause the page to crash.

So I want to optimize the output of the log, the beginning of the download is a line on the page to print the log, just like some IDE in the output program compilation process.

The final implementation method is as follows:

When downloading the file, let the request over a layer of agent, agent write output stream when segmented output:

?

int l;        byte[] buffer = new BYTE[100];        String llength = "";        int lsum = 0;        Do        {            L = proxyresponsestream.read (buffer, 0, buffer. Length);            Llength + = "," + L;            Lsum + = l;            if (L > 0)            {                context. Response.Write (System.Text.Encoding.UTF8.GetString (buffer, 0, L));                Context. Response.Flush ();            }        }        while (L > 0);        Context. Response.End ();

Client Request:

var xhr = new XMLHttpRequest (), Xhr.open (' GET ', url, true), Xhr.onreadystatechange = function (e) {                if (this.readystate = = 3) {                    //alert (1);                    var newload = E.target.responsetext.slice (logs[type].length);                    var newloadlogs = newload.split (' \ n ');                    Newloadlogs.foreach (function (line, index) {                        if (index = = newloadlogs.length-1) return;                        Logs[type] + = line + ' \ n ';                        $ ("#" + Type). Append ("<p class= ' logline" + getlogtype (line) + ">" + line + "</p>");                        Document.body.scrollTop = document.body.scrollHeight;}                    );                }            ;            Xhr.onload = function () {                if (callback) callback ();            }            Xhr.send ();

onReadyStateChange

Response multiple output values, ReadyState has been the 3,onreadystatechange event can be triggered multiple times.

This can really be done without waiting to start the line-by-row print log directly.

but in the process of implementation, it was found that Some questions :

1.response objects in the write output stream to the client, they also have set a buffer-like thing, but this buffer size is very large, for the general txt, even if the capture of the ReadyStateChange event, but also do not feel it is in the segmented output. When the size of the buffer is manually set small, full of direct flush, you can see the log a small paragraph of a small section of the print out. Unfortunately, this greatly reduces the speed at which files are downloaded and consumes a lot of system resources, without the intention of optimizing the log presentation.

2.response in a section of the output file content, in the ReadyStateChange event does not get the value of each piece of output, but the new output of the content continues to the output behind the append. To parse the DOM in such a way, it is necessary to keep track of the location and intercept the string, which is a huge drain on the browser.

3. I take a 30M size txt on this machine to test, start output log after CPU occupies instantaneous to 90%, output about half after the browser crashes ...

So the final conclusion is that this method can only be used as a small log file, an optimization of the interactive experience, and the need to consume a lot of system resources. The browser is not suitable for direct display of the larger txt, its long download wait and the great pressure on the browser is not optimized, either segmented view, or downloaded after viewing or analyzing it to show the results.