Efficient cut TXT file based on BinaryReader

Daily work is unavoidable to face some file operation.

But if it is a log file: With G's. It's not so easy to handle.

Especially when you have a lot of files.

This time it will be used to cut large files.

The example shown below is an experiment with a 10G TXT file to do the following: The effect can also be: It took about a minute to cut all the files into 50M.

Here's the code: The specific process is in the comments. See for yourself.

to experiment. Make a big file out first. Don't look for it. Run these two lines of code to get one.

StreamWriter writer=New StreamWriter (@ "E:\FTPS\data.csv");  for (int0int.) MaxValue; i++) {    writer. WriteLine (i.ToString ()+"\ t    ' fsdjklfjsdl ', ' sdfsdlfjlsdjf ', ' f123 of both sides ', ' F-Hair Stephen sdjklfjsdl ', ' SD place FSDLFJLSDJF ', ' F1 Stephen 2312dffdsfsdf '"); Writer. Dispose ();

There is a line number in front of it. Easy to cut and look at data integrity.

And then the dry goods.

            //source File            stringSourceFile =@"E:\FTPS\data.csv"; //file output directory after splitting            stringSPLITFILEDIRF =@"E:\FTPS\"; //the basic size of each file            LongSplitfilesize =1024x768*1024x768* -; //Start Position            LongReadstart =0; //Split file name ordinal            intSplitfilenum =1; Try{FileStream Stream=NewFileStream (sourcefile, FileMode.Open); LongFiletotallength =Stream.                Length; //creating binary reads                using(BinaryReader reader =NewBinaryReader (Stream,encoding.utf8)) {                    //sets the position of the start read directly to the byte of the base size//the next thing to do is to find the end of the line.Reader. Basestream.position = Splitfilesize-1; //determine the current location does not exceed the total file size                     while(Reader. Basestream.position <=filetotallength) {                        //Next asked character find newline//this is about reader. The ReadByte () method executes automatically when reader is executed. The value of basestream.position backwards +1//Some examples on the web have been performed readbyte in addition to do position++ obviously have characters isolated                         while(Reader. Basestream.position < filetotallength && Reader. ReadByte ()! = -) { }                        //Create a stream for the file address of the outputFileStream filestreamout =NewFileStream (SPLITFILEDIRF + splitfilenum +". csv", FileMode.Create); //Creating a binary write                        using(BinaryWriter BinaryWriter =NewBinaryWriter (Filestreamout, Encoding.UTF8)) {                            //This gets the number of bytes in the middle of the position of the byte that is now found to wrap to the beginning of this traversal//+1 is to take the found line-break .                            intReadwrodcountnow = (int) (reader. Basestream.position-readstart +1); //Resets the starting position of the read to the beginning of this queryReader. Basestream.position =Readstart; //Write the contents of this read to a new fileBinaryWriter.Write (reader.                        Readbytes (Readwrodcountnow)); }                        //take this read to the next location as the starting positionReadstart =Reader.                        Basestream.position; //Next Verification Start location Verify line breaks from hereReader. Basestream.position = Readstart + splitfilesize-1; //Output File namesplitfilenum++; }                }            }            Catch(Exception ex) {Console.WriteLine (ex).            Message); } Console.WriteLine (" Complete"); //Console.readkey ();

All right.. The code is here.

You know what I mean? There is a problem with the code on the Internet. said in the note where the problem is.

The core code is this: To be optimized can be packaged a bit ...

such as partition symbol customization. What is the size of the file customization? Get it yourself.

Efficient cut TXT file based on BinaryReader