Cut large files by line (linux split command Lite version), linuxsplit

Cut large files by line (linux split command Lite version), linuxsplit
Cutting large files by line (linux split command Lite version)

#-*-Coding: UTF-8-*-_ author _ = 'knowlifedeath ''' the Split command on Linux allows you to easily Split large files, this PY script is used to cut the file Python Version: 2.7 ''' import timeimport sysprint time on Windows based on the specified number of lines. strftime ('% Y-% m-% d % H: % M: % s', time. localtime (time. time () ''' the parameter '''bigfile = "E: \ Data \ Big.txt "OneUnitEnd = 1000000 # Number of split blocks ZeroFillWidth = 3 # zfill parameter is the serial number encoding length of small files seed = 0 # Number of Current Blocks UnitNo = 0 # Number of Current Blocks fSmallFile = open ('{0} \ samll1_1}.txt '. format (sys. pa Th [0], '0 '. zfill (ZeroFillWidth), 'w') fBigFile = open (BigFile) for line in fBigFile: rawLineData = line. strip ('\ n '). strip () if len (rawLineData )! = 0: seed + = 1 fSmallFile. write ('{0} \ n '. format (rawLineData) if seed = OneUnitEnd: fSmallFile. close () seed = 0 UnitNo + = 1 fSmallFile = open ('{0} \ samll1_12.16.txt '. format (sys. path [0], str (UnitNo ). zfill (ZeroFillWidth), 'w') # The last small file may be an empty file (full time) # a/w mode has little speed difference, hundreds of millions of data records are handled between 1 minute and a half to 2 minutes. # close multiple times. There will be no problem with fSmallFile. close () print time. strftime ('% Y-% m-% d % H: % M: % s', time. localtime (time. time ()))