Previously wrote a log file (TXT file) to extract a specific log, written to the MySQL database script, because the log is too large, maintenance staff to pack the log compressed into the tar.gz format.
TXT file before a single file of more than 2G, a single TXT file packaging compressed into a tar.gz file. So my Python script needs to be modified as well. (Server centos6.3)
Originally wanted a solution, that is, to extract the tar.gz, and then read, read the completion of the extracted files removed, this solution is not not, but not very good, a large file decompression is slow, another decompression after the comparison of the server disk.
Another scenario was found to read the data of the text file in the GZ compressed file directly by line.
This is my test script under Windows:
Import OS
import os.path
import gzip
def read_gz_file (path):
if Os.path.exists (path):
with Gzip.open (path, ' R ') as PF: for line in
PF:
yield line
else:
print (' The path [{}] is not exist! '. Format (path)
con = read_gz_file (' c:\\1.gz ')
if getattr (Con, ' __iter__ ', None): For line in
Con:
Print (line)
strzipfile = ' c:\\1.gz '
strdstfile = ' c:\\2 '
file = gzip. Gzipfile (Strzipfile, "R")
outfile = Open (Strdstfile , "w")
Outfile.write ( File.read ())
Outfile.close ()
Attachment is 1.gz file
Run Result:
Sdfasfda
Asdfasdf
Asdfasdf
Adsfadf