Python reads data from a text file in the GZ compressed file directly by line __python

Previously wrote a log file (TXT file) to extract a specific log, written to the MySQL database script, because the log is too large, maintenance staff to pack the log compressed into the tar.gz format.

TXT file before a single file of more than 2G, a single TXT file packaging compressed into a tar.gz file. So my Python script needs to be modified as well. (Server centos6.3)

Originally wanted a solution, that is, to extract the tar.gz, and then read, read the completion of the extracted files removed, this solution is not not, but not very good, a large file decompression is slow, another decompression after the comparison of the server disk.

Another scenario was found to read the data of the text file in the GZ compressed file directly by line.

This is my test script under Windows:

Import OS
import os.path
import gzip


def read_gz_file (path):
    if Os.path.exists (path):
        with Gzip.open (path, ' R ') as PF: for line in
            PF:
                yield line
    else:
        print (' The path [{}] is not exist! '. Format (path)

con = read_gz_file (' c:\\1.gz ')
if getattr (Con, ' __iter__ ', None): For line in
    Con:
        Print (line)

strzipfile = ' c:\\1.gz '
strdstfile = ' c:\\2 '
file   =   gzip. Gzipfile (Strzipfile,   "R")
outfile   =   Open (Strdstfile   , "w")
Outfile.write ( File.read ())
Outfile.close ()

Attachment is 1.gz file

Run Result:

Sdfasfda


Asdfasdf






Asdfasdf


Adsfadf