Python reads data from a text file in the GZ compressed file directly by line __python

Source: Internet
Author: User
Tags gz file python script

Previously wrote a log file (TXT file) to extract a specific log, written to the MySQL database script, because the log is too large, maintenance staff to pack the log compressed into the tar.gz format.

TXT file before a single file of more than 2G, a single TXT file packaging compressed into a tar.gz file. So my Python script needs to be modified as well. (Server centos6.3)

Originally wanted a solution, that is, to extract the tar.gz, and then read, read the completion of the extracted files removed, this solution is not not, but not very good, a large file decompression is slow, another decompression after the comparison of the server disk.


Another scenario was found to read the data of the text file in the GZ compressed file directly by line.


This is my test script under Windows:

Import OS
import os.path
import gzip


def read_gz_file (path):
    if Os.path.exists (path):
        with Gzip.open (path, ' R ') as PF: for line in
            PF:
                yield line
    else:
        print (' The path [{}] is not exist! '. Format (path)

con = read_gz_file (' c:\\1.gz ')
if getattr (Con, ' __iter__ ', None): For line in
    Con:
        Print (line)

strzipfile = ' c:\\1.gz '
strdstfile = ' c:\\2 '
file   =   gzip. Gzipfile (Strzipfile,   "R")
outfile   =   Open (Strdstfile   , "w")
Outfile.write ( File.read ())
Outfile.close ()


Attachment is 1.gz file



Run Result:

Sdfasfda


Asdfasdf






Asdfasdf


Adsfadf



Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.