Python Script Learning (i)

Log Read script

Function: Used to read a log file, you can specify a match condition, return the line that matches in the text and the previous n rows.

This script can accept 3 parameters, namely, the file object, the keyword searched, and the number of rows preceding the line that matches. #!/usr/local/python27/bin/python2.7import sysfrom collections import dequedef search ( F,pattern,keep_num): #定义一个队列, sets the maximum number of queues, the data in this queue can be overwritten, and if the maximum number of queues is reached, the newly added data will overwrite the previous one.     pre_lines = deque (Maxlen=keep_num)     for line  in f:        if pattern in line:             yield line,pre_lines# The logic here is to be read from the file object F each line to make a pattern matching judgment, if the mismatch is placed in the Pre_lines queue, continue to find the next line, only the maximum allowable number of rows, this parameter MaxLen control, the extra data is covered in front, until the required keyword matching , a generator is returned, and the generator includes rows that match to, and N rows before the row, which is the row previously saved in the Pre_lines queue.                    Pre_lines.append (line) if __name__ ==  ' __main__ ':    log_file =  Sys.argv[1]    pattern = sys.argv[2]    keEp_num = int (Sys.argv[3])     with open (log_file)  as f:# This loop takes data from the generator returned by the search function, which is stored in the variable and then printed separately.             for line,pre_lines in  Search (F,pattern,keep_num):            for  pline in pre_lines:                 print pline             print line            print  "-"  *  20

Key-value pair processing scripts

To handle a configuration file of key---value, key may appear multiple times, corresponding to the same or different value, requiring that all non-repeating value corresponding to each key be returned.

Here we first talk about the defaultdict and dict of the collections module.

Here Defaultdict (function_factory) constructs an object similar to dictionary, where the value of keys is determined by itself, but the type of values is the class instance of Function_factory, and has a default value. For example, default (int) creates a similar dictionary object in which any values are instances of int, and even if there is a nonexistent key, D[key] has a default value, which is an int The default value of () is 0.

The author's understanding:

Defaultdict accepts a factory function as a parameter, and the type of the incoming factory function determines the type and value of the keys in the Dictionary object.

For example, defaultdict (set) here passed a set type, which means that the keys is a collection, to add data to the key to use the set of the built-in Add method, the corresponding value will also conform to the characteristics of the set, unordered, uniqueness.

If defaultdict (list) is passed into a list type here, which means that the keys in it are lists, to add data to the key to use the list of built-in methods append, the corresponding value will also conform to the characteristics of the list, ordered, repeatable.

Process the following files:

key1=111

key2=222

key1=111

Key1=123

key3=333

key4=111

key5=555

key6=666

key2=222

key7=777

key8=111

To implement all the non-repeating value corresponding to each key, use the set type.

code example:

!/usr/local/python27/bin/python2.7import sysfrom Collections Import defaultdictconf = Defaultdict (set) for line in open (    SYS.ARGV[1]): K,v = line.split (' = ') #由于传入的工厂函数为set, so the key here is the set, to use the Add method of the collection to insert the value. Conf[k.strip ()].add (V.strip ()) for k,v in Conf.items (): print "%s =%s"% (K,V)

Output Result:

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/72/65/wKiom1XilAuQZAu0AAC9tahbXi0683.jpg "title=" ff0v ' Z ]buw@um1ta7$ (6gvn.png "alt=" Wkiom1xilauqzau0aac9tahbxi0683.jpg "/>

You can observe that there are multiple duplicate key1=111 in the text that have only been output once.

Compare the incoming list type below

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/72/65/wKiom1XildqAI66KAAEEIfC75aI494.jpg "title=" 1.png " alt= "Wkiom1xildqai66kaaeeifc75ai494.jpg"/>

Output Result:

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/72/61/wKioL1XimBawg5RWAADGd4vyXYI244.jpg "title=" 2.png " alt= "Wkiol1ximbawg5rwaadgd4vyxyi244.jpg"/>

There are some tasks that need to be saved in the dictionary, key is the name, value is content, but it needs to be kept in order when it is executed.

Method (i)

Use a dictionary to save data and attach a list save order

#!/usr/local/python27/bin/python2.7import sysd1=dict () L1=[]for line in open (Sys.argv[1]): K,v = line.split (' = ') l1.a Ppend (k) D1[k] = Vprint ("%s =%s"% ([i-I in l1],[d1[i] for i in L1]))

Method (ii)

Using Ordereddict

#!/usr/local/python27/bin/python2.7import sysfrom Collections Import Ordereddictod = Ordereddict () for line in open ( SYS.ARGV[1]): K,v = line.split (' = ') Od[k.strip ()] = V.strip () for k,v in Od.items (): Print k,v

The General Dictionary Dict () is unordered, but Ordereddict is an ordered dictionary that stores data in the order in which it is inserted.

Count the top 10 words in an English article

code example:

#!/usr/local/python27/bin/python2.7import sysimport refrom Collections Import Counterwith Open (Sys.argv[1]) as f:#    Matches the words so that all are converted to lowercase and saved in a list. Words = Re.findall (r "\w+", F.read (). Lower ()) #Counter方法可以从一个列表中统计每个元素出现的次数,. Most_common (n) is used to filter out most occurrences of n; print Counter (words). Most_common (10)

Output Result:

./counter.py English_article.txt

[(' To ', (' his ', +), (' him ', "), (' in ', '), (' Tyler ', 9), (' She ', 9), (' and ', 9), (' That ', 8), (' he ', 8), (' I ', 8)

This article is from the "Break Comfort zone" blog, so be sure to keep this source http://tchuairen.blog.51cto.com/3848118/1689877

Python Script Learning (i)