When capturing a page image, to avoid repeated crawls, the IMG result (the result set is the list type) is crawled through the collection. Here is a summary of the following online collection of several methods.
One, Method 1
| The code is as follows |
Copy Code |
|
ids = [1,2,3,3,4,2,3,4,5,6,1]
News_ids = []
For ID in IDs:
If ID not in News_ids:
News_ids.append (ID)
Print News_ids |
The idea seems clear and simple, and you can keep the order of the previous arrangement.
Two, Method 2
Processing by using the Set method
| The code is as follows |
Copy Code |
|
ids = [1,4,3,3,4,2,3,4,5,6,1]
ids = List (set (IDS)) |
It is simpler to work with set method set for processing, but the results do not preserve the previous order.
Third, Method 3
Using lambda anonymous functions and reduce functions to handle
| The code is as follows |
Copy Code |
ids = [1,4,3,3,4,2,3,4,5,6,1]
Func = lambda x,y:x if y in x else x + [y]
Reduce (func, [[],] + IDs)
|
Four, Method 4
Using the Itertools module
| The code is as follows |
Copy Code |
|
Import Itertools
ids = [1,4,3,3,4,2,3,4,5,6,1]
Ids.sort ()
it = itertools.groupby (IDs)
For K, G in it:
Print K |
V. Unable to maintain the original order
| The code is as follows |
Copy Code |
|
Liebiao=set (Liebiao) |
Six, while traversal to heavy
| The code is as follows |
Copy Code |
|
def delrepeat (Liebiao):
For x in Liebiao:
While Liebiao.count (x) >1:
Del Liebiao[liebiao.index (x)]
Return Liebiao |
Vii. examples
The program is very simple, directly on the code:
| The code is as follows |
Copy Code |
#coding =utf-8
Def open_txt (): #打开TXT文本写入数组
Try
xxx = file (R ' C:\Users\Administrator\Desktop\user.txt ', ' R ')
For Xxx_line in Xxx.readlines ():
Passlist.append (Xxx_line)
Xxx.close ()
Except
return 0
Def write_txt (): #打开TXT文本写入数组
Try
yyy = File (R ' C:\Users\Administrator\Desktop\list_user.txt ', ' W ')
For I in list_passwed:
Yyy.write (i)
Yyy.close ()
Except
return 0
Global Passlist #声明全局变量
Passlist = [] #用户名: anonymous password is empty
Open_txt () #TXT导入数组
#passlist = List (set (passlist)) #python listing to weight
Global list_passwed #列表去重, do not disturb the original order
List_passwed=[]
For I in Passlist:
If I not in list_passwed:
List_passwed.append (i)
Write_txt ()
|
The aside# set method can be directly weighed and sorted, and the set is the fastest if it needs to be sorted.