Python implements text deduplication (without disrupting the original order)

This article mainly introduces python's implementation of text deduplication without disrupting the original sequence. For more information about python's implementation of text deduplication without disrupting the original sequence, see the next article, for more information, see

The code is also found on the Internet, which is very efficient and especially suitable for deduplication of dictionary files.

# Coding = utf-8import sysdef open_txt (): # Open the TXT text write array try: xxx = file (sys. argv [1], 'r') for xxx_line in xxx. readlines (): passlist. append (xxx_line) xxx. close () into T: return 0def write_txt (): # Open the TXT text and write the array try: yyy = file (sys. argv [2], 'w') for I in list_passwed: yyy. write (I) yyy. close () failed t: return 0 global passlist # declare the global variable passlist = [] # Username: anonymous password is blank open_txt () # TXT import array # passlist = list (set (passlist) # python list deduplication global list_passwed # list deduplication without disrupting the original sequence list_passwed = [] for I in passlist: if I not in list_passwed: list_passwed.append (I) write_txt ()

Python reads TXT files to the array list to remove duplicates without disrupting the original order.

######################################## ########################### Qq: 316118740 # BLOG: # Remove duplicates from reading TXT files to the array list in python, don't disrupt the original sequence # sorry for the hard work you just learned ########################### ######################################## # def open_txt (): # Open the TXT text and write the array try: infile = file('admin.txt ', 'r') xxx = file('admin.txt', 'r') for xxx_line in xxx. readlines (): passlist. append (xxx_line) xxx. close () cancel T: return 0 def list_del (): # Clear list try: I = 0 # get the first element of list while I <len (passlist ): del passlist [I] del list_passwed [I] doesn't: return 0 ###################################### global passlist # declare the global variable passlist = [] # User name: anonymous password is blank www_cj (www) # domain name dismantling open_txt () # TXT import array # passlist = list (set (passlist) # python list deduplication global list_passwed # list deduplication, do not disrupt the original sequence list_passwed = [] for I in passlist: if I not in list_passwed: list_passwed.append (I) ###################################### traverse an array combine the password I1 = 0 # get the first element of the list while I1 <len (list_passwed): print "WWWWWWWWWWW", I1 if I1 = len (list_passwed): break # exit loop I2 = 0 # get the first element of list while I2 <len (list_passwed ): print "1111:", list_passwed [I1], "2222 :", list_passwed [I2] I2 = I2 + 1 # layer 2 I1 = I1 + 1 # layer 1 ###################### ################

This example describes how to sort TXT files in the original order after deduplication from the array and list. Share it with you for your reference.

The above is the detailed content of the python text deduplication method (without disrupting the original order). For more information, see other related articles in the first PHP community!