Python list deduplication, Python list

Python list deduplication, Python list

Boring statistics on how many methods are available to deduplicate the list.

1. Set

list(set(alist))

To maintain the order:

import randomif __name__=='__main__':    a=[random.randint(0,10) for i in xrange(10)]    b=list(set(a))    b.sort(key=a.index)

2. Dictionary

Most of them use the hash table feature

{}.fromkeys(alist).keys()

Or manually write:

import randomif __name__=='__main__':    a=[random.randint(0,10) for i in xrange(10)]    d={}    for i in a:        d[i]=1    print d.keys()

3. Re-query after sorting

import randomif __name__=='__main__':    a=[random.randint(0,10) for i in xrange(10)]    a.sort()    print [x for i,x in enumerate(a) if not i or x!=a[i-1]]

4. itertools. groupby

import randomimport itertoolsif __name__=='__main__':    a=[random.randint(0,10) for i in xrange(10)]    a.sort()    print [x[0] for x in itertools.groupby(a)]

5. Traverse

import randomif __name__=='__main__':    a=[random.randint(0,10) for i in xrange(10)]    b=[]     for i in a:        if i not in b:            b.append(i)

Or use reduce:

import randomimport functoolsif __name__=='__main__':    a=[random.randint(0,10) for i in xrange(10)]    functools.reduce(lambda x,y:x if y in x else x+[y],[[],]+a)

Are there other methods?

 

Python removes duplicates from the list

List (set (l ))

Python list problems

List = ['my \ tname \ tis \ tandrew ']
New = list [0]. split ('\ t ')