Python list deduplication, python list

Python list deduplication, python list

The problem is to de-duplicate the news id in a list. After de-duplicating, ensure that the order remains unchanged.

Intuitive method

The simplest idea is:

 

Copy the Code as follows:
Ids = [1, 2, 3, 4, 2, 3, 4, 5, 6, 1]
News_ids = []
For id in ids:
If id not in news_ids:
News_ids.append (id)

 

Print news_ids

 

This is also feasible, but it does not look good enough.

Use set

Another solution is to use set:

Copy the Code as follows:
Ids = [, 1]
Ids = list (set (ids ))

The original sequence is not maintained.

 

Sort by index again

Finally, the solution is as follows:

Copy the Code as follows:
Ids = [, 1]
News_ids = list (set (ids ))
News_ids.sort (ids. index)

 

Use itertools. grouby

The article mentioned itertools. grouby at the beginning. This can be used without considering the list order:

Copy the Code as follows:
Ids = [, 1]
Ids. sort ()
It = itertools. groupby (ids)

 

For k, g in it:
Print k

 

For the itertools. groupby principle can be viewed here: http://docs.python.org/2/library/itertools.html#itertools.groupby

Added: reduce

The netizen reatlk sent a message to another solution. I will add and explain here:

Copy the Code as follows:
In [5]: ids = [,]

 

In [6]: func = lambda x, y: x if y in x else x + [y]

In [7]: reduce (func, [[],] + ids)
Out [7]: [1, 4, 3, 2, 5, 6]

The above is the code I run in ipython, where lambda x, y: x if y in x else x + [y] is equivalent to lambda x, y: y in x and x or x + [y].

 

The idea is to first change ids to [[], 3,...], and then use the reduce feature. Reduce explained here: http://docs.python.org/2/library/functions.html#reduce