Python summarizes the de-duplicated methods for values in the list structure, pythonlist

Python summarizes the de-duplicated methods for values in the list structure, pythonlist

I encountered a problem today. I used the itertools. groupby function at my colleague's prompt. However, this is useless.
The problem is to de-duplicate the news id in a list. After de-duplicating, ensure that the order remains unchanged.
Intuitive method
The simplest idea is:

ids = [1,2,3,3,4,2,3,4,5,6,1]news_ids = []for id in ids:  if id not in news_ids:    news_ids.append(id)print news_ids

This is also feasible, but it does not look good enough.
Use set
Another solution is to use set:

ids = [1,4,3,3,4,2,3,4,5,6,1]ids = list(set(ids))

The original sequence is not maintained.
Sort by index again
Finally, the solution is as follows:

ids = [1,4,3,3,4,2,3,4,5,6,1]news_ids = list(set(ids))news_ids.sort(ids.index)

Use itertools. grouby
The article mentioned itertools. grouby at the beginning. This can be used without considering the list order:

ids = [1,4,3,3,4,2,3,4,5,6,1]ids.sort()it = itertools.groupby(ids)for k, g in it:  print k

For the itertools. groupby principle can be viewed here: http://docs.python.org/2/library/itertools.html#itertools.groupby
Reduce
The netizen reatlk sent a message to another solution. I will add and explain here:

In [5]: ids = [1,4,3,3,4,2,3,4,5,6,1]In [6]: func = lambda x,y:x if y in x else x + [y]In [7]: reduce(func, [[], ] + ids)Out[7]: [1, 4, 3, 2, 5, 6]

The above is the code I run in ipython, where lambda x, y: x if y in x else x + [y] is equivalent to lambda x, y: y in x and x or x + [y].
The idea is to first change ids to [[], 3,...], and then use the reduce feature. Reduce explained here: http://docs.python.org/2/library/functions.html#reduce