Today intends to use Redis's bitset to engage in a bloom filter, the advantage is that you can save memory, the disadvantage is that there may be some data because the prompts are repeated and cannot be saved.
The general principle of bloom filter is to map a string to several different bits with different hash functions and set these different bits to 1.
If you look for a string, you find that the bit that is mapped by the hash is not 1, indicating that the string does not exist.
If all the bits are found to be 1, then the string has a certain probability of not being present, which is usually a small probability.
Related content can be viewed:
http://olylakers.iteye.com/blog/2089275
But it's Java, and I'm looking for a python on GitHub.
http://github.com/jaybaird/python-bloomfilter/
But this is based on Python's BitArray, slightly modified, and deleted a lot of things, so that it becomes the use of Redis Bitset
Https://github.com/walkwalkwalk/python_redis_bloomfilter
Redis and Bloom Filter