Extracting strings in Python

Today, when dealing with a MapReduce program with Streaming-python, I found that the reducer failed because of the memory-consuming limit! Look at the code carefully, found that there is a collection of URLs, and the URL length is relatively long, the direct saving is really memory-intensive, so think of using compressed storage, and then use the time to decompress, although processing time increased, but the memory consumption is greatly reduced!

Specifically, the Zlib module is used

Import zlib raw_data = "hello,world,ooooooooooooxxxxxxxxxxx" Zb_data = zlib.compress (raw_data) print "Len (raw_data) =%d, Len (zb_data) =%d, compression ratio=%.2f "      % (Len (raw_data), Len (zb_data), float (len (zb_data))/len (Raw_data)) # len (Raw_data) =35, Len (zb_data) =25, compression ratio=0.71raw_data2 = zlib.decompress (zb_data) Print Raw_data2

Reference: http://my.oschina.net/1123581321/blog/176570

Extracting strings in Python