Python smallseg word segmentation usage Example Analysis

Python smallseg word segmentation usage Example Analysis

This example describes how to use the Python smallseg word segmentation function. Share it with you for your reference. The specific analysis is as follows:

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

# Encoding = UTF-8 # Import psyco # Psyco. full () Words = [x. rstrip () for x in open ("main. dic", mode = 'R', encoding = 'utf-8')] From smallseg import SEG Seg = SEG () Print ('Load dict ...') Seg. set (words) Print ("Dict is OK .") Def cuttest (text ): Wlist = seg. cut (text) Wlist. reverse () Tmp = "". join (wlist) Print (tmp) Print ("================================== ") If _ name __= = "_ main __": Cuttest ("this is a dark night without a finger. My name is Sun Wukong. I love Beijing. I love Python and C ++. ") Cuttest ("I don't like Japanese kimono. ") Cuttest ("the Thunder Monkey returns to the human world. ") Cuttest ") Cuttest ("I need rent a house ") Cuttest ("Yonghe clothing & accessories Co., Ltd ") Cuttest ("I Love Tiananmen Square, Beijing ") Cuttest ("abc ") Cuttest ("Hidden Markov ") Cuttest ("Ray monkey is a good website ") Cuttest (the word "Microsoft" consists of "MICROcomputer" and "SOFTware ") Cuttest ("grass mud horse and deception horse are popular words this year ") Cuttest ("Ito Yang huatang Head Office Store ") Cuttest ("Institute of computing technology, Chinese Emy of Sciences ") Cuttest ("Romeo and Juliette ") Cuttest ("I bought items and costumes ")

Smallseg word segmentation, running on python3.3 is slightly problematic. The py code xrange has been renamed as range in 3. In addition, there is no decode function in 3.

After modifying the above two points, the code will be portable. The effect is acceptable.

I hope this article will help you with Python programming.