Python Chinese Word Segmentation: uses the jieba word segmentation to perform word segmentation on python,

Python Chinese Word Segmentation: uses the jieba word segmentation to perform word segmentation on python,

When collecting the beauty site, you need to perform word segmentation on the keywords. The final method is python's jieba word segmentation.

Chinese Word Segmentation is a basic task of Chinese text processing. It is used for Chinese word segmentation. The basic implementation principle is as follows:

Installation (Linux)

Download the toolkit, decompress the package, go to the directory, and run: python setup. py install

 

Mode

 

Interface

• The component only provides the jieba. cut Method for word segmentation.
• The cut method accepts two input parameters:
  • The first parameter is the string to be segmented.
  • The cut_all parameter is used to control the word segmentation mode.
• The string to be segmented can be a gbk string, UTF-8 string, or unicode string.
• Jieba. the structure returned by cut is an iterative generator. You can use the for loop to obtain each word (unicode) obtained after word segmentation. You can also use list (jieba. cut (...)) convert to list
• Seg = jieba. cut ("http://www.gg4493.cn /"):

 

Instance

#! -*-Coding: UTF-8-*-import jiebaseg_list = jieba. cut ("I came to Beijing Tsinghua University", cut_all = True) print "Full Mode :",''. join (seg_list) seg_list = jieba. cut ("I came to Beijing Tsinghua University") print "Default Mode :",''. join (seg_list)

Result