python-Chinese Word segmentation Statistics Xjieba Library Xwordcloud Library

1. China Soft International Southern Region technical director of the former teacher will also come to two classes, students hope that the teacher said what content? (Think hard to answer)

• How do you deal with a communication or technical problem?

2. Chinese participle

(1) Download a Chinese novel and convert it into UTF-8 code.

1 ImportJieba2Novel = Open ('hundred Years Lonely txt','W', encoding='Utf-8')#new "Write" a txt3Novel.write (" "4 5 6 Hundred Years of Loneliness7  8 Marquez9 Ten Chapter I. One  A (text slightly) -  - " ") the novel.close () -  - Print(novel)

Save text, modify encoding format

#使用print () Check that the text is saved to the parameter.

(2) The use of Jieba library, the Chinese word frequency statistics, output TOP20 words and the number of occurrences.

Jieba Library Installation:

1 ImportJieba2Novel = Open ('hundred Years Lonely txt','W', encoding='Utf-8')#new "Write" a txt3Novel.write (" "4 5 6 Hundred Years of Loneliness7  8 (text slightly)9 Ten " ") One novel.close () A  - Print(novel) -  theTXT = open ('hundred Years Lonely txt','R', encoding='Utf-8'). Read () -  -Words =list (jieba.cut (TXT))#Jieba Library helps participle, and has removed punctuation -Exp = {',','\ n','.','. ','"','"',':','... ..',' ','? ',',','·'} +Key=set (words)-Exp -  +DIC = {}#Empty Dictionary A  at  forIinchKey: -dic[i]=Words.count (i) -  -Items = List (Dic.items ())#divide into tuples -  -Items.Sort (key=LambdaX:x[1],reverse=true)#Sort in  -  forIinchRange (20): to     Print(Items[i]) +  -Novel.close ()

Chinese Word segmentation statistics 1

(3) Excluding some meaningless words and merging the same word.

1 ImportJieba2 3 4TXT = open ('hundred Years Lonely txt','R', encoding='Utf-8'). Read ()5 6Words =list (jieba.cut (TXT))#Jieba Library helps participle, and has removed punctuation7Exp = {',','\ n','.','. ','"','"',':','... ..',' ','? ',',','·'}8Key=set (words)-Exp9 TenDIC = {}#Empty Dictionary One  A  forIinchKey: -     ifLen (i) >1: -dic[i]=Words.count (i) the     Else: -         Continue -  -Items = List (Dic.items ())#divide into tuples +  -Items.Sort (key=LambdaX:x[1],reverse=true)#Sort +  A  forIinchRange (20): at     Print(Items[i]) -  -Novel.close ()

Remove Meaningless words

* * Use the Wordcloud library to draw a word cloud.

python-Chinese Word segmentation Statistics Xjieba Library Xwordcloud Library