Calculation Article TF-IDF

#coding: Utf-8
Import Jieba
Import Jieba.analyse #计算tf-IDF need to call this module Jieba.analyse
Stopkey=[line.strip (). Decode (' Utf-8 ') for line in open (' Stopkey.txt '). ReadLines ()]
#将停止词文件保存到列表stopkey, stop the word download on the Internet.
Neirong = open (R "Ceshi1.txt", "R"). Read () #导入需要计算的内容
zidian={}

Fenci=jieba.cut_for_search (Neirong) #搜索引擎模式分词
For FC in Fenci:

If FC in Zidian:
Zidian[fc]+=1 #字典中如果存在键, key value plus 1,
Else
Zidian.setdefault (fc,1) #字典中如果不存在键, join the key, set the key value to 1

Quanzhong=jieba.analyse.extract_tags (neirong,topk=20) #计算tf-IDF, output the first 20 weight words.

For QG in Quanzhong:
If QG in Stopkey: #如果qg存在停止词stopkey里面, pass
Pass
else: #不存在的话就输出qg和出现qg的次数
Print qg+ "," + ' ZIDIAN[QG] ' #输出权重词和权重词出现的次数

Calculation Article TF-IDF