>>> import nltk, re, pprint
讀取網路上儲存的資料：
>>> from __future__ import division
>>> import nltk,re,pprint
>>> from urllib import urlopen
>>> url = url = "http://www.gutenberg.org/files/2554/2554.txt"
>>> raw = urlopen(url).read()
>>> type(raw)
<type 'str'>
>>> len(raw)
1176893
>>> raw[:75]
'The Project Gutenberg EBook of Crime and Punishment, by Fyodor Dostoevsky\r\n'
如果使用代理，則用下面代碼：
如果你使用的網際網路代理程式Python不能正確檢測出來，你可能需要用下面的方法手動指定代理：
>>> proxies = {'http': 'http://www.someproxy.com:3128'}
>>> raw = urlopen(url, proxies=proxies).read()
對讀入的資料處理：
>>> tokens = nltk.word_tokenize(raw)
>>> type(tokens)
<type 'list'>
>>> len(tokens)
244484
>>> tokens[:10]
['The', 'Project', 'Gutenberg', 'EBook', 'of', 'Crime', 'and', 'Punishment', ',', 'by']
>>> tokens[:15]
['The', 'Project', 'Gutenberg', 'EBook', 'of', 'Crime', 'and', 'Punishment', ',', 'by', 'Fyodor', 'Dostoevsky', 'This', 'eBook', 'is']
對得到的文本進一步處理和操作：
>>> text = nltk.Text(tokens)
>>> type(text)
<class 'nltk.text.Text'>
>>> text[1020:1060]
['had', 'successfully', 'avoided', 'meeting', 'his', 'landlady', 'on', 'the', 'staircase.', 'His', 'garret', 'was', 'under', 'the', 'roof', 'of', 'a', 'high', ',', 'five-storied', 'house', 'and', 'was', 'more', 'like', 'a', 'cupboard', 'than', 'a', 'room.', 'The', 'landlady', 'who', 'provided', 'him', 'with', 'garret', ',', 'dinners', ',']
>>> text.collocations()
Building collocations list
Katerina Ivanovna; Pyotr Petrovitch; Pulcheria Alexandrovna; Avdotya
Romanovna; Marfa Petrovna; Rodion Romanovitch; Sofya Semyonovna; old
woman; Project Gutenberg-tm; Porfiry Petrovitch; Amalia Ivanovna;
great deal; Project Gutenberg; Andrey Semyonovitch; Nikodim Fomitch;
young man; Dmitri Prokofitch; n't know; Ilya Petrovitch; Good heavens
>>> raw.find("PART I")
5338
>>> raw.rfind("End of Project Gutenberg's Crime")
1157743
>>> raw = raw[5303:1157681]
>>> raw.find("PART I")
35
查看網路上html格式的檔案。
>>> url = "http://news.bbc.co.uk/2/hi/health/2284783.stm"
>>> html = urlopen(url).read()
>>> html[:60]
'<!doctype html public "-//W3C//DTD HTML 4.0 Transitional//EN'
通過 print html來列印得到的檔案。
從HTML 中提取文本是極其常見的任務，NLTK 提供了一個輔助函數nltk.clean_html()將HTML 字串作為參數，返回原始文本。然後我們可以對原始文本進行分詞。
>>> raw = nltk.clern_html(html)
Traceback (most recent call last):
File "<pyshell#38>", line 1, in <module>
raw = nltk.clern_html(html)
AttributeError: 'module' object has no attribute 'clern_html'
>>> raw = nltk.clean_html(html) //消除html標記
>>> tokens = nltk.word_tokenize(raw) //把內容轉換為列表
>>> tokens //顯示出所有內容
>>> tokens =tokens[96:399]
>>> text = nltk.Text(tokens)
>>> text.concordance('gene')
Building index...
Displaying 4 of 4 matches:
hey say too few people now carry the gene for blondes to last beyond the next
have blonde hair , it must have the gene on both sides of the family in the g
ere is a disadvantage of having that gene or by chance. They do n't disappear
des would disappear is if having the gene was a disadvantage and I do not thin
處理搜尋結果：
Start building with 50+ products and up to 12 months usage for Elastic Compute Service
1 on 1 presale consultation
24/7 Technical Support 6 Free Tickets per Quarter Faster Response
Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.
A comprehensive suite of global cloud computing services to power your business
Payment Methods We Support
