Python爬蟲入門遇到的坑

標籤：arm   family   爬蟲   sel   library   mac os   install   traceback   bsp   

1. 環境 

- Python
　　mac os預裝的python 

$ python -V  Python 2.7.10$ where python/usr/bin/python$ ls /System/Library/Frameworks/Python.framework/Versions2.3     2.5     2.6     2.7     Current$ ls /Library/Frameworks/Python.framework/Versions （使用者安裝的目錄）

- IDE
　　Pycharm
- 輔助
　　安裝pip

sudo easy_install pip

- Python庫

sudo pip install requests （預設安裝requests 2.13.0） 
sudo pip install BeautifulSoup （預設安裝BeautifulSoup 3.2.1）
sudo pip install lxml （預設安裝lxml 3.7.3)

 

2. 問題

- 問題1

代碼：
soup = BeautifulSoup(html, ‘lxml‘)
報錯：
Traceback (most recent call last):
File "/Users/cuizhenyu/Documents/Codes/Python/DownloadMeitu/LibBeautifulSoupTest.py", line 15, in <module>
soup = BeautifulSoup(html) #soup = BeautifulSoup(html, ‘lxml‘)報錯
TypeError: ‘module‘ object is not callable
解決：
from BeautifulSoup import BeautifulSoup

- 問題2

代碼：
soup = BeautifulSoup(html, ‘lxml‘)
報錯：
Traceback (most recent call last):
File "/Users/cuizhenyu/Documents/Codes/Python/DownloadMeitu/LibBeautifulSoupTest.py", line 15, in <module>
soup = BeautifulSoup(html, ‘lxml‘) #soup = BeautifulSoup(html, ‘lxml‘)報錯
File "/Library/Python/2.7/site-packages/BeautifulSoup.py", line 1522, in __init__
BeautifulStoneSoup.__init__(self, *args, **kwargs)
File "/Library/Python/2.7/site-packages/BeautifulSoup.py", line 1147, in __init__
self._feed(isHTML=isHTML)
File "/Library/Python/2.7/site-packages/BeautifulSoup.py", line 1189, in _feed
SGMLParser.feed(self, markup)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sgmllib.py", line 104, in feed
self.goahead(0)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sgmllib.py", line 138, in goahead
k = self.parse_starttag(i)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sgmllib.py", line 296, in parse_starttag
self.finish_starttag(tag, attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sgmllib.py", line 338, in finish_starttag
self.unknown_starttag(tag, attrs)
File "/Library/Python/2.7/site-packages/BeautifulSoup.py", line 1338, in unknown_starttag
self.endData()
File "/Library/Python/2.7/site-packages/BeautifulSoup.py", line 1251, in endData
(not self.parseOnlyThese.text or \
AttributeError: ‘str‘ object has no attribute ‘text‘
解決：
當前BeautifulSoup是v3版，不支援lxml等，需用v4版。

 

Python爬蟲入門遇到的坑