When extracting Web page content with XPath in learning scrapy, it is sometimes necessary to extract the entire row label content, and then look for the target content from the row label. An error has occurred.
Error code:
def Parse (self, Response): = Scrapy. Selector (response) = Sel.xpath ('/html/body/div[2]/div[5]/div[1]/ul/li') for inch Sel_li: Print (I.xpath ('//h5/a/text ()'). Extract () [0])
Results:
Big mouth second quarter big mouth big mouth second quarter big mouth second quarter Big mouth second season Big mouth second quarter
...
The extracted content is the next content in the first one.
Post-Revision code:
def Parse (self, Response): = Scrapy. Selector (response) = Sel.xpath ('/html/body/div[2]/div[5]/div[1]/ul/li') for inch Sel_li: Print (I.xpath ('.//h5/a/text ()'). Extract () [0])
Results:
Big Mouth second quarter new hundred Days Dragon third quarter Danish kinky first quarter bad history seventh quarter shameless nineth quarter ...
In the child XPath, before the path is added . Indicates from the current lookup, after which subsequent content is normally obtained
Other methods:
def parse (self, Response): Sel = Scrapy. Selector (response) Sel_li = Sel.xpath ( " / html/body/div[2]/div[5]/div[1]/ul/li " ) for i in Sel_li.extract (): print (Scrapy. Selector (text=i). XPath ( ' //h5/a/text () " Span style= "COLOR: #800000" > " ). Extract () [0]) print (scrapy. Selector (text=i). XPath ( ' .//h5/a/text () "). Extract () [0])
Scrapy XPath selector multilevel selection error