Python 2.7_second_try_ Crawl Sunshine Movie Net _ get movie download address and write to file 20161207

1, yesterday article http://www.cnblogs.com/Mr-Cxy/p/6139705.html is to get the movie site main menu and then get the movie URL under each menu

2, today is the movie URL to parse again gets and writes the file call function and the class multithreading has not been implemented step by step

3, the question: I want to achieve is the first to create a file directory on the menu and then each directory to get the movie name. txt as a file file is a download connection, But creating a one-level menu folder is no problem. You can create an error when writing a movie name. txt with the OS module

I thought it was coding. F.open (directory name + movie name. txt, "a") preceded by the character type also used to decode= ' utf-8 ' conversion or error

Helpless I had to write a fixed text TXT file in append mode and see it tomorrow.

4, Python code  

#coding: Utf-8import requestsfrom bs4 import beautifulsoup as bs# crawl entry rooturl= "http://www.ygdy8.com/index.html" # Get Web source Res=requests.get (rooturl) #网站编码gb2312res. encoding= ' gb2312 ' #网页源码html =res.textsoup=bs (HTML, ' Html.parser ') Cate_urls = []for cateurl in Soup.select ('. Contain ul li a '): #网站分类标题 cate_name=cateurl.text.encode (' utf-8 ') #分类u RL crawls again Cate_url= "http://www.ygdy8.com/" + cateurl[' href '] cate_urls.append (cate_url) print "Site level menu:", Cate_name, "Menu URL:", Cate_url # newdir = "e:/moive24/" + cate_name # os.makedirs (Newdir.decode ("Utf-8")) # print "Create catalog successfully-----    -"+ newdir# per menu URL Resolution for I in range (len (cate_urls)): cate_listurl=cate_urls[i] res = requests.get (cate_listurl)    res.encoding = ' gb2312 ' html = res.text soup = bs (HTML, ' Html.parser ') print "parsing section" +str (i+1) + "links", Cate_urls[i]        Contenturls=[] Contents=soup.select ('. Co_content8 ul ') [0].select (' a ') #print contents for title in contents:       Moivetitle=title.text.encode (' Utf-8 ') Moiveurl= "http://www.ygdy8.com/" + title[' href '] contenturls.append (moiveurl) Print Moivetitle,moiveurl         # File_name=newdir + '/' + moivetitle + '. txt ' # print file_name # f = open (File_name.decode ("Utf-8"), "WB") # f.close () res = Requests.get (moiveurl) res.encoding = ' gb2312 ' html = res.text soup             = BS (HTML, ' Html.parser ') moive_sources=soup.select (' #Zoom span tbody tr TD a ') for source in Moive_sources:            moive_source=source[' href '] #print moive_source f=open (' e:/moive24/moive.txt ', ' a ') F.write (Moive_source.encode ("utf-8") + "\ n") f.close

　　

Python 2.7_second_try_ Crawl Sunshine Movie Net _ get movie and write file 20161207