Python crawler--Baidu Billboard--pandas Trend chart

Long time no Update blog, recently done a small project, the use of Python crawler features, Daily Record crawl Baidu list of data, and then draw a trend map, so as to predict the tourist city heat.

Originally prepared with pandas directly read table, later found that pandas can not be packaged into EXE, can only give up, tried Petl also failed.

In the process of drawing in the Chinese can not display the problem, the query is due to matplotlib problem, did not introduce Chinese fonts.

Analyze the Code First

ImportOSImportPandas as Pdrootdir='C:/files'#Delete the result.csv first.
#获取目录下面的所有文件路径
defGet_all_filepath (RootDir): Filepath_list=[]     forParent,dirnames,filenamesinchOs.walk (rootdir): forFileNameinchFilenames:filepath_list.append (Parent+'/'+filename)#print "The full name of the file is:" + os.path.join (parent,filename)    returnfilepath_listfilepath_list=Get_all_filepath (RootDir)PrintFILEPATH_LISTDF=pd.read_csv (filepath_list[0]) DFS=[] forFilePathinchFilepath_list[1:len (filepath_list)-1]: #将字符串转化成变量#locals () [' var ']=2345    #VARs () [' var ']=567    Printfilepath dfs.append (pd.read_csv (filepath)) forIteminchDFS:DF=pd.merge (df,item,how='outer', on=' City', suffixes= ('_x','_y'))#df.to_csv (' c:/files/result.csv ', encoding= ' GBK ')#df1=df[df[' city '].isin ([u ' Hong Kong ', U ' Maldives ', U ' qingdao ', U ' Sanya ', U ' Jeju ', U ' Tokyo ')]df1=df[df[' City'].isin (['Hong Kong','Maldives','Qingdao','Sanya','Jeju Island','Tokyo'])]#df.query ("City in (' Hong Kong ', ' Maldives ')")#df1.query ("City Not In (' Hong Kong ', ' Maldives ')")Df2=DF1. Tdf2.columns=DF2.ILOC[0,:]DF2=df2.drop (' City', axis=0). Astype (float) df2[' Day']=pd.date_range ('20160127','20160218')PrintDf2.columns.name#Df2.index.nameDf2.columns.name=u'City'#Df2.index.name can also be assigned a value. Df2.plot (x=' Day', Title=u'Destination City Search trends')#Chinese is not supported. #不支持中文, because of the Chinese font problem, pandas's drawing is based on matplotlib so put Matplotlib's display#Chinese problem solved, natural Chinese will be displayed, see http://www.pythoner.com/200.html

The results are as follows:

Python crawler--Baidu Billboard--pandas Trend chart