How to identify the code of Python web crawler

Some websites are signed in by the way the verification code, such as today we want to test the site patent search and analysis.

Http://www.pss-system.gov.cn/sipopublicsearch/portal/uilogin-forwardLogin.shtml

The key to logging on to such a site is to identify the verification code. So how to identify the verification code. Let's first look at the source code of the Web page. In the Web page, the verification code is obtained by downloading an image. The picture is src=/sipopublicsearch/portal/login-showpic.shtml

We see from the actual fiddler capture, but also by requesting the above image source address to get a JPEG picture and display in the browser 

So in scrapy we'll first download the image locally and then identify it.

  def   parse (self ,response): 
     ret=response.xpath (  '//*[@id = " Codepic "]/@src '   ). Extract () 
     image_source=ret[ 0] 
     image_url=response.urljoin (image_source) 
     r=requests.get (image_url) 
      with   open (  "E://scrapy_project/image2. JPEG '      )  as   code: 
         code.write (r.content)     

First extract the SRC value out, then use the requests method to download and save the picture. Open the file as follows. 

The next step is to start identifying the verification code in the image, which requires the Pytesser and pil libraries. 

The first is to install TESSERACT-OCR and install it after downloading it online. The default installation path is C:\Program FILES\TESSERACT-OCR. Add the path to the system property's path path. 

Then install pytesseract and PIL via Pip . Let's see how it's used. The code is as follows:  

Im=image.open (' E:\\scrapy_project\\image2. JPEG ')
Im.convert (' L ')
Ret=image_to_string (IM,config='-PSM 7 ')
Ret

The results are as follows: The verification code in the picture has been identified.

Image_to_string to configure PSM N, the parameters are explained as follows, generally we choose the first 7 

-PSM N

For N is:

0 = Orientation and script detection (OSD) only.

With OSD.

2 = Automatic page segmentation, but no OSD, or OCR.

3 = Fully Automatic page segmentation, but no OSD. (Default)

of variable sizes.

of vertically aligned text.

of text.

As a single text line.

As a single word.

In a circle.

As a single character.

E:\python2.7.11\python.exe e:/py_prj/test3.py

8227

How to identify the code of Python web crawler