Get the website via post on the locomotive

Since version 2009, the train collector has enhanced the Post website collection function, which is a good news. We can submit custom data and obtain the desired results. For example, we use search to query a keyword and return some results. We will collect the returned content URLs and then collect them.

We will collect the 51job website provided by the program to describe how to use this function. when we get the relevant positions, we set the search, and then submit it to get what we need. now, we use the packet capture tool to capture the data submitted by the program during the submission. The publishing is to submit the data to this web page.

At this time, we need to set the submitted URL to set, the mining depth to 1, and then fill in the submitted data. We can find that the data submitted for the first time is like this.

Postchannel = 0000 & stype = 2 & jobarea = 0000 & District = & funtype_big = 0000 & funtype = 0000 & industrytype = 00 & issuedate = 9 & keywordtype = 2 & dis_keyword = % BC % c6 % CB % E3 % BB % fa & keyword = % BC % C6 % CB % E3 % BB % fa & workyear = 99 & providesalary = 99 & cotype = 99 & degreefrom = 99 & jobterm = 99 & ord_field = 0 & list_type = 1 & last_list_type = 1 & curr_page = 1

The last item on the second page is 2. In this way, you only need to change the last parameter at the time of submission to obtain the URL. You can specify the page size range when entering this parameter.

Next we will introduce a feature that some friends will use. look at the bottom right corner of the big picture and get a random value. this function is used to process such a page. For example, if you post a parameter to a page to obtain a URL list page, when you get the second page, you need to pass some values on the first list page. This random value is used to get some parameters of the previous page. the following is an example. looking at the picture, interested friends study it on their own.

I tested several pages and looked at the results.

Download rules