PHP Programmers use crawler technology to uncover real data behind rising rents

In the near future, I believe that everyone in the microblog or in the circle of friends have been the rent soared and I love my family vice president in the Circle of friends to announce the resignation of resignation, net exposure chain home freely bid up prices and so on related events brush screen. For the rise in rents, the most affected are young people. Most of the young people have just graduated without a deposit, or a few years of work because of high prices and continue to rent, and now even renting has become a big difficulty. So as a PHP programmer, here is a description of how to write a crawler in PHP to get real rental data.

Here for the Beijing rental market, if you want to rent the main there are three ways: 1, looking for housing intermediary companies are currently the highest market share of the company, called Chain Home, 2, long-term rental apartment at the current market share the highest, called free; 3, on the search-room platform at present the highest market share, the first is the home. And this April, there is a new line of the company, the emergence of a rapid rise to the top five, called Shell to find a house, these three ways to add up almost determine the price of my rental and more surprising is the above several companies, in addition to the home of the chain, freely, Shells find their real controllers are the same person. This is the day of the frequent presence in the news of the chain group boss Zohui.

For those who are ready to fight in Beijing, the rent boom is a pretty angry thing. Some netizens on the way of using programmers to pick up the rent rise behind the matter. So what is the programmer's way?

One of the procedural ideas is: Write a crawler in PHP with it crawled a chain of data. First go to the console to see the loading information, find the relevant data API, according to the request header inside the required parameters, send HTTPS request, after the analysis is completed using XPath or regular expression tool to match the content you want, and then inserted into the database, you can complete the crawl. Finally, the PHP implementation crawler crawled to the chain home on the internet to rent all the houses.

Then continue to crawl in the same way from the long-rent apartment platform such as, eggshell, mushroom Apartments The final data word cloud is like this

According to the data, in Beijing rental industry in several major directions left the boss of the industry either occupy a leading position or are growing rapidly this is no wonder the previous few days there is a heavyweight news said the original I love my family Vice President Hu Jinghui because of some pressure to resign and shelling freely, eggshell and other long-term rental apartments to high market price 20%- 40% of the price of the property to expand completely regardless of cost.

Businessmen to profit is understandable and the pursuit of greater market share is also a normal business goal is only when an enterprise is too strong to form a monopoly or oligopoly and once a monopoly they can use resources and capital advantages to hoarding, Around or even manipulate the direction of the industry and the housing industry in Beijing such a monopoly seems to be forming. The main point here is to tell you that PHP crawler can get web pages, pictures, scripts, file data and so on network resources.