PHP Curl Crawl Ajax Asynchronous content sample

Source: Internet
Author: User
Tags curl http request

Actually grasping Ajax asynchronous content page and grasping ordinary page difference is not big. Ajax is simply an asynchronous HTTP request that uses a firebug-like tool to find the requested backend service URL and the parameters of the pass value, and then crawl the URL pass parameter.

Using Firebug's Network Tools

If the page is captured, the content is not displayed in the data, is a bunch of JS code.


01$cookie_file=tempnam ('./temp ', ' Cookie ');

02$ch = Curl_init ();

03$URL1 = "";

04curl_setopt ($ch, Curlopt_url, $url 1);

05curl_setopt ($ch, curlopt_http_version,curl_http_version_1_1);

06curl_setopt ($ch, curlopt_header,0);

07curl_setopt ($ch, curlopt_returntransfer,1);

08curl_setopt ($ch, curlopt_followlocation,1);

09curl_setopt ($ch, curlopt_encoding, ' gzip '); Add gzip resolution

10//set up a file that stores cookie information after the connection ends

11curl_setopt ($ch, Curlopt_cookiejar, $cookie _file);

12$content=curl_exec ($ch);


14curl_close ($ch);


16$ch3 = Curl_init ();

17$URL3 = "Http://";

18$curlpost = "callcount=1&page=/xww/type/1000020118.html&httpsessionid=

12a9b726e6a2d4d3b09de7952b2f282c&scriptsessionid=295315b4b4141b09da888d3a3adb8faa658&c0-scriptname= Portalajax&c0-methodname=getnewsxml&c0-id=0&c0-param0=string:10000201&c0-param1=string : 1000020118&c0-param2=string:news_&c0-param3=number:5969&c0-param4=number:1&c0-param5=null: Null&c0-param6=null:null&batchid=0 ";

19curl_setopt ($ch 3,curlopt_url, $url 3);

20curl_setopt ($ch 3,curlopt_post,1);

21curl_setopt ($ch 3,curlopt_postfields, $curlPost);


23//set up a file that stores cookie information after the connection ends

24curl_setopt ($ch 3,curlopt_cookiefile, $cookie _file);

25$content1=curl_exec ($ch 3);

26curl_close ($ch 3);

Related Article

E-Commerce Solutions

Leverage the same tools powering the Alibaba Ecosystem

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.