Today curl
, in a Web site encountered a strange problem, the following is the output:
[Email protected]:~$ curl-l http://www.yngs.gov.cn/-V
* Hostname was wasn't found in DNS cache
* Trying 116.52.12.163 ...
* Connected to Www.yngs.gov.cn (116.52.12.163) port (#0)
get/http/1.1
user-agent:curl/7.38.0
Host:www.yngs.gov.cn
Accept: /
< http/1.1 302 Moved temporarily
< date:wed, Geneva 14:08:49 GMT
< transfer-encoding:chunked
< location:http://www.yngs.gov.cn/newweb/template/index.jsp
< content-type:text/html; Charset=utf-8
< set-cookie:jsessionid=slytw6rr3r7zpnkkvzvpj12q1snzzvnfqjypdbdhybvgtxwhsnff!-995202664; path=/; HttpOnly
< x-powered-by:*********
< set-cookie:sangfor_ad=20111157; path=/
<
* Ignoring the Response-body
* Connection #0 to host www.yngs.gov.cn left intact
* Issue Another request to this URL: ' http://www.yngs.gov.cn/newWeb/template/index.jsp '
* Found Bundle for Host WWW.YNGS.GOV.CN:0XB89840C0
* Re-using Existing connection! (#0) with host www.yngs.gov.cn
* Connected to Www.yngs.gov.cn (116.52.12.163) port (#0)
get/newweb/template/index.jsp http/1.1
user-agent:curl/7.38.0
Host:www.yngs.gov.cn
Accept:/
....//The output above is always repeated
* Ignoring the Response-body
* Connection #0 to host www.yngs.gov.cn left intact
* Maximum (redirects) followed
Curl: (Maximum) redirects followed
The last error shows that the curl
maximum 50 jumps are exceeded.
From the above output to see when accessing http://www.yngs.gov.cn/return 302 jump, the URL of the jump is http://www.yngs.gov.cn/newWeb/template/index.jsp, but then access the HTTP ://www.yngs.gov.cn/newweb/template/index.jsp return the same 302 jump, the address after the jump is the target itself, so it will always be http://www.yngs.gov.cn/newWeb /template/index.jsp this URL to jump, when the curl
default maximum number of jumps over the set 50 after the abnormal end.
If curl
there is a problem then try the wget
command to see if this command will also encounter the same error result:
[Email protected]:~$ wget http://www.yngs.gov.cn/–debug
-request begin-
get/http/1.1
user-agent:wget/1.16.1 (LINUX-GNU)
Accept: /
Accept-encoding:identity
Host:www.yngs.gov.cn
Connection:keep-alive
-request end-
-response begin-
http/1.1 302 Moved temporarily
date:wed, 14:18:51 GMT
Transfer-encoding:chunked
location:http://www.yngs.gov.cn/newweb/template/index.jsp
content-type:text/html; Charset=utf-8
set-cookie:jsessionid=7jjtw6tlpkrf0vynxtrpqrnzffkgdfb0vh6vdzq9jhgnvrsmzxyv!-1122044597; path=/; HttpOnly
X-powered-by: *********
set-cookie:sangfor_ad=20111151; path=/
-response end-
302 Moved temporarily
Stored Cookie www.yngs.gov.cn-1 (any)/<session> <insecure> [expiry none] Jsessionid 7JJTW6TLPKRF0VYNXTRPQR nzffkgdfb0vh6vdzq9jhgnvrsmzxyv!-1122044597
Stored Cookie www.yngs.gov.cn-1 (any)/<session> <insecure> [expiry none] Sangfor_ad 20111151
Registered Socket 3 for persistent reuse.
URI content encoding = "UTF-8"
Location: http://www.yngs.gov.cn/newWeb/template/index.jsp [follow to new URL]
URI content encoding = None
–2015-11-04 22:23:09–http://www.yngs.gov.cn/newweb/template/index.jsp
Use the existing connection to the www.yngs.gov.cn:80 again.
Reusing FD 3.
-request begin-
get/newweb/template/index.jsp http/1.1
user-agent:wget/1.16.1 (LINUX-GNU)
Accept: /
Accept-encoding:identity
Host:www.yngs.gov.cn
Connection:keep-alive
cookie:jsessionid=7jjtw6tlpkrf0vynxtrpqrnzffkgdfb0vh6vdzq9jhgnvrsmzxyv!-1122044597; sangfor_ad=20111151
-request end-
-response begin-
http/1.1 OK
date:wed, Geneva 14:18:51 GMT
transfer-encoding:chunked
Con tent-type:text/html; Charset=utf-8
x-powered-by: *********
-response end-
OK
Above is the result of wget
(some extra output removed), we see wget
can get the result of http://www.yngs.gov.cn/normally, and does not appear Curl
encountered an issue with the dead loop jump. So that means the site itself is not a problem, but we have access to the time may be missing some parameters.
Then I compared the request ,
response information for Curl
and wget
, and I found both on the first request http:/ /www.yngs.gov.cn/ Request
, response
are all the same, the difference may be user-agent
. But when requesting 302 again to return the redirect URL http://www.yngs.gov.cn/newWeb/template/index.jsp, the request
There are some different parameters in the wget
request that the first response is returned to the cookie, but Curl
is , ignoring the data returned by the first response, and the second request did not bring the cookie that was returned on the first request.
At this time basically can be judged because curl
access by default when the response
returned data, redirect URL is not set when the cookie is caused, then how to verify it?
The first method is to disable wget
to see if the content can be retrieved normally:
[Email protected]:~$ wget http://www.yngs.gov.cn/–debug–no-cookies
Setting–cookies (cookies) to 0
-request begin-
get/http/1.1
user-agent:wget/1.16.1 (LINUX-GNU)
Accept: /
Accept-encoding:identity
Host:www.yngs.gov.cn
Connection:keep-alive
-request end-
-response begin-
http/1.1 302 Moved temporarily
date:wed, 14:43:41 GMT
Transfer-encoding:chunked
location:http://www.yngs.gov.cn/newweb/template/index.jsp
content-type:text/html; Charset=utf-8
set-cookie:jsessionid=sdltw6zdvqwppqgr5mbf2n1txchnlysvtn8lhdbtqpyp3kvddr0r!-170174379; path=/; HttpOnly
X-powered-by: *********
set-cookie:sangfor_ad=20111158; path=/
-response end-
–2015-11-04 22:46:33–http://www.yngs.gov.cn/newweb/template/index.jsp
Use the existing connection to the www.yngs.gov.cn:80 again.
Reusing FD 3.
-request begin-
get/newweb/template/index.jsp http/1.1
user-agent:wget/1.16.1 (LINUX-GNU)
Accept: /
Accept-encoding:identity
Host:www.yngs.gov.cn
Connection:keep-alive
-request end-
-response begin-
http/1.1 302 Moved temporarily
date:wed, 14:42:16 GMT
Transfer-encoding:chunked
location:http://www.yngs.gov.cn/newweb/template/index.jsp
content-type:text/html; Charset=utf-8
set-cookie:jsessionid=yqjtw6zlpwhm7pfr3lzl6lkqdq1xbnbmchqhjn7vz2yptmjvsjvw!-1122044597; path=/; HttpOnly
X-powered-by: *********
set-cookie:sangfor_ad=20111151; path=/
-response end-
................
URI content encoding = None
More than 20 redirects have been redirected.
We see that wget
the last is also to try 20 jump failures to end.
The second method is curl
to open the cookie:
[Email protected]:~$ curl-l-b/tmp/curl.cookies http://www.yngs.gov.cn/
* Hostname was wasn't found in DNS cache
* Trying 116.52.12.163 ...
* Connected to Www.yngs.gov.cn (116.52.12.163) port (#0)
get/http/1.1
user-agent:curl/7.38.0
Host:www.yngs.gov.cn
Accept: /
< http/1.1 302 Moved temporarily
< date:wed, Geneva 14:55:53 GMT
< transfer-encoding:chunked
< location:http://www.yngs.gov.cn/newweb/template/index.jsp
< content-type:text/html; Charset=utf-8
* Added Cookie jsessionid= "lswqw6czzrtvygkkjm0hl8rscht98bcc3yd4f4v1rcjvllwb2zmj!-1122044597" for domain www.yngs.gov.cn, path/, expire 0
< set-cookie:jsessionid=lswqw6czzrtvygkkjm0hl8rscht98bcc3yd4f4v1rcjvllwb2zmj!-1122044597; path=/; HttpOnly
< x-powered-by:*********
* Added Cookie sangfor_ad= "20111151" for domain www.yngs.gov.cn, path/, expire 0
< set-cookie:sangfor_ad=20111151; path=/
<
* Ignoring the Response-body
* Connection #0 to host www.yngs.gov.cn left intact
* Issue Another request to this URL: ' http://www.yngs.gov.cn/newWeb/template/index.jsp '
* Found Bundle for Host www.yngs.gov.cn:0xb8b74108
* Re-using Existing connection! (#0) with host www.yngs.gov.cn
* Connected to Www.yngs.gov.cn (116.52.12.163) port (#0)
get/newweb/template/index.jsp http/1.1
user-agent:curl/7.38.0
Host:www.yngs.gov.cn
Accept:/
cookie:jsessionid=lswqw6czzrtvygkkjm0hl8rscht98bcc3yd4f4v1rcjvllwb2zmj!-1122044597; sangfor_ad=20111151
< http/1.1 OK
< date:wed, Geneva 14:55:53 GMT
< transfer-encoding:chunked
< content-type:text/html; Charset=utf-8
< x-powered-by: *********
<
<! DOCTYPE html>
....................
Connection #0 to host www.yngs.gov.cn left intact
curl
The results were successfully obtained.
The previous conjecture was also verified from the above results.
Curl Redirection issues