Yesterday in the collection of a website, the acquisition program error.
It's a self written collection program.
After searching, find some useless results, later found, with Msxml2.serverxmlhttp replace Msxml2.xmlhttp, the problem successfully resolved.
The following is an analysis of the error-scheduling process.
It always hangs for a while, specifies that the resource download failed, or denied access.
Later, all of the direct denial of access.
At first, think that the other server did the collection settings, such as a time limit.
Then change the program, the original is to get the list page directly, and then continue to loop to get the list of articles. The collection file structure was changed,
The original is a file, class, database connection, data processing all in a file, and then collect an article, the day after tomorrow added a time cycle. Wait 5 seconds. The code is as follows
Copy Code code as follows:
Stime=timer ()
Dtime=timer ()-stime
Do While DTime < 5
Dtime=timer ()-stime
Loop
Can wait for 5 seconds, but soon found that the cycle is too consuming CPU resources, one to this cycle, the CPU occupancy rate lasted 100%, so immediately abandoned the scheme.
Another file was made. Only according to the article URL to crawl the relevant content of the article and write to the database, written after the output automatically jump code.
The core idea is to use the <meta http-equiv= "Refresh" content= "5;url=someurl.asp" >
This is a timed jump code, I believe we are very familiar with.
My principle is to use the first file, crawl all the article connection address, and related navigation code, saved to the database.
Then use the new crawl article content of the program, read the database records, a collection.
There is a flag tag in the database record, the tinyint type, the default is 0, the acquisition is successfully updated to 1, and the failure is updated to 2
This way, each time from the database to read an flag=0 data, that is, read the data, to operate. Operation completed wait 5 seconds to jump to itself.
Thoughts and ideas are very good. Article URL database is also ready to collect, a collection of article specific content, and error. has been the Msxml3.dll error ' 80070005 '
Access denied error.
That depressed, want to open a night machine, automatic collection, burst.
This morning to search for information, most of the queries are useless information.
Basically, it's about permission issues.
Msxml3.dll error ' 80070005 '
Access denied.
Solution:
The Internet Guest Account (Iusr_web) that is assigned to the program's folder is writable.
Practice:
Select Cachefile-Property-Security-Add (iusr_web)-Write
That's what it's all about. Somehow, I'm FAT32. The security permissions problem does not exist in NTFS format, and the IIS settings are normal.
My problem is that it's part of the collection that denies access. This is certainly not the problem.
Continue to search, find another solution says yes, replace microsoft.xmlhttp with msxml2.xmlhttp, I used to be msxml2.xmlhttp.
Continue searching. Finding the final solution should be with msxml2.serverxmlhttp
Previously also found this result, due to the general effect, did not notice in time, the following is the search process to find useful relevant information.
The code is as follows:
Copy Code code as follows:
posturl=http://www.xxx.com/
Server.scripttimeout=20
Set oxmlhttp = CreateObject ("Microsoft.XMLHTTP")
Call Oxmlhttp.open ("Get", PostURL, False)
Call Oxmlhttp.send ("")
Shtmlstr = Oxmlhttp.responsebody
Set oxmlhttp = Nothing
Preliminary decision may be open page has the reason for the jump, the solution is to use
MSXML2. ServerXMLHTTP
Replace
Microsoft.XMLHTTP
can be resolved.
And found this piece of code on the CSDN.
Copy Code code as follows:
Serverurl=posturl
Set Mail1 = Server.CreateObject ("CDO.") Message ")
Mail1.createmhtmlbody serverurl,31
Aa=mail1.htmlbody
Set Mail1 = Nothing
Response.Write AA
Try it, you can use it, copy it, collect it.
Copy Code code as follows:
<%
' Define variables
Dim objXmlHttp
Dim strhtml
' This is a stable version of the Msxml3.0.
' Use Msxml2.serverxmlhttp, do not use Msxml2.xmlhttp
Set objxmlhttp = Server.CreateObject ("Msxml2.serverxmlhttp")
' If you find the following error
' Msxml3.dll error ' 80070005 '
' Access is denied.
' may be the use of the agent when the Internet has made an error
' Use the Proxycfg.exe tool. :
'
' Proxycfg-d
' Check to see if the agent is using the Internet
' also refer to this post: http://www.asp101.com/forum/display_message.asp?mid=51841
'
' This version is too old and unstable.
' Set objxmlhttp = Server.CreateObject (' msxml2.xmlhttp ')
' Now we're starting to send the request.
' In the words of Microsoft: Initialize a request and specify the method of the request (Get,post, etc.)
' URL, and permission authentication information (username, password, etc.).
Format
'. Open (Bstrmethod, bstrURL, Basync, Bstruser, Bstrpassword)
' Objxmlhttp.open ' get ', ' http://www.yahoo.com ', False
Objxmlhttp.open "Get", "http://www.yahoo.com", False
' Send the request in a direct way.
Objxmlhttp.send
' Print return Status:
Response.Write "Status:" & Objxmlhttp.status & "" _
& Objxmlhttp.statustext & "<br/>"
' Gets the text returned by the response.
' The object itself is used to manipulate XML, so it has the following attributes:
' Responsebody, Responsestream, and Responsexml.
' But now we just need text information
strHTML = Objxmlhttp.responsetext
' But when this object instance is no longer in use, you need to clear the object instance.
Set objXmlHttp = Nothing
' Now all we have to do is show the HTML we get.
' First look at the effect of browser interpretation
' and then display its source code
' The following are similar to %>
<table border= "1" bgcolor= "#FFFFFF" >
<tr><td>
<%= strhtml%>
</td></tr>
</table>
<br/>
<table border= "1" bgcolor= "#FFFFFF" >
<tr><td>
<pre>
<%= Server.HTMLEncode (strhtml)%>
</pre>
</td></tr>
</table>
