Thieves & thieves warehouse receiving & collection warehouse receiving

Reference: application reference of MSXML. XMLHTTP

XMLHTTP application reference
1. steps:
1. Create an XMLHTTP object // msxml4.0 is required
2. Open the connection with the server, and define the command sending method, Service webpage (URL), and request permissions. The client opens a connection to the Service webpage of the server through the open command. Like normal HTTP Command Transmission, you can use the "get" method or "Post" method to direct to the Service webpage of the server.
3. Send commands.
4. Wait for and receive the processing results returned by the server.
5. Release the XMLHTTP object

Ii. XMLHTTP method:
1. XMLHTTP object
Note: The client can use the XMLHTTP object to send any HTTP request, accept the HTTP response, and parse the XML document for the response.

Open Method: Initialize an msxml2.xmlhttp request, specifying the HTTP request method, URL, and authentication information.
Open (bstrmethod, bstrurl, varasync, bstruser, bstrpassword)
Bstrmethod: data transmission method, that is, get or post.
Bstrurl: the URL of the Service webpage.
Varasync: whether to run the command synchronously. The default value is true, that is, synchronous execution, but can only be performed in the Dom. Set it to false, that is, asynchronous execution.
Bstruser: user name, which can be omitted.
Bstrpassword: user password, which can be omitted.

Send method: Send an HTTP request to the server and return a response.
Syntax:
Oxmlhttprequest. Send (varbody)
Note: whether the method is synchronized depends on the varasync parameter of the open method. If it is set to true, It is synchronous, and the call is immediately returned. If it is set to false, the call is not returned until the entire response is received.

SetRequestHeader (bstrheader, bstrvalue)
Bstrheader: HTTP Header)
Bstrvalue: HTTP header value

If the open method is defined as post, you can define form upload:
XMLHTTP. setRequestHeader ("Content-Type", "application/X-WWW-form-urlencoded ")

Iii. XMLHTTP attributes:
Onreadystatechange: obtains the event handle of the returned result in synchronous execution mode. It can only be called in the Dom.
Responsebody: returns an unsigned integer array.
Responsestream: The returned result is an istream stream.
Responsetext: returns a string.
Responsexml: The returned result is in XML format.

Iv. Example:

Nowadays, there are many popular thieves on the Internet, such as news thieves, music thieves, and download thieves. How do they do this? I will give a brief introduction below, I hope it will be helpful to all webmasters.
(1) Principles
The thief program actually calls webpages on other websites through the XMLHTTP component in XML. For example, many of the news thief programs call Sina's news pages, replace the HTML and filter advertisements. The advantages of using the thief program are: No need to maintain the website, because the data in the thief program comes from other websites, it will be updated with the updates of the website; can save server resources, generally, a thief program has several files, and all the webpage content comes from other websites. Disadvantages: unstable. If an error occurs on the target website, the program will also go wrong. If the target website is upgraded and maintained, the thief program must be modified accordingly; speed, because it is a remote call, it must be slower than reading data on the local server.
(2) Examples

The following is a brief description of the Application of XMLHTTP in ASP.

Code:

Visual Basic code:

<%
'Common Functions

1. Enter the URL of the target webpage. The returned value gethttppage is the HTML code of the target webpage.
Function gethttppage (URL)
Dim HTTP
Set HTTP = server. Createobject ("msxml2.xmlhttp"
HTTP. Open "get", URL, false
HTTP. Send ()
If HTTP. readystate <> 4 then
Exit Function
End if
Gethttppage = bytestobstr (HTTP. responsebody, "gb2312"
Set HTTP = nothing
If err. Number <> 0 then err. Clear
End Function

'2. If you use XMLHTTP to call a webpage with Chinese characters, you can use the ADODB. Stream component to convert the webpage.
Function bytestobstr (body, cset)
Dim objstream
Set objstream = server. Createobject ("ADODB. Stream"
Objstream. type = 1
Objstream. mode = 3
Objstream. Open
Objstream. Write body
Objstream. Position = 0
Objstream. type = 2
Objstream. charset = cset
Bytestobstr = objstream. readtext
Objstream. Close
Set objstream = nothing
End Function

'Next, try to call the HTML content of http://www.998w.net/class/
Dim URL, html
Url = "http://www.998w.net/class"
Html = gethttppage (URL)
Response. Write html
%>

------------------------------------------------------
Code:

Visual Basic code:

'Code] Use XMLHTTP to read remote files
<%
Response. Buffer = true
Dim objxmlhttp, XML
Set xml = server. Createobject ("Microsoft. XMLHTTP"

xml.Open "GET", "http://www.998w.net/down/998w1.0.rar", False

xml.Send

' Add a header to give it a file name:
Response.AddHeader "Content-Disposition", _
"attachment;filename=mitchell-pres.zip"

' Specify the content type to tell the browser what to do:
Response.ContentType = "application/zip"

' Binarywrite the bytes to the browser
Response.BinaryWrite xml.responseBody

Set xml = Nothing
%>

-------------------------------------
How to Write ASP database theft program
The principle of warehouse receiving thieves is also very simple: XMLHTTP is used to remotely read the content of a webpage, and then process (filter, replace, and classify) the content as needed ), finally, get the data you need and add it to the database.
First, read the remote Webpage Through XMLHTTP (introduced in another article ).
Secondly, it is a key step to filter the content. For example, what should I do if I want to extract all URL connections from a remote webpage?
Code:

Visual Basic code:

'The regular expression is used here.
Set objregexp = new Regexp 'create an object
Objregexp. ignorecase = true 'case insensitive
Objregexp. Global = true' global true
Objregexp. pattern = "http: //. +? "'Matching Field
Set Mm = objregexp. Execute (STR) 'to perform the search. STR is the input parameter.
For each match in mm' enters the loop
Response. Write (match. Value) 'output URL address
Next

Then, we need to make some replacement functions as needed to replace unnecessary data. This is relatively simple, just use the replace function.
Finally, perform database operations
-------------------------------
Example
Code:

Visual Basic code:

<%
On Error Resume Next
Server.ScriptTimeOut=9999999
Function getHTTPPage(Path)
t = GetBody(Path)
getHTTPPage=BytesToBstr(t,"GB2312"
End Function

'First, perform initialization settings for the thief program. The above code is used to ignore all non-fatal errors, set the time-out period of the thief program to a long time (so there will be no running time-out error), convert the original default UTF-8 encoding to gb2312 encoding, otherwise, it is garbled to call a webpage with Chinese characters directly using the XMLHTTP component.

Function getbody (URL)
On Error resume next
Set retrieval = Createobject ("Microsoft. XMLHTTP"
With Retrieval
. Open "get", URL, false ,"",""
. Send
Getbody =. responsebody
End
Set retrieval = nothing
End Function

'Then call the XMLHTTP component to create an object and perform initialization settings.

Function bytestobstr (body, cset)
Dim objstream
Set objstream = server. Createobject ("ADODB. Stream"
Objstream. type = 1
Objstream. mode = 3
Objstream. Open
Objstream. Write body
Objstream. Position = 0
Objstream. type = 2
Objstream. charset = cset
Bytestobstr = objstream. readtext
Objstream. Close
Set objstream = nothing
End Function

Function Newstring(wstr,strng)
Newstring=Instr(LCase(wstr),LCase(strng))
If Newstring<=0 Then Newstring=Len(wstr)
End Function

'To process the captured data, you need to call the ADODB. Stream component and perform initialization settings. %>

'Below is the page Display Section

<%
Dim wstr, STR, URL, start, over, City
'Defines the variables to be used.

City = request. querystring ("ID ")
'The ID variable returned by the Program (that is, the selected city) is assigned to the ID

Url = "http://appnews.qq.com/cgi-bin/news_qq_search? City = "& City &""
'Set the page address to be crawled here. Of course, you can also directly specify an address without using variables.

Wstr = gethttppage (URL)
'Retrieve all data on the specified page

Start = newstring (wstr, "<HTML> ")
'Set the header of the data to be processed. This variable should be set according to different situations. The specific content can be determined by viewing the source code of the page to be crawled. Because we need to capture the whole page in this program, we set it to capture all the pages. Note: The set content must be unique and cannot be repeated.

Over = newstring (wstr, "'Corresponds to the end of the data to be processed. Similarly, the set content must be unique on the page.

Body = mid (wstr, start, over-start)
'Set the display page range

'The next is the time to use the Qiankun big shift method. With replace, you can use some characters to replace the specified characters in the data.

Body = Replace (body, "skin1", "Weather Forecast-SK network ")
Body = Replace (body, "http://appnews.qq.com/cgi-bin/news_qq_search? City "," Tianqi. asp? ID ")

'The replacement has been completed in this program. If there are other requirements, you can continue with similar replacement operations.

Response. Write body
%>

**************************************** **************************************** ***************************
Reference: remotely obtain the content and store it on a local computer, including any files.

Visual Basic code:

<%
'---------- Obtain the content remotely and store the content on the local computer, including any files! ----------
'On error resume next
'Set the content type to the specific type that you are sending.
'Response. contenttype = "image/JPEG"
'------------------------------- Define the output format -----------------------------

Path = request. querystring ("p ")
Spath = path
If left (lcase (PATH), 7) <> "http: //"; then
'------------- If there is no HTTP before, it is a local file. submit it to localfile for processing ------------
Localfile (PATH)
Else
'------------------ Otherwise, the remote file will be handed to remotefile for processing ------------------
Remotefile (PATH)
End if
'Response. Write err. Description

Sub localfile (PATH)
'------------------- If the file is a local file, you can simply jump to this page -------------------
Response. Redirect path
End sub

Sub remotefile (Spath)
'------------------------- Remote File Processing Function ------------------------------
Filename = getfilename (Spath)
'------------- Getfilename is the process of converting an address to a qualified file name -------------
Filename = server. mappath ("/uploadfile/Cache/" & filename)
Set objfso = server. Createobject ("scripting. FileSystemObject ")
'Response. Write filename
If objfso. fileexists (filename) then
'-------------- Check whether the file has been accessed. If yes, simply jump ------------
Response. Redirect "/uploadfile/Cache/" & getfilename (PATH)
Else
'---------------- Otherwise, use the getbody function to read ----------------------
'Response. Write path
T = getbody (PATH)
'----------------- Write data in binary format to the browser --------------------------
Response. binarywrite t
Response. Flush
'----------------- Output buffer ------------------------------------------
SaveFile T, getfilename (PATH)
'---------------- Cache the file content to the local path for the next visit -----------
End if
Set objfso = nothing
End sub

Function getbody (URL)
'----------------------- This function is a function for remotely obtaining content ---------------------
'On error resume next
'Response. Write URL
Set retrieval = Createobject ("Microsoft. XMLHTTP ")
'---------------------- Create an XMLHTTP object -----------------------------
With Retrieval
. Open "get", URL, false ,"",""
'---------------- Use the get, Asynchronous Method to send -----------------------
. Send
'Getbody =. responsetext
Getbody =. responsebody
'---------------- The retrieved content returned by the function --------------------------
End
Set retrieval = nothing
'Response. Write err. Description
End Function

Function getfilename (STR)
'------------------------- This function is a qualified file name function -------------------
STR = Replace (lcase (STR), "http ://";,"")
STR = Replace (lcase (STR ),"//","/")
STR = Replace (STR ,"/","")
STR = Replace (STR, vbcrlf ,"")
Getfilename = Str
End Function

Sub SaveFile (STR, fname)
'------------------------- This function is the function of saving stream content to disk -------------------
'On error resume next
Set objstream = server. Createobject ("ADODB. Stream ")
'-------------- Create an ADODB. Stream object. It must be of the ADO 2.5 or later version ---------
Objstream. type = adtypebinary
'------------- Open in binary mode -------------------------------------
Objstream. Open
Objstream. Write Str
'------------------ Write string content to buffer --------------------------
'Response. Write fname
Objstream. savetofile "C: \ Inetpub \ myweb \ uploadfile \ cache \" & fname, adsavecreateoverwrite
'------------------ Write the buffered content to the file --------------------------
'Response. binarywrite objstream. Read
Objstream. Close ()
Set objstream = nothing
'----------------------- Close the object and release the resource -------------------------
'Response. Write err. Description
End sub
%>