How can we determine whether a search engine is used?

<%
'Check whether the current user is a spider.
Function check (user_agent)
Allow_agent = Split ("baiduspider, Scooter, ia_archiver, googlebot, fast-webcrawler, msnbot, slurp ",",")
Check_agent = false
For agenti = lbound (allow_agent) to ubound (allow_agent)
If instr (user_agent, allow_agent (agenti)> 0 then
Check_agent = true
Exit
End if
Next
Check = check_agent
End Function
User_agent = request. servervariables ("http_user_agent ")
'Check (user_agent) = true indicates that the access is a spider.
%>

The above Code has not been tested. Try it... I don't know if it works...

 

 

<%
Sub robot ()
Dim robots: robots = "baiduspider + @ Baidu | googlebot @ Google | ia_archiver @ Alexa | iaarchiver @ Alexa | asw.ek @ asw.ek | robot @ yahoo | sogoubot @ sogou | Baidu | Sohu-search @ Sohu | msnbot @ MSN"
Dim I1, I2, L1, L2, L3, I, RS
L2 = false
L1 = request. servervariables ("http_user_agent ")
F1 = request. servervariables ("script_name ")
I1 = Split (robots, CHR (124 ))
For I = 0 to ubound (I1)
I2 = Split (I1 (I ),"@")
If instr (lcase (L1), lcase (I2 (0)> 0 then
L2 = true: l3 = I2 (1): exit
End if
Next
If L2 and Len (L3)> 0 then', if it is a crawler, The crawler information is updated.
Filepath = server. mappath ("Robots/" & L3 & "_robots.txt ")
'Record spider Crawlers
Set FSO = server. Createobject ("scripting. FileSystemObject ")
Set fout = FSO. opentextfile (filepath, 8, true)
Fout. writeline "index page:" & F1
Fout. writeline "Spider:" & L3 & CHR (32) & CHR (32) & "last updated:" & now ()
Fout. writeline "-----------------------------------------------"
Fout. Close
Set fout = nothing
Set FSO = nothing
End if
End sub
%>

Create a robots folder in your website, put robots. asp under the robots folder, and then put the called function in the common function file. Generally, databases are used for websites, so you only need to write the call robot () into the database.
The following is a reference for my database file on the small site:

<! -- # Include file = "Robots/robots. asp" -->
<%
Set conn = server. Createobject ("ADODB. Connection ")
Connstr = "DBQ =" + server. mappath ("Data/GATA. mdb") + "; defaultdir =; driver = {Microsoft Access Driver (*. mdb )}"
Conn. Open connstr

Call robot ()

%>

Another note: A spider crawls your site, but it may not be included