firewall url filtering

[Crawler learning notes] Url filtering module UrlFilter, learning notes urlfilter Url Filter filters the extracted URLs again. The criteria for filtering different applications are different. For example, for baidu/google search, the criteria are generally not filtered, but for vertical search or targeted crawling a

The following is the definition of a class.Copy codeThe Code is as follows:Using System;Using System. Web;Using System. Web. SessionState;Namespace QTJZ{Public class Filters: IHttpModule, IRequiresSessionState{Public void Dispose (){}Public void Init (HttpApplication application){Application. AcquireRequestState + = new EventHandler (application_AcquireRequestState );}Public void application_AcquireRequestState (object sender, EventArgs e){HttpApplication application = sender as HttpApplication;

Search keywords include spaces, after submission is automatically turned into a "+" number, reported the following error:HTTP error 404.11-not the found request filtering module is configured to reject requests that contain a double escape sequence.Workaround:Add the following configuration node to the The URL contains the + sign, and the error "The Request filtering