Java HttpClient + Jsoup is no longer afraid of fire

I don't know how long ago I had written a small software for automatic reply, but I 've been idle for a while recently. I can only ask google for advice on the HTTP protocol, after talking about my thoughts to the google gods, the google gods said that this guy is good. This is a contribution to the fire prevention industry! HttpClient 4.3.1 (GA) lists the main functions provided by HttpClient. For more detailed functions, see the HttpClient homepage. All HTTP methods (GET, POST, PUT, HEAD, etc.) Support automatic redirection support HTTPS support Proxy Server 2. Main Functions of Jsoupjsoup are as follows, parse HTML in a file or string and use DOM or CSS selector to find and retrieve data. HTML elements, attributes, and text can be operated with almost the same syntax as jquery, include the example folder in the HTTPClient source code package. This folder contains some basic usage. These examples are sufficient to find ClientFormLogin. java annotations have been clearly explained, which means simulating HTTP requests to store cookies. Test site: http://bbs.dakele.com/because this site has made a special deal for login, so with the standard DZ forum may have some access, Please modify the analysis of the site using chrome built-in review elements, this tossing a lot of time login address: http://passport.dakele.com/login.do? Product = bbs enter the wrong user name and password will find the actual login address for the http://passport.dakele.com/logon.do note [I/n the difference at the beginning did not pay attention to think that the ghost] Return Error message {"err_msg ": "account or Password error"} enter the correct information and return {"result": true, "redirect": http://bbs.dakele.com/member.php? Mod = logging & action = login & loginsubmit = yes & infloat = yes & lssubmit = yes & inajax = 0 & fastloginfield = username & quickforward = yes & handlekey = ls & cookietime = 2592000 & remember = 0 & username = youname & AccessKey = []} enter the rediret connection and normal logon to get the jump link: copy the code private LoginResult getRedirectUrl () {LoginResult loginResult = null; CloseableHttpClient httpClient = HttpClients. createDefault (); HttpPost httpost = new HttpPost (LOGINURL); httpos T. setHeader ("Accept", "text/html, application/xhtml + xml, application/xml; q = 0.9, image/webp, */*; q = 0.8 "); httpost. setHeader ("Accept-Language", "zh-CN, zh; q = 0.8"); httpost. setHeader ("Cache-Control", "max-age = 0"); httpost. setHeader ("Connection", "keep-alive"); httpost. setHeader ("Host", "passport.dakele.com"); httpost. setHeader ("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, l Ike Gecko) Chrome/30.0.1599.101 Safari/537.36 "); List <NameValuePair> nvps = new ArrayList <NameValuePair> (); nvps. add (new BasicNameValuePair ("product", "bbs"); nvps. add (new BasicNameValuePair ("surl", "http://bbs.dakele.com/"); nvps. add (new BasicNameValuePair ("username", "yourname"); // user name nvps. add (new BasicNameValuePair ("password", "yourpass"); // password nvps. add (new BasicNameValuePair ("remember "," 0 "); httpost. setEntity (new UrlEncodedFormEntity (nvps, Consts. UTF_8); CloseableHttpResponse response2 = null; try {response2 = httpClient.exe cute (httpost); if (response2.getStatusLine (). getStatusCode () = 200) {HttpEntity entity = response2.getEntity (); String entityString = EntityUtils. toString (entity); JSONArray jsonArray = JSONArray. fromObject ("[" + entityString + "]"); JsonConfig jsonConfig = new Jso NConfig (); jsonConfig. setArrayMode (JsonConfig. MODE_OBJECT_ARRAY); jsonConfig. setRootClass (LoginResult. class); LoginResult [] results = (LoginResult []) JSONSerializer. toJava (jsonArray, jsonConfig); if (results. length = 1) {loginResult = results [0] ;}} catch (ClientProtocolException e) {e. printStackTrace ();} catch (IOException e) {e. printStackTrace ();} finally {try {response2.close (); httpClien T. close ();} catch (IOException e) {e. printStackTrace () ;}} return loginResult;} copy the code logon code: copy the code public boolean login () {boolean flag = false; LoginResult loginResult = getRedirectUrl (); if (loginResult. getResult (). equals ("true") {cookieStore = new BasicCookieStore (); globalClient = HttpClients. custom (). setdefacookcookiestore (cookieStore ). build (); HttpGet httpGet = new HttpGet (loginResult. getRedire Ct (); httpGet. setHeader ("Accept", "text/html, application/xhtml + xml, application/xml; q = 0.9, image/webp, */*; q = 0.8 "); httpGet. setHeader ("Accept-Language", "zh-CN, zh; q = 0.8"); httpGet. setHeader ("Connection", "keep-alive"); httpGet. setHeader ("Host", HOST); httpGet. setHeader ("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36"); try {GlobalClient.exe cute (httpGet);} catch (ClientProtocolException e) {e. printStackTrace ();} catch (IOException e) {e. printStackTrace ();} List <Cookie> cookies2 = cookieStore. getCookies (); if (cookies2.isEmpty () {log. error ("cookie is empty");} else {for (int I = 0; I <cookies2.size (); I ++) {}} return flag ;} after copying the code, you have successfully logged on to the console. You can only perform the tasks with the logon number. What? You do not know, of course, the fire suppression is the first thing you need to reply, the list page is relatively regular. If no code is found automatically, a loop @ 1 is written to copy the code for (int I = 1; I <200; I ++) {String basurl = "http://bbs.dakele.com/forum-43-" + I + ". html "; log.info (basurl); List <String> urls = dakele. getThreadURLs (basurl); for (String url: urls) {// log.info (url); ReplayContent content = dakele. preReplay (url); if (content! = Null) {log.info (content. getUrl (); log.info (content. getMessage (); // dakele. replay (content); // Thread. sleep (15300) ;}} copy the code to obtain the post address on the list page: copy the code String html = EntityUtils. toString (entity); Document document = Jsoup. parse (html, HOST); Elements elements = document. select ("tbody [id ^ = normalthread _]> tr> td. new>. xst "); for (int I = 0; I <elements. size (); I ++) {Element e = elements. get (I); urList. add (e. attr (" Abs: href ");} copy the code to obtain the form address to be submitted and construct the reply content in the post to be replied. Copy the code public ReplayContent preReplay (String url) {ReplayContent content = null; HttpGet get = new HttpGet (url); get. setHeader ("Accept", "text/html, application/xhtml + xml, application/xml; q = 0.9, image/webp, */*; q = 0.8 "); get. setHeader ("Accept-Language", "zh-CN, zh; q = 0.8"); get. setHeader ("Connection", "keep-alive"); get. setHeader ("Host", HOST); get. setHeade R ("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 "); try {CloseableHttpResponse response = globalClient.exe cute (get); HttpEntity entity = response. getEntity (); String html = EntityUtils. toString (entity); Document document = Jsoup. parse (html, HOST); Element postForm = document. getElementById ("fastpostform"); if (! PostForm. toString (). contains ("You are not authorized to post now") {content = new ReplayContent (); content. setUrl (url); log. debug (postForm. attr ("abs: action"); content. setAction (postForm. attr ("abs: action"); // Elements teElements = document. select ("td [id ^ = postmessage _]"); String message = ""; for (int I = 0; I <teElements. size (); I ++) {String temp = teElements.get( I ).html (). replaceAll ("(? Is) <. *?> "," "); If (temp. contains ("posted on") {String [] me = temp. split ("\ s +"); temp = me [me. length-1];} message + = temp. replaceAll ("\ s +", "");} log. debug (message. replaceAll ("\ s +", ""); // * obtain the last comment Element messageElement = document. select ("td [id ^ = postmessage _]"). last (); // String message = messageElement.html (). replaceAll ("\ & [a-zA-Z };",""). replaceAll ("<[^>] *> ",""). replaceAll ("[(/>) <]", ""); String message = messageElement.html (). replaceAll ("(? Is) <. *?> "," "); */If (message. contains ("posted on") {String [] me = message. split ("\ s +"); message = me [me. length-1];} content. setMessage (message. replaceAll ("& nbsp ;",""). replaceAll ("Upload ",""). replaceAll ("attachment ",""). replaceAll ("Download", ""); Elements inputs = postForm. getElementsByTag ("input"); for (Element input: inputs) {log. debug (input. attr ("name") + ":" + input. attr ("value"); if (input. attr ("name "). equals ("posttime") {content. setPosttime (input. attr ("value");} else if (input. attr ("name "). equals ("formhash") {content. setFormhash (input. attr ("value");} else if (input. attr ("name "). equals ("usesig") {content. setUsesig (input. attr ("value");} else if (input. attr ("name "). equals ("subject") {content. setSubject (input. attr ("value") ;}} else {log. warn ("You are not authorized to post now:" + url) ;}} catch (ClientProtocolException e) {e. printStackTrace ();} catch (IOException e) {e. printStackTrace ();} return content;} the copy Code address is available. When the content is available, the copy code public void replay (ReplayContent content) {HttpPost httpost = new HttpPost (content. getAction (); httpost. setHeader ("Accept", "text/html, application/xhtml + xml, application/xml; q = 0.9, image/webp, */*; q = 0.8 "); httpost. setHeader ("Accept-Language", "zh-CN, zh; q = 0.8"); httpost. setHeader ("Cache-Control", "max-age = 0"); httpost. setHeader ("Connection", "keep-alive"); httpost. setHeader ("Host", HOST); httpost. setHeader ("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 "); list <NameValuePair> nvps = new ArrayList <NameValuePair> (); nvps. add (new BasicNameValuePair ("posttime", content. getPosttime (); nvps. add (new BasicNameValuePair ("formhash", content. getFormhash (); nvps. add (new BasicNameValuePair ("usesig", content. getUsesig (); nvps. add (new BasicNameValuePair ("subject", content. getSubject (); nvps. add (new BasicNameValuePair ("message", content. getMessage (); httpost. setEntity (new UrlEncodedFormEntity (nvps, Consts. UTF_8); // The HTTP three-way handshake must process the response. At the beginning, the response is not noticed. CloseableHttpResponse response2 = null; try {response2 = globalClient.exe cute (httpost ); // log.info (content. getAction (); // log.info (content. getMessage (); HttpEntity entity = response2.getEntity (); EntityUtils. consume (entity); // BufferedWriter bw = new BufferedWriter (new FileWriter ("d:/tt1.html"); // bw. write (EntityUtils. toString (response2.getEntity (); // bw. flush (); // bw. close (); // System. out. println (EntityUtils. toString (response2.getEntity ();} catch (ClientProtocolException e) {e. printStackTrace ();} catch (IOException e) {e. printStackTrace () ;}} copy the code