The application of Java web crawler in batch download of pea clip

Today finally put the script, although it is patchwork, finally a bit of accomplishment, the following code post this ~

 PackageStudyjava;ImportJava.io.BufferedReader;ImportJava.io.File;ImportJava.io.FileOutputStream;ImportJava.io.FileWriter;Importjava.io.IOException;ImportJava.io.InputStream;ImportJava.io.InputStreamReader;Importjava.net.HttpURLConnection;Importjava.net.MalformedURLException;ImportJava.net.URL;Importjava.net.URLConnection;ImportJava.util.regex.Matcher;ImportJava.util.regex.Pattern;ImportJava.util.stream.Stream;Importjava.net.HttpURLConnection;ImportJavax.xml.transform.stream.StreamSource; Public classclient{ Public Static voidMain (string[] args)throwsIOException {//create two files in the D drive, one for all apk download links, one for all downloaded app namesFile f1=NewFile ("D:\\down.txt"); File F2=NewFile ("D:\\down_appname.txt"); if(F1.exists ()) F1.delete (); ElseF1.createnewfile (); if(F2.exists ()) F2.delete (); ElseF2.createnewfile (); FileWriter FW=NewFileWriter (F1,true); FileWriter fw2=NewFileWriter (F2,true);        String str; //download URLs such as the Pea Clip Essentials app pageURL url=NewURL ("Http://www.wandoujia.com/essential/app"); //here is a given URL, how do you get its source code?URLConnection conn=url.openconnection (); BufferedReader BR=NewBufferedReader (NewInputStreamReader (Conn.getinputstream ())); //here is the contention for each line of input stream to filter, is the URL to get me out, and then create a download link         while((Str=br.readline ())! =NULL) {pattern pattern=pattern.compile ("http://apps.wandoujia.com/apps/.+." ( [a-z]+\\.) *.+/download\ "\\sdownload=\". +\\s*.*\\.apk\ ""); Matcher Matcher=Pattern.matcher (str); if(Matcher.find ()) {//formally write the download URL and app name to the appropriate fileString Downurl=matcher.group (). Split ("\\s") [0].split ("\" ") [0]; String Downname=matcher.group (). Split ("=") [1];                Fw.write (Downurl); Fw.write ("\ n");                Fw2.write (Downname); Fw2.write ("\ n"); //here is the download app                byte[] buffer =New byte[4 * 1024]; //Gets the network resource connection for the app,URL urldown=NewURL (Downurl); HttpURLConnection conn2=(HttpURLConnection) urldown.openconnection (); Conn2.setdoinput (true);                Conn2.connect (); //Get input streamInputStream in=Conn2.getinputstream (); //Create a folder to place download appsFile dir=NewFile ("D:\\downapp"); if(!dir.exists ()) Dir.mkdir (); //Create a downloaded app, file name and storage pathFile appdown=NewFile (Dir,downname.split ("\" ") [1]); if(!appdown.exists ()) appdown.createnewfile (); //Get output streamFileOutputStream out=NewFileOutputStream (Appdown); intBr1=-1;  while(Br1= (in.read (buffer)))!=-1) {out.write (buffer,0, BR1);                } out.flush ();                In.close ();            Out.close ();        }} fw.close (); Fw2.close ();}}

The application of Java web crawler in batch download of pea clip