Java cracking ip shielding + multithreading synchronous dialing-[5 to multithreading Data Collection]

Ip address shielding for single-thread cracking. Relatively easy. You only need to dial.

Multi-threaded data capture is also relatively easy, but after multithreading is introduced. Dialing is prone to problems.
During multi-thread crawling, this dial-up is more troublesome. Because, multithreading dialing,

The following occurs: when the first thread is dialing, the second thread also enters dialing, and the third thread enters the same four. This will cause the thread to continue dialing. This leads to an infinite loop.

Recent csdn blog election
From 88 candidates, 20 registrants will be elected
Please vote for me!

How can we control the dial-up synchronization to achieve high efficiency.
I am a newbie and have adopted a simple method.

That is, when you start dialing, we will tell you that I am already dialing.

If you cannot capture the data, please wait. When my first dial-up is complete, all the threads will capture the data in the next round.

1. Thread

Where:

AdslThead. isadsl = true; is used to set and tell the following thread that someone is dialing.

Package com. yjf. util;
Import java. util. Date;
Import java. util. List;

Public class GetWebThread extends Thread {
 
/**
* Thread
*/
Public void run (){
Try {
While (true ){
Int day = 0;
Long time1 = new Date (). getTime ();
// Used to synchronously capture threads
Synchronized ("searchthead "){
Main. thisdaycount ++;
If (Main. thisdaycount> Main. daycount ){
Break;
}
System. out. println ("START querying" + (Main. thisdaycount) + "day ");
Thread. sleep (133 );
Day = Main. thisdaycount-1;
}
// Obtain the capture time
String datetext = TimeUtil. date. format (TimeUtil. addDaysForDate (day ));
String [] txt = FileUtil. getCityByTxtFile ();
For (int t = 0; t <txt. length; t ++ ){

String [] way = txt [t]. split ("\\| ");
String start = way [0];
String end = way [1];
// The captured data list
List <DataBean> datalist = Main. getDataList (datetext, start, end );
If (datalist! = Null ){
Main. isadsl = 0;
CheckAdsl. adsllasttime = new Date (). getTime ();
FileUtil. addDataToFileCsv (datalist );
Main. log. printLog ("=" + datetext + "=" + start + "=" + end + "=" + t + "= total data volume:" + datalist. size ());
} Else {
Thread. sleep (11 );
AdslThead. isadsl = true;
Thread. sleep (11 );
// Determine whether the dialing is in progress and pause the thread
While (AdslThead. isadsl ){
Thread. sleep (5000 );
}
T --;
}
}
Long time2 = new Date (). getTime ();
Main. log. printLog (datetext + "= query completed ========== time consumed" + (time2-time1 ));
}
} Catch (Exception e ){
Main. log. printLog (e. getMessage ());
E. printStackTrace ();
}
}
 
}

2. Dialing listening thread

Package com. yjf. util;

Import java. util. Date;
Import java. util. TimerTask;

Public class AdslThead extends TimerTask {
 
Public static boolean isadsl = false;

@ Override
Public void run (){
Try {
If (isadsl ){
System. out. println ("------- dialing thread startup -----");
ConnectAdslNet. reconnectAdsl ("Broadband", Main. adslname, Main. adslpass );
Isadsl = false;
}
} Catch (Exception e ){
System. out. println (e. getMessage ());
}
}
}
3. Start the dialing listener. It repeats every other time. In this way, the stability and accuracy are higher.

Timer adsl = new Timer ();
Adsl. schedule (new AdslThead (), * 5); // start the dialing detection thread