Java線程的傳說(2)——HttpClient逾時機制(安全問題處理:訪問超大檔案控制)

 

說明：   項目中使用的HttpClient版本是3.1

測試

一般的HttpClient使用例子：

MultiThreadedHttpConnectionManager manager = new MultiThreadedHttpConnectionManager();        HttpClient client = new HttpClient(manager);        client.setConnectionTimeout(30000);        client.setTimeout(30000);        GetMethod get = new GetMethod("http://download.jboss.org/jbossas/7.0/jboss-7.0.0.Alpha1/jboss-7.0.0.Alpha1.zip");        try {            client.executeMethod(get);  //發起請求            String result = get.getResponseBodyAsString(); //擷取資料        } catch (Exception e) {        } finally {            get.releaseConnection(); //釋放連結        }

這裡一個url是近20MB的一個下載資源，很快發現線程要等個很久。得加個timeout逾時機制。

分析

目前httpClient3.1隻支援3種timeout的設定：

 

1. connectionTimeout  :  socket建立連結的逾時時間，Httpclient包中通過一個非同步線程去建立socket連結，對應的逾時控制。
2. timeoutInMilliseconds :  socket read資料的逾時時間， socket.setSoTimeout(timeout);
3. httpConnectionTimeout :  如果那個的是MultiThreadedHttpConnectionManager，對應的是從串連池擷取連結的逾時時間。

分析一下問題，我們需要的是一個HttpClient整個連結讀取的一個逾時時間，包括請求發起，Http Head解析，response流讀取的一系列時間的總和。 
目標很明確，對應的修正後的測試代碼：

final MultiThreadedHttpConnectionManager manager = new MultiThreadedHttpConnectionManager();        final HttpClient client = new HttpClient(manager);        client.setConnectionTimeout(30000);        client.setTimeout(30000);        final GetMethod get = new GetMethod(                                            "http://download.jboss.org/jbossas/7.0/jboss-7.0.0.Alpha1/jboss-7.0.0.Alpha1.zip");        Thread t = new Thread(new Runnable() {            @Override            public void run() {                try {                    client.executeMethod(get);                    String result = get.getResponseBodyAsString();                } catch (Exception e) {                    // ignore                }            }        }, "Timeout guard");        t.setDaemon(true);        t.start();        try {            t.join(5000l);  //等待5s後結束        } catch (InterruptedException e) {            System.out.println("out finally start");            ((MultiThreadedHttpConnectionManager) client.getHttpConnectionManager()).shutdown();            System.out.println("out finally end");        }        if (t.isAlive()) {            System.out.println("out finally start");            ((MultiThreadedHttpConnectionManager) client.getHttpConnectionManager()).shutdown();            System.out.println("out finally end");            t.interrupt();            // throw new TimeoutException();        }        System.out.println("done");

這裡通過Thread.join方法，設定了逾時時間為5000
ms，這是比較早的用法。 如果熟悉cocurrent包的，可以直接使用Future和ThreadPoolExecutor進行非同步處理，緩衝對應的Thread。Cocurrent代碼例子代碼

ExecutorService service = Executors.newCachedThreadPool();        Future future = service.submit(new Callable<String>() {            @Override            public String call() throws Exception {                try {                    client.executeMethod(get);                    return get.getResponseBodyAsString();                } catch (Exception e) {                    e.printStackTrace();                } finally {                    System.out.println("future finally start");                    ((MultiThreadedHttpConnectionManager) client.getHttpConnectionManager()).shutdown();                    System.out.println("future finally end");                }                return "";            }        });        try {            future.get(5000, TimeUnit.MILLISECONDS);        } catch (Exception e) {            System.out.println("out finally");            e.printStackTrace();            ((MultiThreadedHttpConnectionManager) client.getHttpConnectionManager()).shutdown();            System.out.println("out finally end");        }        service.shutdown();

說明： 這裡為什麼釋放連結未採用get.releaseConnection()看下release的實現：
public void releaseConnection() {        if (responseStream != null) {            try {                // FYI - this may indirectly invoke responseBodyConsumed.                responseStream.close(); // 會先關閉流            } catch (IOException e) {                // the connection may not have been released, let's make sure                ensureConnectionRelease();            }        } else {            // Make sure the connection has been released. If the response             // stream has not been set, this is the only way to release the             // connection.             ensureConnectionRelease();        }    }
這裡會先關閉responseStream流，這就是問題點。
對應的responseStream是在方法：readResponseBody(HttpConnection conn)。一般的html頁面返回的是一個ContentLengthInputStream對象
ContentLengthInputStream在調用close方法時會用ChunkedInputStream.exhaustInputStream讀完所有流資料
public void close() throws IOException {        if (!closed) {            try {                ChunkedInputStream.exhaustInputStream(this);            } finally {                // close after above so that we don't throw an exception trying                // to read after closed!                closed = true;            }        }    }
  

   
ChunkedInputStream.exhaustInputStream代碼
  
static void exhaustInputStream(InputStream inStream) throws IOException {        // read and discard the remainder of the message        byte buffer[] = new byte[1024];        while (inStream.read(buffer) >= 0) {            ;         }    }
說明： 
  

   
因為非sleep和park的方法，不會響應InterruptedException事件，所以普通future逾時發起的Thread.interrpt()並沒有效果。
   
預設的SimpleHttpConnectionManager不支援這樣的操作，所以選擇MultiThreadedHttpConnectionManager.shutdown()方法，強制關閉底層HttpConnection的sock的輸入輸出資料流。
  
總結
 
  

   
理解一下HttpClient這樣設計的理由： socket重用，keepAlive協議的支援等，保證上一次資料不會對新的請求有影響。
   
Thread.interrpt()處理，只會在Thread處於sleep或者wait狀態才會被喚醒(api的描述)。而且該方法的調用並不自動產生InterruptedException異常，一般是需要自己判斷Thread.isInterrupted()，然後throw異常。 我們目前使用的一些jdk cocurrent類比如future.cancel也是類似處理。
  
 Ref:http://agapple.iteye.com/blog/916837
上一篇：Java線程的傳說(1)——中斷線程Interrupted的用處
下一篇：常用的linux系統監控命令