HBase1.0.0源碼分析之請求處理流程分析以Put操作為例(二),hbase1.0.0put

來源:互聯網
上載者:User

HBase1.0.0源碼分析之請求處理流程分析以Put操作為例(二),hbase1.0.0put
HBase1.0.0源碼分析之請求處理流程分析以Put操作為例(二)

1.通過mutate(put)操作,將單個put操作添加到緩衝操作中,這些緩衝操作其實就是Put的父類的一個List的集合。如下:

    private List<Row> writeAsyncBuffer = new LinkedList<>();    writeAsyncBuffer.add(m);

writeAsyncBuffer滿了之後或者是人為的調用backgroundFlushCommits操作促使緩衝池中的操作被執行。

2.backgroundFlushCommits(boolean synchronous)執行操作緩衝池中的操作,其實他也並不是自己去處理響應的操作,而是委託給一個AsyncProcess具體進行響應操作的執行,該類是類比一個非同步處理持續請求流的類。這其中主要發生以下幾個主要操作:

    ap.submit(tableName, writeAsyncBuffer, true, null, false);    Map<ServerName, MultiAction<Row>> actionsByServer =        new HashMap<ServerName, MultiAction<Row>>();   List<Action<Row>> retainedActions = new  ArrayList<Action<Row>>(rows.size());   addAction(loc.getServerName(), regionName, action, actionsByServer, nonceGroup    return submitMultiActions(tableName, retainedActions, nonceGroup, callback, null, needResults,      locationErrors, locationErrorRows, actionsByServer, pool);

總結以下這裡主要幹了以下幾件事:
* 對Put操作進行封裝,封裝成Action
* 找出每個操作對應的regionServer形成ServerName - > MultiAction的索引值對,然後繼續submit
3. 在submitMultiActions方法裡面,作者使用了一個AsyncRequestFutureImpl的實現來儲存結果資料

 AsyncRequestFutureImpl<CResult> ars = createAsyncRequestFuture(      tableName, retainedActions, nonceGroup, pool, callback, results, needResults);

4.類ars中的sendMultiAction函數sendMultiAction才是最後真正的邏輯層調用的地方,完整的函數如下:

    private void sendMultiAction(Map<ServerName, MultiAction<Row>> actionsByServer,        int numAttempt, List<Action<Row>> actionsForReplicaThread, boolean reuseThread) {      // Run the last item on the same thread if we are already on a send thread.      // We hope most of the time it will be the only item, so we can cut down on threads.      int actionsRemaining = actionsByServer.size();      // This iteration is by server (the HRegionLocation comparator is by server portion only).      for (Map.Entry<ServerName, MultiAction<Row>> e : actionsByServer.entrySet()) {        ServerName server = e.getKey();        MultiAction<Row> multiAction = e.getValue();        incTaskCounters(multiAction.getRegions(), server);        Collection<? extends Runnable> runnables = getNewMultiActionRunnable(server, multiAction,            numAttempt);        // make sure we correctly count the number of runnables before we try to reuse the send        // thread, in case we had to split the request into different runnables because of backoff        if (runnables.size() > actionsRemaining) {          actionsRemaining = runnables.size();        }        // run all the runnables        for (Runnable runnable : runnables) {          if ((--actionsRemaining == 0) && reuseThread) {            runnable.run();          } else {            try {              pool.submit(runnable);            } catch (RejectedExecutionException ree) {              // This should never happen. But as the pool is provided by the end user, let's secure              //  this a little.              decTaskCounters(multiAction.getRegions(), server);              LOG.warn("#" + id + ", the task was rejected by the pool. This is unexpected." +                  " Server is " + server.getServerName(), ree);              // We're likely to fail again, but this will increment the attempt counter, so it will              //  finish.              receiveGlobalFailure(multiAction, server, numAttempt, ree);            }          }        }      }      if (actionsForReplicaThread != null) {        startWaitingForReplicaCalls(actionsForReplicaThread);      }    }

函數蠻長,但是做的事情的邏輯也是比較清晰的,從代碼中可以看出,程式的處理邏輯是按照regionServer進行區分的,將每個操作封裝成可啟動並執行任務(SingleServerRequestRunnable),然後用現成池pool依次執行。
這裡有必要將他們所構造的任務的run 函數貼出來研究以下,從run函數我們才能夠看到請求的真正邏輯:

    MultiResponse res;    MultiServerCallable<Row> callable = createCallable(server, tableName, multiAction);    res = createCaller(callable).callWithoutRetries(callable, timeout);    receiveMultiAction(multiAction, server, res, numAttempt);

每個任務都是通過HBase的RPC架構與伺服器進行通訊,並擷取返回的結果。至此,用戶端的Put操作的流程也就結束了,至於具體的RPC端如何執行相關內容,後續部落格繼續關注。

總結

1.用戶端有個緩衝機制,批量處理
2.預設的Put的處理是非同步進行的
3.Put最後是按照RegionServer分別RPC處理的

相關文章

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.