HBase1.0.0源碼分析之請求處理流程分析以Put操作為例(二),hbase1.0.0put
HBase1.0.0源碼分析之請求處理流程分析以Put操作為例(二)
1.通過mutate(put)
操作,將單個put操作添加到緩衝操作中,這些緩衝操作其實就是Put的父類的一個List的集合。如下:
private List<Row> writeAsyncBuffer = new LinkedList<>(); writeAsyncBuffer.add(m);
當writeAsyncBuffer
滿了之後或者是人為的調用backgroundFlushCommits操作促使緩衝池中的操作被執行。
2.backgroundFlushCommits(boolean synchronous)
執行操作緩衝池中的操作,其實他也並不是自己去處理響應的操作,而是委託給一個AsyncProcess
具體進行響應操作的執行,該類是類比一個非同步處理持續請求流的類。這其中主要發生以下幾個主要操作:
ap.submit(tableName, writeAsyncBuffer, true, null, false); Map<ServerName, MultiAction<Row>> actionsByServer = new HashMap<ServerName, MultiAction<Row>>(); List<Action<Row>> retainedActions = new ArrayList<Action<Row>>(rows.size()); addAction(loc.getServerName(), regionName, action, actionsByServer, nonceGroup return submitMultiActions(tableName, retainedActions, nonceGroup, callback, null, needResults, locationErrors, locationErrorRows, actionsByServer, pool);
總結以下這裡主要幹了以下幾件事:
* 對Put操作進行封裝,封裝成Action
* 找出每個操作對應的regionServer形成ServerName - > MultiAction的索引值對,然後繼續submit
3. 在submitMultiActions
方法裡面,作者使用了一個AsyncRequestFutureImpl的實現來儲存結果資料
AsyncRequestFutureImpl<CResult> ars = createAsyncRequestFuture( tableName, retainedActions, nonceGroup, pool, callback, results, needResults);
4.類ars中的sendMultiAction
函數sendMultiAction才是最後真正的邏輯層調用的地方,完整的函數如下:
private void sendMultiAction(Map<ServerName, MultiAction<Row>> actionsByServer, int numAttempt, List<Action<Row>> actionsForReplicaThread, boolean reuseThread) { // Run the last item on the same thread if we are already on a send thread. // We hope most of the time it will be the only item, so we can cut down on threads. int actionsRemaining = actionsByServer.size(); // This iteration is by server (the HRegionLocation comparator is by server portion only). for (Map.Entry<ServerName, MultiAction<Row>> e : actionsByServer.entrySet()) { ServerName server = e.getKey(); MultiAction<Row> multiAction = e.getValue(); incTaskCounters(multiAction.getRegions(), server); Collection<? extends Runnable> runnables = getNewMultiActionRunnable(server, multiAction, numAttempt); // make sure we correctly count the number of runnables before we try to reuse the send // thread, in case we had to split the request into different runnables because of backoff if (runnables.size() > actionsRemaining) { actionsRemaining = runnables.size(); } // run all the runnables for (Runnable runnable : runnables) { if ((--actionsRemaining == 0) && reuseThread) { runnable.run(); } else { try { pool.submit(runnable); } catch (RejectedExecutionException ree) { // This should never happen. But as the pool is provided by the end user, let's secure // this a little. decTaskCounters(multiAction.getRegions(), server); LOG.warn("#" + id + ", the task was rejected by the pool. This is unexpected." + " Server is " + server.getServerName(), ree); // We're likely to fail again, but this will increment the attempt counter, so it will // finish. receiveGlobalFailure(multiAction, server, numAttempt, ree); } } } } if (actionsForReplicaThread != null) { startWaitingForReplicaCalls(actionsForReplicaThread); } }
函數蠻長,但是做的事情的邏輯也是比較清晰的,從代碼中可以看出,程式的處理邏輯是按照regionServer進行區分的,將每個操作封裝成可啟動並執行任務(SingleServerRequestRunnable),然後用現成池pool依次執行。
這裡有必要將他們所構造的任務的run 函數貼出來研究以下,從run函數我們才能夠看到請求的真正邏輯:
MultiResponse res; MultiServerCallable<Row> callable = createCallable(server, tableName, multiAction); res = createCaller(callable).callWithoutRetries(callable, timeout); receiveMultiAction(multiAction, server, res, numAttempt);
每個任務都是通過HBase的RPC架構與伺服器進行通訊,並擷取返回的結果。至此,用戶端的Put操作的流程也就結束了,至於具體的RPC端如何執行相關內容,後續部落格繼續關注。
總結
1.用戶端有個緩衝機制,批量處理
2.預設的Put的處理是非同步進行的
3.Put最後是按照RegionServer分別RPC處理的