The Wal of HBase source analysis

WAL (Write-ahead Logging) is a technology that guarantees atomicity and persistence in a database system, and can improve the performance of data writes by using WAL to turn random writes of data into sequential writes. When writing data in HBase, the data is written to memory and the Wal log is written to prevent the log from being lost and the log is written on HDFs. &NBSP
By default, there are 1 wal per Regionserver, and multiple WALHBASE-5699 are supported at HBase1.0, which can improve write throughput. The configuration parameters are Hbase.wal.provider=multiwal and the supported values are defaultprovider and filesystem (these 2 are the same implementations). &NBSP
WAL has the following levels of persistence: Skip_wal: Do not write WAL log, this can greatly improve the performance of writing, but there will be the risk of data loss, only in the case of High-volume writes (error can be rerun), other cases are not recommended. Async_wal: Asynchronous Write Sync_wal: Writes the WAL log file synchronously, ensuring that the data is written to the Datanode node. Fsync_wal: Currently unsupported, the performance is consistent with Sync_wal Use_default: If no persistence level is specified, the default is Use_default, which is written to

using the HBase Global default level (Sync_wal) WAL

First look at some of the main classes in Wal writes
1. Key to the Walkey:wal log, including Regionname: Log-owned region
TableName: The table to which the log belongs, Writetime: Log write time, Clusterids:cluster ID, which is used when data is replicated.
2.WALEdit: Record a series of modified transaction logs in the HBase transaction log. In addition, the Waledit implements the writable interface, which can be used for serialization processing.
3. Fshlog:wal implementation class, responsible for writing data to the file system
In each Wal write here is the model of a multiple producer single consumer, where the disruptor framework is used to encapsulate Walkey and waledit information as Fswalentry and then ringbuffertruck into Ringbuffer. Next look at the Hlog write process, which is divided into the following 3 steps: Log Write cache: Rpchandler writes log information to cache Ringbuffer. Cached data is written to the file system: Each fshlog has a thread responsible for writing data to file system (HDFS) data synchronization: If the operation's persistence level is (Sync_wal or use_default) data synchronization is required

The following is a detailed description of how the various threads work together to achieve these steps, the Rpchandler thread is responsible for writing log information (fswalentry) to the cache Ringbbuffer, after the operation log is written, Rpchandler will invoke the Wal sync method, Data synchronization, which is actually processed to write a syncfuture to Ringbuffer, and then blocking until syncfuture processing is complete. The Wal thread takes data from the cache Ringbuffer, and if the log (Fswalentry) invokes the writer to write the data to the file system, and if it is syncfuture, it is synchronized by a dedicated synchronization thread.
The overall process flow chart is as follows:
Write of Hlog

The Wal write file system is written through writer, its actual class is Protobuflogwriter, and the PROTOBUF format is used for persistent processing. Using the PROTOBUF format has the following advantages: higher performance, more compact structure, space saving

Easy to expand and support other languages to parse logs in other languages.

Written logs are stored sequentially by Walkey and Waledit (as described in the previous Walkey and Waledit classes), and Walkey and Waledit are compressed separately. Wal synchronization Process

Each Wal has a Ringbuffereventhandler object in which multiple Syncrunner threads are managed by an array (parameter Hbase.regionserver.hlog.syncer.count configuration, default 5) for synchronous processing , each Syncrunner object has a linkedblockingqueue (syncfutures, size is the parameter {hbase.regionserver.handler.count default value 200}*3
In addition here the Syncfuture is owned by each Rpchandler thread, by the private final Map of Wal

Class Ringbuffereventhandler implements Eventhandler<ringbuffertruck> Lifecycleaware {
    Private final Syncrunner [] syncrunners;
    Private final syncfuture [] syncfutures;
    ...
}

Private class Syncrunner extends Hasthread {
    private volatile long sequence;
    Keep around last exception thrown. Clear on successful sync.
    Private final blockingqueue<syncfuture> syncfutures;
    ...
}

Here, when dealing with the syncfuture in Ringbuffer, not every one is submitted to the Syncrunner processing, but is processed by batch, where the batch is divided into 2 cases: a batch of data taken from the Ringbuffer (to improve efficiency, In the disruptor framework is to take the data from the Ringbuffer in batches, see Disruptor related documents, if the number of syncfuture in this batch of data <{ Hbase.regionserver.handler.count default value of 200}, batch processing if the number of syncfuture in this batch of data is >={ Hbase.regionserver.handler.count default value of 200}, then batch processing by {Hbase.regionserver.handler.count default value of 200}.

If the batch size is reached, the next Syncrunner is selected sequentially from the Syncrunner array, and the batch of data is inserted into the blockingqueue of the Syncrunner. Finally, the HDFs file is synchronized by the Syncrunner thread. To ensure that the data is not lost, RPC requests need to ensure that the Wal log write successfully before returning, here HBase do a series of optimized processing operations. Wal scrolling

By Wal log switching, this avoids the creation of a single, too large, Wal log file, which can facilitate subsequent log cleanup (you can delete the expired log files directly) and if you need to use the log for recovery, you can also resolve multiple small log files at the same time, shortening the recovery times.
The WAL trigger switch scenarios are as follows: The Syncrunner thread, after processing the log synchronization, invokes the Requestlogroll initiation log scrolling request Syncrunner thread after the synchronization of the log, when an exception occurs Check to see if the log size of the currently written Wal exceeds the configuration {hbase.regionserver.hlog.blocksize defaults to the HDFS directory block size}*{ Hbase.regionserver.logroll.multiplier default 0.95}, after the same call to the Requestlogroll log scrolling request each regionserver has a logroller thread periodically scrolls the log, The scrolling period is controlled by the parameter {hbase.regionserver.logroll.period default value of 1 hours}

The previous 2 scenarios call Requestlogroll to initiate log scrolling requests, and ultimately to perform log scrolling by Logroller. Wal failure

When the data in the Memstore is flushed to the HDFs, the corresponding Wal log is not needed, and Fshlog has the oldest memstore corresponding to the region in the current SequenceID. If the most recent sequenceid of each region operation in a log is less than the oldest SequenceID recorded in Wal for each region that needs to be refreshed, the log file is not needed, and the log file is moved from the./wals directory to./ Oldwals directory. This block is called Cleanoldlogs after the previous log scroll completes. Wal Delete

Because the Wal log is also used for synchronous processing across clusters, the Wal log is not deleted immediately after it is invalidated, but is moved to the Oldwals directory. The chore thread in the Hmaster is responsible for Wal log deletion, filtering out the log files that can be deleted by using the parameter {Logcleaner} as a plug-in within the hbase.master.logcleaner.plugins. Currently configured Plug-ins have Replicationlogcleaner, Snapshotlogcleaner, and Timetolivelogcleaner timetolivelogcleaner: Log file Last modified in configuration parameter { Hbase.master.logcleaner.ttl can delete replicationlogcleaner before default 600 seconds: if there is a need to synchronize across cluster data, the cleaner to ensure that the logs in sync are not deleted Snapshotlogcleaner: The Wal that is used by the snapshot of the table is not deleted summary

In this paper, the whole cycle of the Wal log in the HBase is narrated, can have the whole understanding to the Wal processing process, the follow-up in the separate said Wal log recovery content.

Resources:
1. http://hbasefly.com/2016/03/23/hbase_writer/
2. http://hbasefly.com/2016/10/29/hbase-regionserver-recovering/

Ext: https://blog.csdn.net/xiangel/article/details/54424900