LevelDB源碼分析-Write

標籤：current   love   mat   option   cte   turn   push   during   tor   

Write

LevelDB提供了write和put兩個介面進行插入操作，但是put實際上是調用write實現的，所以我在這裡只分析write函數：

Status DBImpl::Write(const WriteOptions &options, WriteBatch *my_batch)

首先初始化一個Writer對象，Writer對象用於封裝一個插入操作，LevelDB用一個deque來管理Writer對象，建立的Writer對象被插入到這個deque的尾部，如果Writer對象未被處理且不在deque頭部，則會一直等待：

    Writer w(&mutex_);    w.batch = my_batch;    w.sync = options.sync;    w.done = false;    MutexLock l(&mutex_);    writers_.push_back(&w);    while (!w.done && &w != writers_.front())    {        w.cv.Wait();    }    if (w.done)    {        return w.status;    }

然後調用MakeRoomForWrite函數保證memtable中有插入的空間：

    // May temporarily unlock and wait.    Status status = MakeRoomForWrite(my_batch == nullptr);    uint64_t last_sequence = versions_->LastSequence();    Writer *last_writer = &w;

接下來調用BuildBatchGroup函數將此時writers_隊列中的Writer對象全部封裝為一個WriteBatch，也就是說LevelDB實際上一次會處理當前的所有插入任務：

    if (status.ok() && my_batch != nullptr)    { // nullptr batch is for compactions        WriteBatch *updates = BuildBatchGroup(&last_writer);        WriteBatchInternal::SetSequence(updates, last_sequence + 1);        last_sequence += WriteBatchInternal::Count(updates);

再調用函數將KV值插入memtable中：

        // Add to log and apply to memtable.  We can release the lock        // during this phase since &w is currently responsible for logging        // and protects against concurrent loggers and concurrent writes        // into mem_.        {            mutex_.Unlock();            status = log_->AddRecord(WriteBatchInternal::Contents(updates));            bool sync_error = false;            if (status.ok() && options.sync)            {                status = logfile_->Sync();                if (!status.ok())                {                    sync_error = true;                }            }            if (status.ok())            {                status = WriteBatchInternal::InsertInto(updates, mem_);            }            mutex_.Lock();            if (sync_error)            {                // The state of the log file is indeterminate: the log record we                // just added may or may not show up when the DB is re-opened.                // So we force the DB into a mode where all future writes fail.                RecordBackgroundError(status);            }        }        if (updates == tmp_batch_)            tmp_batch_->Clear();        versions_->SetLastSequence(last_sequence);    }

將隊列中此次已經處理的Writer對象都刪除，並且給那些Writer對象發送訊號，使它們能夠結束自己的任務：

    while (true)    {        Writer *ready = writers_.front();        writers_.pop_front();        if (ready != &w)        {            ready->status = status;            ready->done = true;            ready->cv.Signal();        }        if (ready == last_writer)            break;    }

如果當前隊列中有新的Writer對象，發送訊號啟用隊首的Writer對象：

    // Notify new head of write queue    if (!writers_.empty())    {        writers_.front()->cv.Signal();    }    return status;

Write函數調用的MakeRoomForWrite函數為：

// REQUIRES: mutex_ is held// REQUIRES: this thread is currently at the front of the writer queueStatus DBImpl::MakeRoomForWrite(bool force)

函數將一直進行迴圈，判斷各個條件並執行相應操作，直到memtable中有足夠空間可以插入。

如果level0的檔案數量超過閾值，且這是第一次檢測到這種情況，那麼sleep1ms：

        else if (            allow_delay &&            versions_->NumLevelFiles(0) >= config::kL0_SlowdownWritesTrigger)        {            // We are getting close to hitting a hard limit on the number of            // L0 files.  Rather than delaying a single write by several            // seconds when we hit the hard limit, start delaying each            // individual write by 1ms to reduce latency variance.  Also,            // this delay hands over some CPU to the compaction thread in            // case it is sharing the same core as the writer.            mutex_.Unlock();            env_->SleepForMicroseconds(1000);            allow_delay = false; // Do not delay a single write more than once            mutex_.Lock();        }

如果當前memtable中有足夠的空間，則跳出迴圈：

        else if (!force &&                 (mem_->ApproximateMemoryUsage() <= options_.write_buffer_size))        {            // There is room in current memtable            break;        }

如果當前memtable中空間不足，immutable memtable也沒有被寫出，則等待compact的背景線程完成compact（immutable memtable需要compact）：

        else if (imm_ != nullptr)        {            // We have filled up the current memtable, but the previous            // one is still being compacted, so we wait.            Log(options_.info_log, "Current memtable full; waiting...\n");            background_work_finished_signal_.Wait();        }

如果當前memtable空間不足，level0中的檔案數量超過了閾值，且不是第一次檢測到這種情況，則等待compact的背景線程完成compact（level0需要compact）：

        else if (versions_->NumLevelFiles(0) >= config::kL0_StopWritesTrigger)        {            // There are too many level-0 files.            Log(options_.info_log, "Too many L0 files; waiting...\n");            background_work_finished_signal_.Wait();        }

如果以上情況都不存在，則說明可以將當前memtable寫入immutable memtable，然後建立一個新的memtable，當然需要調用MaybeScheduleCompaction函數，因為產生了immutable memtable需要compact：

        else        {            // Attempt to switch to a new memtable and trigger compaction of old            assert(versions_->PrevLogNumber() == 0);            uint64_t new_log_number = versions_->NewFileNumber();            WritableFile *lfile = nullptr;            s = env_->NewWritableFile(LogFileName(dbname_, new_log_number), &lfile);            if (!s.ok())            {                // Avoid chewing through file number space in a tight loop.                versions_->ReuseFileNumber(new_log_number);                break;            }            delete log_;            delete logfile_;            logfile_ = lfile;            logfile_number_ = new_log_number;            log_ = new log::Writer(lfile);            imm_ = mem_;            has_imm_.Release_Store(imm_);            mem_ = new MemTable(internal_comparator_);            mem_->Ref();            force = false; // Do not force another compaction if have room            MaybeScheduleCompaction();        }

231 Love u

LevelDB源碼分析-Write