Using go to achieve the five persistence of Redis

Write in front

The Godis code version implemented in this article is: v0.1

Redis Persistence mode

RDB Persistence

The Bgsave and save commands generate an Rdb file that stores the database information. When the server is started, the Rdb file is also used as raw data to load the near service memory. There is a priority problem here-when aof persistence is on, it takes precedence to load data from the AoF file and restore the database state.

The Save command blocks the service, and Bgsave derives the standalone process without blocking. You can also automate the cycle of RDB persistence through the option configuration.

The REDIS server maintains database modifications by documenting several parameters, such as the Server.dirty field in the first section that records how many database modifications have been made since the last save. When a recurring background operation Servercon executes, it checks that the database's update status satisfies the RDB persistence condition, which saves the database state.
Note: The Rdb file stores the database data in a way that stores the key-value pairs.

AOF Persistence

The previous article mentions that the Rdb file holds the data itself, while the AoF file stores the protocol that the executed command is converted to. You can view file understanding by turning on Redis aof persistence and manipulating several commands appendonly.aof .
Because it is a backup operation to the data, the read command does not need to be logged, only the modified operation is logged.

If aof persists for each modification of the command, it will log some more invalid commands. Such as:

set alpha 123set alpha 1set alpha 321set alpha 123

The four command is a process, and the final value of the database record 123 is the final result of the process.
To avoid the "invalid command" record for the operation of the same key, Redis has a aof rewrite mechanism-reads the current data state as the command record to append the AoF file.

Godis implementation of AOF persistence

Godis only implements AOF persistence and does not overwrite the commands, all modifications are recorded into the AoF file. This also means that there are many invalid I/O operations during the data preservation phase, and many invalid commands are executed during the loading phase.

Data persistence to disk

Redis-like event loops are not used in Godis encoding, and we rely on the Server.dirty field as the identity. Dirty change is the time for persistence.

First, add the aof persistence judgment at the command call and persist if the dirty changes:

func call(c *Client, s *Server) {    dirty := s.Dirty    c.Cmd.Proc(c, s)    dirty = s.Dirty - dirty    if dirty > 0 {//dirty变化 进行持久化        AppendToFile(s.AofFilename, c.QueryBuf)    }}

The function that performs the persistence operation Appendtofile is also very simple, appends the file to write, and closes immediately:

func AppendToFile(fileName string, content string) error {    // 以只写的模式，打开文件    f, err := os.OpenFile(fileName, os.O_WRONLY|syscall.O_CREAT, 0644)    if err != nil {        log.Println("log file open failed" + err.Error())    } else {        n, _ := f.Seek(0, os.SEEK_END)        _, err = f.WriteAt([]byte(content), n)    }    defer f.Close()    return err}

Finally, at the implementation of the modified command (such as set the command), add an update to the Server.dirty.

func SetCommand(c *Client, s *Server) {    ···    s.Dirty++    ···}

Let's test the effect, recompile the Godis-server.go, and execute set alpha 123 :

The command protocol has been successfully written to the file.

Service startup Load Data

The way persistent data is loaded into memory from a file is to impersonate the client to execute the command and send the aof file command to the server.

func LoadData() {    c := godis.CreateClient()    pros := core.ReadAof(godis.AofFilename)    for _, v := range pros {        c.QueryBuf = string(v)        err := c.ProcessInputBuffer()        if err != nil {            log.Println("ProcessInputBuffer err", err)        }        godis.ProcessCommand(c)    }}

Core. Readaof reads the aof file into memory and stores it in a sub-bar. Then the ProcessCommand in the Set/get command implementation of the introduction, no longer explained.

Integration Testing

Close the server, restart the server, and execute directly on the client get alpha to see if it can get the value of the previous set:

Check the aof file without reading the command get related records.

This article questions

The pseudo-client does not perform persistent operations and adds pseudo-terminal flag bits to the client structure for persistent judgment:

func LoadData() {    c := godis.CreateClient()    c.FakeFlag = true    ···}

Summary

There's no preview.
The original intention of these five essays was to record the small demo that was written when learning go, and the result was that the energy spent in other languages was beyond the budget, so there was no further development of Godis's new feature.

Tomorrow in the company's work ushered in a busy project transformation period, taking advantage of today's Dragon Boat Festival, the last day of a small holiday, understand the V0.1 version.
However, in the near future, perhaps the next small holiday, will be mentioned before the key expiration, network optimization, API development, stream and other new feature and optimize the public.
Welcome to the discussion.