PostgreSQL xlog file name specification source code parsing

PostgreSQL xlog file name specification source code parsing

PostgreSQL xlog, that is, WAL (Write Ahead Log), is located In the pg_xlog directory and is the Log on which PostgreSQL fully or PITR (Point In-Time Recovery) depends.

Why is there WAL logs? This is the same as mainstream databases such as Oracle and MySQL to ensure transaction persistence. That is, any transaction that has been committed must be persistent.

If I write the redo log to the disk prior to the data, the transaction can always be restored, although the time may be longer. The reason for writing logs first is that writing logs to a disk is faster than writing data to a disk.

Logs are flushed out by continuous IO, while data files are generally random IO.

In addition, you may find that the PostgreSQL log file name is very long and the size is 16 Mb. Currently, PG's default xlog segment size is 16 M, which can only be set during source code compilation. If I set it to 64 M,

The configure parameter "-- with-wal-segsize = 64" is required during compilation.

In addition, the xlog file name can be divided into two parts, taking "000000010000000000000002" as an example, and divided into the following two parts:

00000001,0000000000000002

The first part is "00000001", which is the TimeLineId, which is commonly referred to as TLI. It is used to identify the database status. After each restoration, it will increase, the main purpose is to distinguish the recovered database from the previous database,

For example, Point In-Time Recovery. Normally, shutting down and starting the database does not change this value. PG uses a 32-bit TimeLineId. The source code is defined as follows:

This is somewhat similar to the concept of Incarnation in Oracle.

The second part is "0000000000000002", which is xlogsegno, expressed in 64 bits. The source code is defined:

This is actually the maximum value that a 64-bit system can represent. This is the theoretical maximum value of xlogsgno, that is, "ffffffffffffff ". In fact, the file number cannot reach this value. The following is a detailed description.

First, let's understand the memory addressing representation of the log stream. In PostgreSQL, logs are expressed by memory offset addresses. The source code is defined:

PG uses a 64-bit address to indicate the location of the log. Many people think that it is easy to use up 64-bit. Is that actually the case? We can calculate:

Assume that the number of logs generated in a day is 10 TB, that is, 2 ^ 40 power. The number of years that can be used is 2 ^ 64/2 ^ 24/365 = 4596.49. This is enough.

At this point, many people are in a hurry. What is the relationship between this and the log file name?

The answer is yes. As mentioned above, the second part of the log file xlogsegno can be split into two 32-bit segments. The 32-bit high represents the serial number of the xlog, and the 32-bit low represents the segment number contained in the xlog.

The source code of Xlog segment is defined as follows:

File Name Definition source code:

Assume that the segment size of xlog is 16 Mb. The number of segments corresponding to an xlog is 2 ^ 32/2 ^ 24 = 2 ^ 8 = 256. Therefore, the maximum xlog file name should be "00000001ffffff000000ff", rather than the theoretical maximum "00000001FFFFFFFFFFFFFFFF" mentioned above ".

In addition, when the PostgreSQL database is initialized, the first position of xlog starts at the logical offset of XLOG_SEG_SIZE. Therefore, when the size is 16 MB, The Minimum File number is "0001ffffffffff00000001" instead of "0001ffffffff00000000 ". The xlog record pointor is invalid starting from "0". The following is an illegal xlog record pointor defined in the source code.

Next we can use an experiment to verify the above process:

1. First, reset the current xlog location to an xlog that is less than the maximum value.

2. Perform checkpoint and switch logfile

First, an error is reported, the instance is terminated, and the xlog is cyclic, starting from the minimum value. After FFFFFFFF000000FF --> 0000000000000001

Background log process:

------------------------------------ Lili split line ------------------------------------

Install PostgreSQL 6.3 on yum in CentOS 9.3

PostgreSQL cache details

Compiling PostgreSQL on Windows

Configuration and installation of LAPP (Linux + Apache + PostgreSQL + PHP) Environment in Ubuntu

Install and configure phppgAdmin on Ubuntu

Install PostgreSQL9.3 on CentOS

Configure a Streaming Replication cluster in PostgreSQL

How to install PostgreSQL 7/6 and phpPgAdmin in CentOS 5/6. 4

------------------------------------ Lili split line ------------------------------------

PostgreSQL details: click here
PostgreSQL: click here

This article permanently updates the link address: