Abnormal power loss --- xfs file system

This article is the author's own learning summary or sentiment (linux environment). If you have any questions, please join us !!

Directory structure

1. Related knowledge

II. Question proposal

III. Handling methods

IV. Final results

 

I. Related knowledge

1. xfs file system (there are a lot of conceptual knowledge online. Check it yourself)

File system definition:

The file system is the method and data structure used by the operating system to identify the storage devices (disks and SSD based on NAND Flash) or partition files; that is, the method of organizing files on the storage device.

Xfs file system:

Is a log file system.

Log File system? Add a log to record changes to the file system. Ensure file system consistency even when the power is down or the operating system crashes.

How to keep it?

When writing data to a disk, you must change the metadata. Before that, the log must record how to change the metadata. When an exception powers down or the file system crashes, the consistency of the file system will be checked during the restoration. When there is an inconsistency, you can use it to restore it.

(Changes to metadata are stored in logs before data is flushed to the disk)

2. VM parameters

1. What is virtual memory?

1). VM parameters are different for different linux kernel versions. The kernel 3.4.24-v64

2) VM parameters are located in/proc/sys/vm to optimize the virtual memory

3)./proc directory? The Linux kernel provides a mechanism to access the internal data structure of the kernel and change the kernel settings through the/proc file system during runtime. The proc file system is a pseudo-file system that only exists in the memory and does not occupy the external storage space. It provides interfaces for accessing system kernel data in the form of a file system. Users and applications can obtain system information through proc and change certain kernel parameters. Because system information, such as processes, is dynamically changed, when a user or application reads the proc file, the proc file system dynamically reads the required information from the system kernel and submits it.

4)./proc/sys contains system information and kernel parameters.

5)./etc/sysctl. conf all kernel configuration information, which can be permanently modified

Note: Due to the large number of VM parameters, if you are interested, refer to another article titled Introduction to vm parameters.

 

3. XFS file system parameters

1. The content on the Internet is relatively small. I will introduce the following points:

1) parameters of the xfs file system can be obtained only when the xfs file system is mounted to the linux system.

2) parameters of the xfs file system are located in/proc/sys/fs/xfs. You can view or debug parameters in real time.

Note: There are many XFS parameters. If you are interested, refer to my other article "XFS parameter introduction".

 

4. My understanding: VM parameters and XFS parameters are two sets of cache control parameters. They should have the following relationships:

Vm parameters and xfs parameters have different impact stages:

This involves the linux Cache mechanism. For details, see Introduction to the file Cache management mechanism of Linux kernel.

I will give a brief introduction here with "Linux file system analysis": linux has a virtual file system on a specific file system (ext3, xfs, fat32, and so on. The virtual file system encapsulates a specific file system, which allows linux to Mount multiple file systems, but only uses the same set of functions or commands. The cache of the file system is subdivided into page cache and buffer cache. They all exist in the memory, but the page cache is mainly used to cache file content for frequent reading, and the buffer cache is used to fl data to the disk. Experiments show that when a raw device is written using dd, the buffer cache grows rapidly. When data is copied in a file system, the page cache increases rapidly. For more information, see

II. Question proposal

This article is written because, on machines with xfs file systems, the file system will be damaged (inconsistency, data loss, etc.) when an exception or power loss occurs in the case of IO ). When the xfs file is repaired and the file system is dozens of TB, the repair time is very slow and we have not solved the problem in the end, it is estimated that, in addition to PMBUS battery or UPS, there is no better solution to the abnormal power loss problem in the xfs file system.

 

III. Handling methods

We tested the following variables:

1. dirty_radio and dirty_background_radio of VM parameters

2. The xfs parameters include age_buffer_centisecs, xfsbufd_centisecs, and xfssyncd_centisecs.

3. mkfs. xfs parameters

4. mount parameters

5. Memory size

6. Multi-thread cpu

 

Note: detailed test results and handling methods cannot be disclosed due to trade secrets.

 

IV. Final results

Here I will show you a process for xfs restoration by Huawei:

 

Conclusion: I hope to provide more suggestions on xfs file system restoration. Thank you!

Abnormal power loss --- xfs file system