NVMe Disk Write Amplification

1. SSD Disk Internal write operation Source 1.1 user request

As with other storage media, the user's write request data and related meta Data updates are the main drivers of SSD write operations.

1.2 Garbage collection (GC)

Since SSD media needs to be erased to write the same location, overwriting the SSD's logical address becomes an append write to the physical address of the internal SSD, and the physical address space previously occupied needs to be reclaimed. These spaces are usually collected as much as possible for the entire block, so that valid data within the block needs to be relocated, which leads to additional writing.

1.3 Wear leveling (wear leveling)

Similar restrictions are wear-leveling, and in order to ensure that different blocks of PE are as balanced as possible, relocation data may sometimes be needed internally.

1.4 Bad block Management

The data on the bad block needs to be relocated, which can also result in additional write operations inside.

1.5 reading Disturbances (read disturb)

When the number of reads for a flash block reaches a certain threshold, FTL moves the data from the Flash block to avoid data errors.

1.6 Data Hold (Retention)

Because of the charge loss, the data on the Flash block is subject to potential problems, and the FTL needs to relocate the data.

2. NVMe Disk Write Amplification statistics

From the introduction above you can see that there are many kinds of additional write operations inside the SSD, in order to analyze the impact of these write operations on the user write requests, you need to count this data.
For NVMe disks, you can use the NVMe CLI tool to get relevant data.

2.1 Preparatory work

On Linux systems, the NVMe CLI tools can be downloaded and compiled:
git clone https://github.com/linux-nvme/nvme-cli.git
CD NVME-CLI
Make

If the following problem occurs:
Nvme_version = 1.6.72.g43f2.dirty
cc-d_gnu_source-d__check_endian__-o2-g-wall-werror-std=gnu99-dnvme_version= ' "1.6.72.g43f2.dirty" '-C Nvme-print.c
In file included from nvme.h:34:0,
From Nvme-print.h:4,
From Nvme-print.c:6:
Linux/nvme.h:19:24:fatal error:linux/uuid.h:no such file or directory

The following methods can be used to resolve:
Modify the following:
#ifndef _linux_nvme_h
#define _linux_nvme_h

#include <linux/types.h>
#include <linux/uuid.h>//?? Comment out this line

2.2 Determining the NVMe disk type

Use Lsscsi/smart? Get the type of the relevant disk

2.3 Read NVMe Disk write statistics

[Email protected] nvme-cli]#/nvme Intel SMART-LOG-ADD/DEV/NVME0N1
Additional Smart Log for NVME device:nvme0n1 namespace-id:ffffffff
Key Normalized Raw
program_fail_count:100% 0
erase_fail_count:100% 0
wear_leveling:96% min:164, max:206, avg:190
end_to_end_error_detection_count:100% 0
crc_error_count:100% 0
timed_workload_media_wear:100% 63.999%
timed_workload_host_reads:100% 65,535%
timed_workload_timer:100% 65535 min
thermal_throttle_status:100% 0%, cnt:0
retry_buffer_overflow_count:100% 0
pll_lock_loss_count:100% 0
nand_bytes_written:100% sectors:7654336
host_bytes_written:100% sectors:6398681
[Email protected] nvme-cli]#/nvme Intel SMART-LOG-ADD/DEV/NVME0N1
Additional Smart Log for NVME device:nvme0n1 namespace-id:ffffffff
Key Normalized Raw
program_fail_count:100% 0
erase_fail_count:100% 0
wear_leveling:96% min:164, max:206, avg:190
end_to_end_error_detection_count:100% 0
crc_error_count:100% 0
timed_workload_media_wear:100% 63.999%
timed_workload_host_reads:100% 65,535%
timed_workload_timer:100% 65535 min
thermal_throttle_status:100% 0%, cnt:0
retry_buffer_overflow_count:100% 0
pll_lock_loss_count:100% 0
nand_bytes_written:100% sectors:7654576
host_bytes_written:100% sectors:6398903

Comparing the above two sets of data can be seen, nand_bytes_written/host_bytes_written are increasing, the former is the amount of physical space of the internal SSD, the latter can be considered as the total amount of write requests from the user, It can be considered that the ratio of the two increments over time is the write amplification factor in the time period.

NVMe Disk Write Amplification