The Blktrace of Linux system analysis tools

Blktrace Introduction:

Blktrace is a tracking tool for the I/O layer of the Linux kernel, used to collect the details of the disk IO information when the IO is carried over to the block layer (block level, so called Blk Trace), which is the information of some columns such as IO request submission, queue, merge, completion, etc.
The maintainer of the Linux kernel block device layer has been developed and is now integrated into kernel 2.6.17 and its subsequent kernel versions. By using this tool, the user can obtain various details of the I/O request queue, including the process name, process number, execution time, and read/write physical
Block number, block size, and so on, is a good tool for analyzing I/O-related content under Linux

When observing IO behavior through blktrace, the first thing to do is to select the target device in order to analyze the IO behavior of the device. The blktrace is divided into kernel and application parts, and the application receives the list of devices we want to capture and passes them to the kernel. The cores are distributed across the block layer
Tracepoint will begin to work, passing the relevant data through the RELAYFS to the Blktrace Application section, which is used to write the data to disk for subsequent analysis.

Blktrace Frame Composition Reference:

http://blog.yufeng.info/archives/2524

Block Layer Location Map reference:

http://blog.yufeng.info/archives/751

Blktrace is a tool that can display the IO details of a block, but his output information is too professional, it is difficult to understand, you can use the Blkiomon, BLKPARSE,BTT and other tools to view

Blktrace Working principle:

_IOWR (0x12,115,struct blk_user_trace_setup),& blk_user_trace_setup three parameters), resulting in a system call to the kernel to invoke the corresponding function to handle, The kernel writes data to this file descriptor via the Debugfs file System (3) Blktrace needs to be used in conjunction with Blkparse, and Blkparse to parse the Blktrace-generated binary data in a particular format (4) Blkparse only open blktrace generated files, take data from the file to do the display and finally do the statistics output per CPU, but the data displayed in Blkparse (such as a,u,q, detailed see below) is Blkparse in T->action & 0xFFFF after himself
The values are converted to "a,q,u and other states" to show.

Blktrace Installation:

$ sudo apt-get install Blktrace

Debugfs Mount:

By blktrace working principle, blktrace need to use the kernel through the Debugfs file system (Debugfs file system in memory) to output information, so the Blktrace tool needs to mount the Debugfs file system before

$ sudo mount–t debugfs debugfs/sys/kernel/debug

Blktrace Syntax:

Blktrace-d Dev [-R Debugfs_path] [-o output] [-K] [-w time] [-a action] [-a action_mask] [-v]

Blktrace options:

-A hex-mask# set filter information mask into hexadecimal mask-a mask# add mask to current filter-B size# Specify cache size for fetch results, default to 512kb-d dev# add a device trace-I file#adds the de Vices found in file as devices to trace-k# kill the running trace-n num-sub# Specify the buffer pool size, default to 4 child buffers-  o file# Specify the name of the output file-r rel-path# the specified Deb Ugfs mount point-v# version number  -W seconds# Set run time

Output mode: file output:

$ blktrace–d/dev/sda–o Test1

  #对/DEV/SDA Trace, the output file is named Test1. Blktrace. [0-CPU number-1] (the file contains binary data, need to Blkparse to parse) (as mentioned in the Blktrace principle, each logical CPU has a thread, produces a file,
This results in a number of CPU files)

Terminal output:

$ blktrace–d/dev/sda–o-| Blkparse  -i–

  #输出到终端用 "-" said, but are a bunch of binary things, can't see, so need real-time blkparse to parse  #Blkparse "-i" after the file name, Blktrace output as "-" on behalf of the terminal (the code is written dead, is to use this symbol to represent the terminal), Blkparse also use "-" to represent the terminal resolution

A few examples:

Blktrace-d/dev/sda-o-|blkparse-i-This command prints the results of the blktrace to the screen, then blkparse the results of the blktrace on the screen as input to the analysis, and finally outputs the results of the analysis to the screen. It should be noted here that Blkparse is a blktrace-based analysis tool,
Because the blktrace itself does not have an analytical function, it is only monitored and the rest of the work is done by Blkparse.

blktrace-d/DEV/SDA |blkparse-i-This command prints the results of the Blktrace to a local folder, The file name is sda.blktrace.0 and sda.blktrace.1, here are two files because the machine has two CPU, blktrace based on the number of CPUs to generate files,
There is a corresponding monitoring data file for each CPU.

Blktrace-d/dev/sda-o Trace |blkparse-i-This command outputs the results of the blktrace to a previously specified file trace, note that the trace file must exist in the local folder without any suffix. After running, two new files are generated called trace.blktrace.0 and trace.blktrace.1.

Blkparse-i Trace This command is the input of the trace file as a blkparse, and the results of blkparse are still output to the screen

 blkparse-i trace-o/root/trace.txt This command is the input of the trace file as a blkparse, while outputting the analysis results to/root/trace.txt this file for a more in-depth analysis, because