Using SYSTEMTAP injection latency to simulate IO device jitter in Linux

Source: Internet
Author: User
Tags sleep systemtap

This behavior can be tricky when our IO-intensive applications suspect device IO jitter, such as a period of long wait times leading to performance or other problems, because the hardware jitter is infrequent and costly to reproduce.

Luckily, Systemtap can save us. In principle, the IO we use is accessed through the file system, regardless of read/write/sync, and most of our files are opened in buffered mode. In this mode, if the Pagecache is not hit, you need to access the device. With this basic principle in view, we can achieve this by using a controlled injection delay in the universal Systemtap to the VFS's read and write requests.

The main points are as follows:
1. Controlled points of time.
2. Delay time controllable.
3. Target equipment controllable.

I wrote a script to inject IO latency, simulate SSD/FIO hardware jitter to verify that the IO jitter will affect the application, three steps as follows:
Step 1: Compile the module

The code is as follows Copy Code

$ cat INJECT_KA.STP

Global inject, ka_cnt

Probe PROCFS ("CNT"). Read {

$value = sprintf ("%dn", ka_cnt);

}

Probe PROCFS ("inject"). Write {

Inject= $value;

printf ("Inject count%d, ka%s", ka_cnt, inject);

}

Probe Vfs.read.return,

Vfs.write.return {

if ($return &&

Devname = @1 &&

inject = = "Onn")

{

ka_cnt++;

Udelay ($);

}

}

Probe begin{

println ("IK module begin:)");

}

$ stap-v

Systemtap translator/driver (version 2.1/0.152, commit release-2.0-385-gab733d5)

Copyright (C) 2005-2013 Red Hat, Inc. and others

This is free software; The source for copying conditions.

Enabled Features:libsqlite3 NSS boost_shared_ptr tr1_unordered_map NLS

$ sudo stap-p4-dmaxskipped=9999-m ik-g INJECT_KA.STP sda6 300

Ik.ko

Where the parameter sda6 is the name of the target device, 300 is the time to delay, the unit us (more than 300 is prone to error, because usually Systemtap will be the script to perform the CPU to check, too much CPU will trigger the protection mechanism, causing STAP complained of exit), is usually sufficient for SSD devices.

This step generates Ik.ko, verify that the machine that generates the module and the target machine, the operating system version is identical, and make sure that your STAP version is higher because the Udelay function is available in a higher version of STAP.

Step 2:

Copy the Ik.ko to the target machine and execute

The code is as follows Copy Code

$ sudo staprun Ik.ko

IK module begin:)

Step 3:
After starting the application to start the test, run the following command to start the injection:

The code is as follows Copy Code

$ echo on|sudo tee/proc/systemtap/ik/inject && sleep && echo Off|sudo tee/proc/systemtap/ik/inject

Where Sleep N is the time when you want to open the injection switch.

Summary: Systemtap use good very invincible!

Have a good time!

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.