LTP test process

LTP Test Analysis
Date: 2008.10.29
Author: yinkaizhong (kaizhongyin@hotmail.com)

1. Ltp Introduction
LTP -- linut test project, LTP suite is a system test suite developed by the Linux test project. It develops a test Combination Based on the utilization rate of system resources to provide sufficient pressure for the system.
The stability and reliability of the system are determined through stress testing.
Stress testing is a destructive test, that is, the operation of the system under abnormal and overloaded conditions. It is used to assess how the system will run when the maximum load is exceeded. It is a test of the system's ability to withstand a certain load intensity under normal circumstances.
LTP test suite is used to test the Linux operating system for a long time, focusing on the workload related to the Linux User Environment (see references for more information about LTP ). Instead of proving defects.
Key points: Test selection, evaluation of system resource utilization, analysis of kernel code coverage, evaluation of final stress testing
More specific reference: http://www.linuxeden.com/html/front/20060728/24305.html
Http://www.ibm.com/developerworks/cn/linux/l-rel/index.html
Or online information
Here, we will mainly introduce the LTP structure, test method, selection of test combinations, tool introduction, Environment setup, where to modify, analysis of test results, and LTP operation process.

2. Ltp Structure
Download the ltptest source code package from http://ltp.sourceforge.net/

The directory structure of LTP is basically divided into the document directory (DOC), the test driver directory (PAN), the test Script directory (testscripts), and the test case library (testcase), testing command file directory (runtest), header file directory (include), library directory (LIB) and so on.

DOC: This directory is the location of the instruction file and help document. The content of LTP and each tool are described in detail in this directory.
Pan: This directory stores the test driver pan of the LTP test suite.
Testscripts: This directory stores a set of executable test scripts and different test scripts.
Testcase: This directory stores the source code of all test cases used in the LTP test suite.
Runtest: each file in this directory is a set of test case commands to be executed, each file for different aspects of the test.
Include: the header file directory of the LTP test suite, which defines the data structure and function structure of LTP.
Lib: library file required by the LTP test suite during runtime, which defines various functions of LTP.

Links between file structures:
----> Ltpstress. Sh in testscripts
----> Runtest (stress. Part1, stress. Part2, stress. Part3)
----> Running on the pan background
----> Various test cases in the final run of testcase

In runtest, Stree. part1, Stree. part2, Stree. test commands or scripts in Part3, such as mmstress, are derived from mmstress in testcase/bin, while mmstress in testcase/bin can execute scripts or commands from/testcase such as kernel, network, pounder21, commands and other source code compilation and generation,
Mmstress: kernel/MEM/mtest05/mmstress. c

In stress. part [N] (n = 1, 2, 3), how does one test the command?

These command files contain the tag of the test case and the test use with parameters. The format is as follows:

# Tag test case
Test1 test1-L 10
Mtest01 mtest01-M 20
Fork01 fork01

How Pan works: the LTP test suite has a dedicated test driver pan. The execution of specific test cases is called and executed by Pan, it can track orphan processes and capture test output information. It works like this:
Read the command line to be executed for the entry to be tested from a test command file, wait for the end of the test, and record the detailed test output. By default, Pan selects a command line randomly to run the task. You can specify the number of times the test is executed at the same time.
Pan will record the detailed output produced by the test in a complex format, but it does not sort and collect data. The data sort and collect statistics is completed by workers, explain is a testing result analysis tool that understands the pan output format and outputs it into a table
Form to summarize those tests for passed or failed.

3. Ltp Test Method

There are two phases of the test method: "initial test" and "stress test ".

Initial testing is a necessary condition for starting testing. Initial tests include the successful operation of the LTP test suite on hardware and operating systems that will be used for reliability. The driver script runalltest. Sh attached to the LTP test kit is used to verify the kernel. This script serializes a testing of a component package and reports all the results. You can also choose to run several instances in parallel. By default, this script is executed:

-File System stress test.
-Hard Disk I/O test.
-Memory Management stress test.
-IPC stress test.
-Sched test.
-Test the command function.
-Verification test of the system call function.

Stress testing can verify the robustness of a product during high system usage. As runalltest. sh supplement, specially designed a name named ltpstress. sh testing scenario, while using network and memory management, runs a wide range of kernel components concurrently, and generates high load on the testing system. Ltpstress. Sh is also part of the LTP test suite. This script runs similar test cases in parallel and runs different test cases in sequence to avoid intermittent faults caused by simultaneous access to the same resource or mutual interference. By default, this script is executed:

-NFS stress test.
-Memory Management stress test.
-File System stress test.
-Mathematical (floating point) testing.
-Multi-thread stress testing.
-Hard Disk I/O test.
-IPC (pipeio, semaphore) test.
-Verification test of the system call function.
-Network pressure test.

4. Select a test combination

The combination of the selected tests must put sufficient pressure on system resources. The four main aspects of the Linux kernel can affect the system response and execution time:
-CPU: the time used to process data on the machine's CPU (s.
-Memory: the time when the data is read and written from the real memory.
-I/O: the time when data is read and written from the disk memory.
-Networking: the time when data is read and written from the network.

The evaluation phase of system resource utilization usually requires multiple attempts to obtain a suitable test combination and get the expected level of utilization. In this evaluation process, the SAR tool should also be running. In the conclusion of evaluation operation, you should collect and evaluate the utilization levels of all four types of resources.

The specific test combination modification method.
Method 1:
In runtest, stress. Part1, stress. Part2, and stress. Part3.
For example, if you modify stress. Part1 to have such a test mem02, you can modify it to mem02-M 15 based on the source code of testcases/kernel/MEM/mem02.c, which means to test the memory of 15 MB.
You can also add or delete some tests in stress. Part1, stress. Part2, and stress. Part3,
For example
Remove swap partition in stress. Part3
# Swapoff01 swapoff01
# Swapoff02 swapoff02

# Swapon01 swapon01
# Swapon02 swapon02
# Swapon03 swapon03

Method 2:
The initial test or stress test mentioned above is the default test. If you want to test other or your own test cases, you need to modify the testcase
Makefile, because it can be seen through its makefile

Subdirs = 'ls */makefile | sed "s/makefile // G" | grep-V open | grep-V pounder | grep-V dots | grep-V kdump | grep- V realtime'

Sed's //: // G': You want to replace the colon in the text with a space.

Grep-V: displays all rows that do not match the specified mode.

That is to say, open, Pounder, dots, kdump, and realtime are not tested. You can choose or add your own test cases.

5. Tool Introduction
Gcov, lcove:
Analyze code coverage
SAR:
It helps us understand the usage of system resources, especially the memory and CPU usage,
It is one of the tools that Unix users should master.
In our test, the SAR tool takes a snapshot of the System Utilization every 10 seconds (of course the interval can be modified) and saves it to the result file.
How to use SAR. data generated by SAR:
Sar-u sar. Data // view CPU usage
Sar-r sar. Data // check memory and swap usage
........................
Top:
Similar to SAR, we use SAR here.

RSH:
You don't need to describe the usage. It mainly describes how to build a trusted host.
Http://blog.chinaunix.net/u/7121/showart_576215.html this is comprehensive

Http://gforge.loogson.com/forum/forum.php? Thread_id = 59 & forum_id = 53
We also wrote comprehensive information on our 211 server, but the details of special attention are/root/. rhosts permission 600. Because I wasted half a day on it.

SSH:
Build a trusted host
Http://gforge.loogson.com/forum/forum.php? Thread_id = 59 & forum_id = 53

# Aptitude install OpenSSH-server openssh-Client
1. A> change the/etc/hosts file http://download.zdnet.com.cn/software_zone/2007/0831/481970.shtml
# Vi/etc/hosts open the hosts file and change it as follows:
127.0.0.1 localhost. localdomain localhost
192.168.4.211 UB-Server
192.168.4.40 gengmj-Desktop
192.168.4.32 zjx-desktop // ip: IP address of the trusted host, zjx-desktop: username of the trusted host
B> change the host in the/etc/hosts. equiv file // etc/hosts. equiv to access the host without providing a password.
# Vim/etc/hosts. equiv
#/Etc/hosts. equiv: List of hosts and users that are granted "trusted" R
# Command access to your system.
#127.0.0.1 localhost. localdomain localhost
UB-server mpich
Debian mpich
GMJ-desktop mpich
Zjx-desktop mpich // The trusted zjx-desktop user logs on to the local machine as the mpich user
Festival-desktop mpich
Liu-desktop mpich
Yym-desktop mpich

2. generate an SSH key pair in node01.
$ Ssh-keygen-t rsa one-way carriage return
Generate a. SSH file,
$ LS-a check whether there is a. Ssh Folder/cluster/Server/
3. Enter the. Ssh directory.
$ Cd. SSH
4. Generate the authorized_keys File
$ CP id_rsa.pub authorized_keys
5. log out to the root directory.
$ CD ..
6. Establish a trusted connection
$ Ssh ub-server enter yes as prompted (three letters are required)
7. Set node02 (under the root directory of node02)
$ Ssh-keygen-t rsa generate. Ssh folder
$ SCP mpich @ UB-server:/home/mpich/. Ssh /*~ /. Ssh/copy the. Ssh folder on the UB-server to overwrite the local
# SCP root @ UB-server:/etc/hosts copy the hosts file on node01 to overwrite the local
$ SSH node01 enter Yes at the prompt and press ENTER
8. confirm that the machine's trusted connection has been established.
Run the following command on each node:
$ Ssh ub-Server
$ SSH gengmj-Desktop
Enter Yes at the prompt and press Enter. Finally, make sure you do not need to enter the password and have no prompt information to log on (except for "Last login: time and date" prompt information)

6. Establish the environment

Development Board: v1.2
CPU:
Kernel: For Linux 2.6.22.8, select IDE and NFS server support.
LTP version: 20080831 cross-Compilation
 
PC:
CPU: Intel (r) celeon (r) CPU 2.26 GHz
Kernel: 2.6.24-16-generic
LTP version: 20080831

Use a PC as a server

7. locations to be modified

Change 1:/ltp-full-20080831/testcases/Network/nfs
For nfs01, nfs01, nfs03 nfs04, and nfsstress under, rHost = 192.168.4.68 will be modified to test the remote host.
Nfs04: nfs04_create_file in row 139 -- absolute path
2./ltp-full-20080831/testcases/Network/rpc/basic_tests/rpc01
In rpc01, rHost = 192.168.4.68 will be modified to test the remote host

Modify 3:/ltp-full-20080831/testcases/kernel/MEM/mem02.c ---> # define memsize 15*1024*1024 #15 m

Modify 4: fork09: Open up to 800 files before creating a process

.................. And so on

8. Test Result Analysis

By default, the test results are stored in/tmp
Ltpstress. log --- record the relevant log information, mainly to test whether it passes (pass or fail)
Ltpstress. Data ---- days files recorded by the SAR tool, including CPU, memory, I/O, etc.
Ltpstress.611.output1 --- corresponding to stress. Part1, some output information of the test command
Ltpstress.611.output2 --- corresponding to stress. Part2, some output information of the test command
Ltpstress.611.output3 --- corresponding to stress. Part3, some output information of the test command

Average CPU usage: Sar-u-F ltpstress. Data
Average memory usage: Sar-r-F ltpstress. Data

Fail analysis:
Ltpstress. Log filters all fail to obtain the complete testcase of all fail.
The method is as follows: Use sort to sort fail items and use uniq to exclude repeated items and output them to a file:
Grep fail ltpstress. Log | sort | uniq> failcase.txt

In this case, the failcase.txt is the names of all fail testcase. Analyze the cause of case failure and draw a conclusion: whether it is a configuration problem or a stability problem (failure or success ). And add the statement to failcase.txt for ease of viewing.

9. Software operation process

1. Download the source code package

Development Board
Tar zxzf ltp-full-20050608.tgz
CD ltp-full-20050608
Make cross_compile = mipsel-Linux -;
Make install cross_compile = mipsel-Linux -;
Copy to/root directory of the Hard Disk File System

PC
Tar zxzf ltp-full-20050608.tgz
CD ltp-full-20050608
Make;
Make install;
Note: The ltp-full-20050608 must be the same path as on the Development Board, that is, under the/root directory

2. Test Run

Initial test
./Runltp-P-l/tmp/resultlog.20061222-D/tmp-O/tmp/ltpscreen.20061222-T 24 h

Stress Testing
./Ltpstress. Sh-M 35-s-t 72 h //-M: indicates that genload is constantly allocating and releasing 35 m memory,
-S: Use the SAR tool to record
-T: Test Time
3. Result Analysis
By default

CD/tmp
Sar-u-F ltpstress. Data // average CPU utilization
Sar-r-F ltpstress. Data // average memory usage
Grep fail ltpstress. log | sort | uniq> failcase.txt // Failure Analysis

Calculate the success rate

Stress Testing Reference results (DVR)
The test tool is ltpstress. the test cases are executed in parallel and serial mode.
Time 24 h
Average CPU usage: 99.99% (users: 33.21% systems: 66.78%)
Memory average utilization: 68%
Success rate: 96.17%
No serious system faults

10. Summary

High kernel code coverage. A small number of kernel running failures imply an increase in the reliability of the Linux kernel. However, the integrity of these test cases still needs to be analyzed and enhanced, if kernel developers and testers are diligent in submitting test defects they have discovered, this suite can help reduce regression defects found in Linux.