0x00 Linux Advanced Command
When we are familiar with certain basic commands, we tend to be exposed to some advanced command operations, or unfamiliar command operations.
0x01 Program Construction
General source code provided by the program installation needs to be configured, compiled, installed three steps;
Configuration:
① Dependencies to install software
② set the initialization information required for the installation of the program, such as the installation path, which components need to be installed
The ③ configuration is completed and a makefile file is generated for use in the second step
Compile: Is the source file to compile the link to the executable program;
Install: Do the work is much simpler, is to copy the resulting executable file to the configuration set the initial path;
1.1. Configuration
Query the available configuration options: #./configure--help
To configure the path: #./configure--PREFIX=/USR/LOCAL/SNMP
–prefix is the most commonly used option for configuration, setting the path of the program installation;
1.2. Compiling
compile with make compile: #make-F Mymakefile
[Email protected] Target file name
@^ all previous nominations, except copies
@+ all previous nominations, including copies
@< a precondition name
@？ All new pre-nominations to target documents
@* the base name of the target file
Compiling a dependent library
The non-standard library and header file paths that are dependent on the makefile compilation process need to be displayed as indicated:
Cppflags-i tag non-standard header file storage path
Ldflags-l marking non-standard inventory drop paths
Make-f mymakefile ldflags= '-l/var/xxx/lib-l/opt/mysql/lib '
cppflags= '-i/usr/local/libcom/include-i/usr/local/libpng/include '
g++ compiling
Application: Query macro to expand the intermediate file:
In the g++ compilation option, add the-e option, then remove the-o option and redirect to a file:
g++-g-e unixapp.cpp-i/opt/app/source > Midfile
1.3. Installation
The installation is much simpler, and it is to copy the resulting executable file to the initial path set at the time of configuration:
$make Install
0x02 Program Debugging2.1. Process debugging
GDB Program Interactive Debugging
GDB is a command-line-based, powerful program debugging tool published by the GNU Open source organization under the Unix/linux operating system.
GDB is an essential tool for a C + + programmer working under Linux.
Start GDB $g + +-G hello.cpp-o Hello
To debug an executable file: #gdb <program>
To debug a service program:
$gdb <program> <PID>
$gdb Hello 11127
The following is a brief description of the most basic commands from a complete commissioning process;
$GDB Programmer # Start GDB
>break Main # setting breakpoints
>run # running the Debug program
>next # single-Step debugging
>print Var1 # During the debugging process, we need to check the value of the current variable, print the value using the Print command
>list # Displays the source code at the current debug
>info B # Displays current breakpoint settings
At the same time, you need more efficient debugging: Common debugging commands will have single-character abbreviation, the use of abbreviations more convenient, at the same time, directly hit ENTER to repeat the previous command, which is very useful in single-step debugging;
Pstack Tracking Stack Space #pstrack <program-pid>
Strace is commonly used to track system calls and received signals when a process executes. User mode switch to kernel mode
Trace all system calls to the 28979 process and count the time spent on the system call, as well as the start time (displayed in the visual hour-in-seconds format), and finally the recorded result exists in the Output.txt file
#strace-o output.txt-t-tt-e trace=all-p 28979
See what the process is doing (the real-time output process executes system calls) #strace-P <process-pid>
2.2. Target file analysis
Nm --a list of symbols used to list the target files.
$NM Myprogrammer
The body segment-the segment of the executable code, the data segment-information or data that cannot be executed; BSS Segment--block beginning with symbol data
The NM command lists each symbol, whose value is represented by hexadecimal (the default behavior), and preceded by an encoded character that represents the symbol type. Common types of encodings include:
A represents absolute (absolute), which means that the value cannot be changed to a different connection;
B represents the symbol in the BSS segment;
C represents a generic symbol that refers to uninitialized data.
Objdump Displays information about binary files, which allows you to learn more about the additional information that binary files may have with a readable format
#objdump-D Myprogrammer
Common parameter Description
-F Display File header information
-D Disassembly of all sections (-D Disassembly specific section)
-H Displays header summary information for each section of the destination file
-X displays all available header information, including the symbol table, reposition entry. -X is equivalent to-a-f-h-r-T specified at the same time.
-I displays a list of schema and destination formats available for the-B or-M options.
-r Displays the file's relocation entry. If used with-D or-D, the relocation section is displayed in an disassembled format.
-r Displays the file's dynamic relocation entry, which is only meaningful for dynamic target files, such as some shared libraries.
-S to disassemble the source code as much as possible, especially when the compiler specifies the debug parameter-G, the effect is more obvious. The-d parameter is implied.
-T displays the symbol table entry for the file. Similar to the information provided by Nm-s
View native target structure (with big or small end storage): $objdump-I
Disassembly Program: $objdump-D MAIN.O
Display symbol table entry: $objdump-T MAIN.O
Readelf
$readelf-all a.out
$readelf--debug-dump a.out | More
Size View Program memory consumption
#top
#whereis FCITX
#size/USR/BIN/FCITX
File type Query
$file core.22355
Strings querying the text information in the data
A file contains binary data and text data, if you only need to view its text information, it is convenient to use this command; filter out non-character data and output text information: $strings <objfile>
Displays all process information that is using the specified file, file system, or sockets;
Fuser Show file Users
Displays all process information that is using the specified file, file system, or sockets;
$fuser-M-u redis-server
Fuser is commonly used in the "resource busy" problem of diagnostic systems. If you want to kill all processes that are using a specified file system or sockets, you can use the-K option:
$fuser –k/path/to/your/filename
XXD Hexadecimal Display data
#xxd/ROOT/DESKTOP/1
Od
You typically use the OD command to view file content in a special format. You can display files in decimal, octal, hexadecimal, and ASCII codes by specifying different options for the command.
Parameter description:
-a specifies the address cardinality, including:
D Decimal
o Octal (System default)
X hex
n Do not print displacement values
-t specifies the display format of the data, and the main parameters are:
C ASCII character or backslash sequence
D Signed Decimal number
F Floating point number
o Octal (system default is 02)
U unsigned decimal number
x hexadecimal number
$od-AX/ROOT/DESKTOP/1
0x03 Performance Optimization
Optimize performance from the system level and program level.
3.1. Analyzing system bottlenecks---top command
Slow system response: IO Bottlenecks, CPU bottlenecks, memory bottlenecks, system problems caused by programs
Using the top tool gives you a more comprehensive view of the points we're following: #top
After entering interactive mode:
Enter m, the process list is sorted in descending order of memory usage, which allows us to observe the problem of maximum memory usage (detection of memory leaks);
Enter P, the process list is sorted by CPU size in descending order, so that we could see if there is a problem with the user who consumes the most CPU resources;
The top third line shows the current system, where two values are critical:
%id: The percentage of idle CPU time, if this value is too low, indicates that the system CPU has a bottleneck;
%wa: The percentage of CPU time waiting for I/O, if this value is too high, indicates that IO has a bottleneck;
3.2. Analyze memory Bottleneck--free command
One using the free command
#free-MH
Two using the Vmstat command
Vmstat is virtual meomory real-time dynamic monitoring of the operating system's fictitious memory, process, CPU activity.
Vmstat [-v] [-n] [delay [count]]
-V indicates that the version information is printed;
-n means that the output header information is displayed only once during cyclic output;
Delay is the lag time between two outputs;
Count refers to the number of times that are counted at this interval.
/root$vmstat 5 5
procs-----------------memory-----------------------swap-------------io----------system-----------CPU-----
R b swpd free buff cache si so bi bo in CS us sy ID WA St
0 0 5524 172304 178400 969340 0 0 8 29 50 146 1 0 98 1 0
0 05524 172252 178400 969348 0 0 0 0 378 1089 19 3 78 0 0
3 05524 170888 178412 969352 0 0 0 31 636 1460 38 6 56 0 0
Procs MemorySwap System:
R: Number of processes in the running queue SWPD: Using virtual memory size SI: The size of writes from swap to memory per second In: Number of interrupts per second, including clock interrupts
B: Number of processes waiting for IO Free: Available memory size so: The amount of memory written to the swap area per secondCS: Number of context switches per second
Buff: Memory size IO as buffer: Now the size of the Linux version block is 1024bCPU (expressed as a percentage)
Cache: Memory size as Cached BI: number of blocks read per secondUS: User Process Execution Time
Bo: Number of blocks written per second SY: System Process Execution time
ID: Idle time (including IO wait time)
WA: Waiting for IO time
3.3. Analyzing IO Bottlenecks
If the IO has a performance bottleneck, the%wa in the top tool will be high;
Further analysis using the Iostat tool: root$iostat-d-x-k 1 1
If the value of%iowait is too high, the hard disk has an I/O bottleneck.
If%util is close to 100%, it indicates that there are too many I/O requests, the I/O system is full, and the disk may have bottlenecks.
If SVCTM is closer to await, I/O has almost no waiting time;
If the await is much larger than SVCTM, stating that the I/O queue is too long and the IO response is too slow, the necessary optimizations need to be made.
If the Avgqu-sz is larger, it also indicates that there is a large amount of Io waiting.
3.4. Parsing process calls--pstack and Pstrace
Pstack is used to track the process stack, which is useful for troubleshooting process problems
View Bash program process stack:/opt/app/tdev1$ps-fe| grep bash
Strace is used to track system calls in the process, which dynamically tracks system calls and received signals while the process is executing. is a very effective tool for testing, directing and debugging. This command allows system administrators to easily troubleshoot program issues.
$strace Cat/dev/null
Command Instance 1:
Trace executable Program
Strace-f-f-o ~/straceout.txt MyServer
The-F-F option tells Strace to track both the fork and the vfork process, and the-o option writes all strace output to ~/straceout.txt, which is the program to start and Debug.
Tracking Service Program
Strace-o output.txt-t-tt-e trace=all-p 28979
Trace all system calls (-e Trace=all) of the 28979 process and count the time spent on the system call, as well as the start time (and display in the visual hour-in-seconds format), and finally the recorded result exists in the Output.txt file.
3.5. Optimizing Program Code
Finish coding, and then optimize
Focus on optimizing the 20% most time-consuming code
Gprof Use steps
When compiling a program with GCC, g++, xlc, use the-PG parameter, such as: g++-pg-o test.exe The Test.cpp compiler automatically inserts a snippet of code for performance testing in the target code that collects and records the call and invocation times of the function while the program is running, and records the execution time of the function itself and the execution time of the called function.
Executes the compiled executable program, such as:./test.exe. This step runs the program slightly slower than when the normally compiled executable program runs. After the program runs, a file with the default file name of Gmon.out is generated under the path of the program, which is a data file that records the performance of the program, the call relationship, the number of calls, and so on.
Use the GPROF command to analyze the Gmon.out file for logging program run information, such as: Gprof test.exe gmon.out You can see statistics and analysis information about function calls on the monitor. The above information can also be redirected to a text file using Gprof test.exe gmon.out> gprofresult.txt for subsequent analysis.
3.6. Other tools
Debug memory Leak tool Valgrind, interested friends can Google understand;
A powerful performance analysis tool on the Oprofile:linux platform, using the reference [F2];
In addition to the tools described above, there are a number of more comprehensive performance analysis tools, such as the SAR (the Linux system is not installed by default, requires manual installation), the SAR's resident monitoring tool can be opened to collect more comprehensive performance analysis data;
: 0x04 llsof All DocumentsList open files is a tool to view the current system files. In a Linux environment, everything is in the form of files, with files that not only access regular data, but also access to network connectivity and hardware. such as Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) sockets, the system assigns a file descriptor to the application in the background, which provides a lot of information about the application itself
Lsof open files can be:
Normal file
Directory
File for Network File system
Character or device file
(function) Shared library
Pipes, Named Pipes
Symbolic Links
Network files (for example: NFS file, network Socket,unix domain name socket)
There are other types of files, etc.
Command parameters
-a lists the processes that exist for open files
-c< Process name > List files opened by the specified process
-G list GID process details
-d< File Number > list the process that occupies the file number
+d< directory > List files that are open under directory
+d< directory > recursively list files opened in directory
-n< directory > List files that use NFS
-i< conditions > lists the processes that meet the criteria. (4, 6, protocol,: Port, @ip)
-p< Process number > List files opened by the specified process number
-U list UID number process details
-H Display Help information
-V Display version information
Example 1: No Parameters $lsof| More
The meaning of the lsof output column information is as follows:
COMMAND: Name of the process
PID: Process Identifier
PPID: Parent Process Identifier (the-R parameter needs to be specified)
USER: Process Owner
Pgid: The group to which the process belongs
FD: File descriptor in which the application recognizes the file through a file descriptor. such as CWD, TXT, etc.:
(1) CWD: Represents the current working dirctory, which is the directory in which the application starts, unless it makes changes to the directory itself
(2) TXT: This type of file is a program code, such as the application binary file itself or a shared library, as shown in the list above/sbin/init program
(3) Lnn:library references (AIX);
(4) ER:FD information error (see NAME column);
(5) Jld:jail directory (FreeBSD);
(6) Ltx:shared Library text (code and data);
(7) Mxx:hex memory-mapped type number XX.
(8) M86:dos Merge mapped file;
(9) mem:memory-mapped file;
(ten) mmap:memory-mapped device;
(one) Pd:parent directory;
(rtd:root) directory;
(Tr:kernel) trace file (OpenBSD);
(+) v86 vp/ix mapped file;
(15) 0: Indicates standard output
(16) 1: Indicates standard input
(17) 2: standard error indication
Generally after standard output, standard error, standard input followed by file state mode: R, W, u, etc.
(1) U: Indicates that the file is open and in read/write mode
(2) R: Indicates that the file is open and is in read-only mode
(3) W: Indicates that the file is open and is in
(4) Space: Indicates that the file's status mode is Unknow and is not locked
(5)-: Indicates that the file's state mode is unknow and is locked
At the same time, after the file state mode, followed by the relevant lock
(1) n:for a Solaris NFS lock of unknown type;
(2) R:for read lock on part of the file;
(3) R:for A read lock on the entire file;
(4) W:for a write lock on part of the file;
(5) W:for A write lock on the entire file; (write lock for entire file)
(6) U:for a read and write lock of any length;
(7) U:for a lock of unknown type;
(8) x:for an SCO openserver Xenix lock in part of the file;
(9) x:for an SCO openserver Xenix lock on the entire file;
(space:if) There is no lock.
Type: File types, such as Dir, Reg, etc., common file types:
(1) DIR: Indicates directory
(2) CHR: denotes character type
(3) BLK: Block device type
(4) Unix:unix domain sockets
(5) FIFO: Advanced First Out (FIFO) queue
(6) IPv4: Internet Protocol (IP) sockets
DEVICE: Specifies the name of the disk
Size: Sizes of files
Node: Index node (the identity of the file on disk)
Name: Open the exact name of the file
Find a file-related process $lsof/bin/bash
Example 3: List file information opened by a user $lsof-u username
Example 4: File information opened by a program process $lsof-c MySQL
Example 5: List the file information opened by a user and a process $lsof-u test-c MySQL
Example 6: Display the file opened by the process by a process number $lsof-p 11968
Example 7: List all network Connections $lsof-i
Example 8: List all TCP network connection information $lsof-i TCP $lsof-N-I TCP
Example 9: List who is using a port $lsof-I: 3306
Example 10: List all active network ports for a user $lsof-a-u test-i
Example 11: Listing the corresponding file information according to the file description $lsof-d description (like 2)
Example 12: List all IPV4 network files that are open by processes with process number 1234
$lsof-I 4-a-p 1234
Example 13: List all file information that is currently connected to host NF5260I5-TD Port: 20,21,80, and repeat every 3 seconds
Lsof-i @nf5260i5-td:20,21,80-r 3
Welcome everyone to share a better idea, eagerly looking forward ^ ^_^ ^!
Advanced Linux Commands
