Monitoring of inter-process communication in Embedded Systems

This article describes in detail a monitoring method that uses the ptrace system call to implement process communication within the embedded system, and provides corresponding implementation solutions.

LinuxInptraceSystem calling is the key to monitoring inter-process communication.ptraceIt provides us with a way to observe and control other processes. ExploitationptraceWe can intercept all system calls of running processes. Interception means that the monitoring program can obtain system call parameters or even modify parameters when these system calls occur or exit. These system calls include:read,write,sendto,recvAnd so on. InLinuxYou canman syscallsTo view the current versionLinuxSupported system calls.

In ourLinuxEmbedded products,AF_UNIXDomainsocketIt is widely used. It is used to complete inter-process communication.AF_UNIXDomainsocketProgramming ModelsocketThe programming model is identical. We use the following method: the receiving process createsAF_UNIXDomainsocket, Set its mode to datagram (SOCK_DGRAM). After that, bind a file name with a path to it, for example:/var/tmp/receive.unix. The file name is used by the kernel to identify the socket. The sender createsAF_UNIXDomainsocket. Then, callsendtoSend messages to the receiving process. Used to identify the receiving processsocketIs the file name mentioned above, that is/var/tmp/receive.unix. While the receiving process usesrecvfromThe system calls the message to receive the message sent by the sending process.

ThereforeptraceOnce we take over the monitored processsendtoAndrecvfromSystem calls enable us to intercept data that uses these two system calls for communication.

ptraceSystem Call is defined as follows:

#include <sys/ptrace.h> long int ptrace(enum __ptrace_request request, pid_t pid, / void * addr, void * data);

 

It has four parameters.requestDetermined by the valueptraceWhat kind of task is executed.pidSpecifies the process to be trackedid.requestThe parameter determines whether a validaddrParameter, or use onlyNULLYou can. If necessary, use validaddrParameter, which indicates the offset of the process space of the tracked process.dataSimilaraddrParameter.NULL. If it is used, it means to point to some data that is to be placed in the user space of the monitored process.

A complete sample code will show us the technical details and key points of monitoring inter-process communication. The code is segmented according to the sequence.

#include <stdio.h> #include <stdlib.h> #include <sys/ptrace.h> #include <sys/wait.h> #include <linux/user.h> #include <sys/socket.h> #include <sys/un.h> #include <linux/net.h>

 

To useptraceSystem Call, we need to addptrace.hHeader file. To obtain the intercepted system call function input parameters, we need to usestruct user_regs_struct Structure. It is inuser.h. Because the signal is used in the program, we also needwait.h . We want to monitor communication actions,socket.hAndun.hIt is essential.

The main function of the program entry is as follows:

int main (int argc, char *argv[]) { int status; int syscall_entry = 0; int traced_process; struct user_regs_struct u_in;

 

statusUsed to record status changes of monitored processes;syscall_entryRecord whether the monitored process is currently in the system call or returned from the system call;u_inUsed to obtain the intercepted System Call parameters;traced_process The process is monitored.PID Value.

Traced_process = atoi (argv [1]);/* obtain the PID of the monitored process from the command line */ptrace (ptrace_attach, traced_process, null, null); wait (& status ); /* Wait for the status change of the monitored process */ptrace (ptrace_syscall, traced_process, null, null );

 

The parameter isPTRACE_ATTACHOfptraceModify the process structure of the monitored process in the kernel. Make the monitored process a child process of the current program. Once the status of the monitored process changes,wait()Will return. Program call againptrace. This time the parameter isPTRACE_SYSCALL. The process structure of the monitored process is modified again.traceThe flag is activated. The kernel will trigger the running of the current program during every system call of the monitored process.

While (1) {/* Wait for the Monitored Program to call the system or make other status changes */Wait (& status);/* If the monitored process exits, the function returns true. Program exit */If (wifexited (Status) break; ptrace (ptrace_getregs, traced_process, 0, & u_in); If (u_in.orig_eax = 102 & u_in.ebx = sys_sendto) {If (syscall_entry = 0) {/* syscall entry */insyscall = 1; printf ("Call sendto ()/n ");} else {/* syscall exit */syscall_entry = 0 ;}} ptrace (ptrace_syscall, traced_process, null, null);}/* While */return 0;}/* main */

 

Monitored ProcesstraceAfter the flag is activated, each system call is checked by the kernel. Our program is also notified by the kernel signal. Parameters UsedPTRACE_GETREGS Ofptrace()Will getInterceptedSystem Call parameters. The most important parameter is the system call number. It is saved inu_in.orig_eax .By using the system call number, we can determine the system call.The system call number can be found inLinuxIn the source code. It is defined inlinux-source-2.6.xx/arch/x86/kernel/syscall_table_32.S.Some of its code is as follows:

.long sys_fstatfs /* 100 */ .long sys_ioperm .long sys_socketcall .long sys_syslog

 

Here, we are most concerned with sendto system calls. In the Linux kernel, the real endpoint of sendto is the socketcall system call. It is the entry for socket-related system calls such as bind and sendto. In this system call, a call number is used to distinguish different subsystem calls, such as bind and sendto. In our program, this call number is saved in u_in.ebx. From the sample code of syscall_table_32.s, we can see that the system call number of socketcall is 102 (from 100 to two rows ). The call number is defined in net. h. The call number of sendto that we care about is defined as sys_sendto, and its absolute value is 11. With these two important data, our program determines whether the current system call is sendto. This is represented by the Code:

if (u_in.orig_eax == 102 && u_in.ebx == SYS_SENDTO)

 

When the monitored process enters the system call or exits the system call, it will trigger the wait () Return, giving our program a chance to run. Therefore, we need to use syscall_entry to record whether the monitored process enters the system call or exits the system call at the current time. This is a switch type, which is easy to understand. Finally, after each processing, you must call ptrace with the ptrace_syscall parameter again to monitor the next system call.

Although the above program is very simple, it can be fully used.ptraceIntercepted by the monitored processsendtoThe process of system calling. It is worth adding thatptraceYou can also obtainsendtoData to be sent.

sendtoSystem Call is defined:

#include <sys/types.h> #include <sys/socket.h> size_t sendto(int s, const void *msg, size_t len, int flags, / const struct sockaddr *to, socket len_t tolen);

 

sendtoContains six parameters, especiallymsgThe parameter indicates the content of the sent data. ParameterstoIndicates the Sending target. ExploitationPTRACE_PEEKDATAParameterptrace, The monitoring program will be able to obtainsendtoAll six parameters. In this way, the monitoring program completely obtains the data to be sent and the target to be sent by the monitored process. The specific implementation details will not be discussed here. Seeman ptraceInstruction manual.

 

Back to Top

 

System and Application of Monitoring System

Using the technology discussed above, we developedmipsThe monitor program on the target board, namedipcmsg. It is a command line program. In our application environment, it is used as follows:

root@host:~$ ipcmsg -p pid -l xxx.xxx.xxx.xxx -b 6000

 

pid The process is monitored.pid, You can usepsCommand.-lSpecifyPCHostIPAddress.-bThe parameter indicates the port number to receive.

During initial monitoring,ipcmsg Yes NoIPAddress and port number parameters. All information is output to the serial port console. This affects both the running efficiency (a large number of output on the serial port will affect the running speed of the target board) and information processing. Because our target board has Ethernet interfaces, we can easily thinkipcmsgThe intercepted packets are forwardedPCHost. UsePCThe host makes it easier to analyze data packets for inter-process communication. InPCOn the host, we usewiresharkThis popular open-source network packet analysis software receives information from the target board. Shows the architecture of the entire monitoring system:

Figure 1 Architecture

In actual use, we use Ethernet cablesPCHost connection. Then, start on the target boardipcmsgAnd specifypid .ipcmsgAfter running, we start it on the PC hostwiresharkReceive fromipcmsg. These data packages containmipsInformation about inter-process communication on the target board. We useipcmsgSpecially developedwiresharkPlug-ins inwiresharkOn, We can detail the decompositionipcmsgIt intuitively analyzes the process of inter-process communication and possible problems. Below iswiresharkDecompositionipcmsgThe actual running diagram of the data packet:

Figure 2 operation diagram

We can see fromipcmsgObtained the method of inter-process communication, parameter (pathYesAF_UNIXDomainsocketAddress parameter), direction, content, and process name. This information helps us analyze the running status of embedded systems. It is very intuitive and easy to operate.

Overview

In complex embedded systems, a considerable number of processes are often run at the same time. These processes frequently communicate with each other. The running status of a process is directly and closely related to these ongoing communications. By monitoring inter-process communication, developers can control the internal running status of the system. When an error is found, the debugging engineers can easily find the problem by using the obtained inter-process communication information.

However, the interfaces between embedded systems and developers are usually relatively simple. Developers usually use terminals based on serial ports or network interfaces (console. In this mode, it is difficult for developers to precisely observe inter-process communication. Moreover, for Embedded Systems with weak computing capabilities, printing a communication packet on a terminal will not only affect the operation of the system, but also cause excessive useless information on the screen, this makes analysis impossible for developers.

To solve this problemLinuxOn the platform, we developed a complete set of software for monitoring inter-process communication in the embedded system, used to debug our developed embedded products. This article describes in detail the technical principles of inter-process communication monitoring in embedded systems and the recommended solution for implementing monitoring software.

 

Basic principles of monitoring methods