Application layer packet capture solution and implementation

Application layer packet capture solution and implementation

Author: Fang

Introduction

Packet Capture needs generally come from filtering, conversion protocol, and packet capture analysis.

Filters have many applications, typically packet filter firewall.

The application of the conversion protocol is limited to some specific environments. For example, if a third party develops network protocol software and cannot integrate it with the original operating system software, it has to adopt the "embedded protocol stack block" (BITs) method. For example, the third-party implementation of IPSec on Windows cannot be integrated with the IP software provided by the operating system vendor, so it must be implemented between the IP layer and the link layer as a layer of the protocol stack. Third-party pppoe software is also implemented in this way.

The purpose of intercepting a packet is to use "packet capture" to describe more appropriate. "packet capture" generally indicates that the packet has the ability to be truncated. "packet capture" only needs to be obtained. The implementation is generally implemented as the protocol layer.

The "application-layer packet capture" mentioned in this article is specifically used in the driverProgramTo the application layer.

Packet Capture Mode

Network packet interception methods in user mode are as follows:

1. Winsock layered service provider;

2. Windows 2000 package filtering interface;

3. Replace the Winsock dynamic connection library that comes with the system;

You can use the driver to intercept network data packets by using the following methods:

1. TDI filter driver)

2. NDIS intermediate layer driver (NDIS intermediate driver)

3. Win2k filter-hook driver

4. NDIS Hook driver

The interception of data packets in the user mode has some limitations. "Obviously, the most fatal disadvantage of packet interception in the user mode is that it can only be performed at the Winsock level, however, data packets of the underlying protocol in the network protocol stack cannot be processed. For some Trojans and viruses, it is easy to avoid this layer of firewall ."

The "Application layer packet capture" does not refer to the user-mode packet interception described above. It is intercepted in the driver and processed at the application layer. To obtain a common method, you must intercept it at the IP layer. This article uses the intermediate layer mode for comprehensive comparison.

Why do we need to process intercepted packets at the application layer?

Generally, network applications such as firewalls and protocol software work in the kernel. Why should we, in turn, propose to process packets at the application layer? The reason can also be found out (even more far-fetched ):

As we all know, driver development is difficult. For an experienced programmer, there may be no technical problems in the development process, but for beginners, especially the first time I came into contact with programmers, it was simply a painful experience.

In addition, the development cycle is also an issue that must be considered. The program runs in the kernel, and the stability and compatibility all require a large number of tests, and the available function libraries are quite small compared to the application layer. In application layer development, debugging and modification are much easier.

Unfavorable factors include:

Performance impact: working at the application layer changes the working mode. Whenever the driver captures data, it is sent to the application layer for processing and then sent back to the kernel, and then to the IP protocol. Therefore, the performance is greatly affected and the efficiency is very low. On a 80% Mbps network, only of the performance is achieved.

In summary, it is suitable for specific applications:

When used on a desktop, the network load on the desktop is quite small, and less than 512 Mbps is enough to meet the requirements, especially for Internet access and other environments. The network connection traffic is less than kbps, so there is no need to consider performance factors. As a single-host firewall or some other protocols, analysis and so on can be easily implemented based on this method.

Solution

Packet receiving process:

1. The Network Interface receives the packet, the middle layer intercepts the packet, and 2 sends the packet to the application layer for processing;

2. After processing at the application layer, return the intermediate layer processing result;

3. The middle layer discards the packet based on the processing result, or sends the processed packet to the IP protocol through 1;

4. The IP protocol and upper-layer applications receive packets;

Packet sending process:

1. Upper-layer applications send data to send packets through IP protocol;

2. The message is intercepted by the middle layer and sent to the application layer for processing through 2;

3. After processing at the application layer, return the intermediate layer processing result;

4. The middle layer discards the message based on the processing result or sends the processed message to the network;

Implementation Details

Io and communication

There is an easy way to use an event between the driver and the application.

When the application createfile is used, the driver iocreatesynchronizationevent is a famous event, and then the application createevent/openevent is a famous event.

Note:

1. Do not create events during driver initialization. At this time, most events cannot be created successfully;

2. If the driver is created first, the application can only read (waitxxxx) but not write (setevent/resetevent) when it is opened later ). If an application is created first, both the application and the driver have the read and write permissions;

3. The name is ideal. Note that the driver name is under \ basenamedobjects \. For example, if the application uses "xxxevent", the driver is "\ basenamedobjects \ xxxevent ";

4. You can use handle, but it is unknown whether it is feasible in Win98.

5. After that, the driver should immediately return the Read Request; otherwise, the return fails. Otherwise, the meaning of event notification will be lost (read is not waiting to be completed, but will be read only when there is a need (Notification event );

6. When an application finds an event, it should read it in a loop until the reading fails, indicating that no data is readable; otherwise, subsequent data is missed and not read in time;

Processing thread priority

The Application Layer processing thread should increase the priority because the thread serves other upper-layer applications. If the priority is lower than the priority of other threads, a wait state similar to a deadlock will occur.

In addition, when raising the priority, you must note that the running time of the thread should be shortened as much as possible, and the CPU should not be used for a long time; otherwise, other threads will not be able to get the service. The priority does not need to be raised to the realtime_priority_class level. In this case, the thread cannot perform operations such as disk Io, but also affects the mouse, keyboard, and other work.

The driver can also dynamically increase the thread priority.

Cache

After the driver receives the packet, there should be at least one buffer for temporary storage, waiting for processing at the application layer. The buffer does not need to be large. As long as the buffer does not overflow before the application layer obtains the time slice, it is enough to store dozens of packets in practice.

The buffer usage method is a first-in-first-out queue. To facilitate the implementation of a ring queue for static storage, you do not need to allocate memory each time. Instead, you need to allocate a large block of memory at a time to use the ring.

Initial, head = tail = 0;

Both tail and head are infinitely increasing.

Tail-head <= size;

When a packet is placed, tail = tail + packetlen;

Head = head + packetlen;

Tail = head indicates NULL;

Tail> head indicates that there is data;

Tail + input packet length-head> size indicates full;

When retrieving data:

Ppacket getpacket ()

{

Assert (tail> = head );

If (tail = head)

Return NULL;

// Else

Ppacket = & start [head % size];

If (Head % size + ppacket-> length> size)

// Data is not continuous (some are at the end and some are at the header );

Else

// The data is continuous.

Return ppacket;

}

Put data:

Bool inputpacket (ppacket)

{

If (tail + input packet length-head> size) // full

Return false;

// Copy packet to & start [tail % size]

// If (tail % size + packet length> size)

// Data is not continuous (some are at the end and some are at the header );

// Else

// The data is continuous.

Tail = tail + packet length;

Return true;

}

The preceding method uses arrays to provide a maximum packet length space for each packet. Because the number of buffers is limited, this method can meet the needs. If you want to reduce the waste of space, you can store the data according to the actual length of each message.AlgorithmThis method cannot be adapted.

Communication between application layer and driver

When the NIC receives or sends an IP address, the driver caches the message and notifies the application layer of the message to be processed. Then, the application layer can obtain this message through Io or shared memory.

Practice shows that the above two methods can meet the needs at a rate of Mbps, And the easiest way is to use the buffer I/O method.

After the application layer completes processing, you can use either of the above two methods to submit the results to the driver. However, because I/O can only send one packet at a time, the network speed of 70% Mbps is reduced ~ 80% network speed, 10 Mbps will not be affected. That is to say, the maximum speed sent by the host is only 70% of the network speed, which is the same as the speed at which the application sends UDP datagram up to MTU. For TCP, due to two-way communication, the loss is greater, about 40% ~ 60% speed.

In this case, the shared memory mode is used to reduce the overhead of system calls and avoid the speed decrease.

Speed Control of message transmission

When the IP protocol sends packets, in general, our middle-layer driver must cache these packets, tell the IP software to send successfully, and then let the application layer make a decision after processing. Obviously, the speed of storing packets far exceeds the speed that the network adapter can send. However, IP software (especially UDP) will send packets at the speed that we store. The cache is quickly exhausted. The subsequent messages have to be discarded. In this way, UDP transmission will not work properly. Because TCP can adapt to the network conditions on its own, it can still work in this case, and the speed is around 70%. In passthru, it can be forwarded to the low-layer driver and then returned asynchronously or synchronously, so as to achieve the same sending speed of the NIC.

Therefore, there must be a way to avoid this situation. The middle layer driver caches these packets and tells the IP software that the sending status is pending (pending ). After the final processing is completed, the IP address software will be notified of the completion of the sending. This coordinates the sending speed. This approach brings about a problem that the driver must give up ownership of these buffered packets when the sending times out. Specifically, when the miniportreset is called, it may be that NDIS detects the sending timeout, thus giving up all the unfinished sending operations. If this situation is not properly handled, this will cause serious problems. If the middle layer sets the ndis_attribute_ignore_packet_timeout flag by calling the ndismsetattributesex function during miniport initialization, the middle layer driver will not receive message timeout notifications, and the middle layer must process cached packets by itself.

Work with passthru

When Upper-layer applications no longer require packet capture, the driver should be completely Passthru. This requires that all the sending/receiving functions should correctly handle the packet capture and non-packet capture statuses, so as not to cause harmful behavior.

Specifically, all the receiving/sending functions are correctly processed in four directions: receiving/sending data packets to the IP protocol and sending packets to the IP protocol.

Other auxiliary facilities

Add some control functions to provide more fine-grained control and allow more freedom for applications. For example, you can control which Nic to intercept, the traffic to intercept in a certain direction, and whether the network is changed (NIC unmount/disable.

Implementation

Implement passthru SelectionSource code, And the main modifications include:

1. Modify the receiving function

2. Modify the sending Function

3. Added message Cache

4. Add Io

5. added the control function.

6. added subsequent processing after application layer Processing

This implementation uses the shared memory mode, which has a buffer pool before processing and a buffer pool after processing by an application. Because the same buffer pool is used for receiving messages and to-be-sent messages, this implementation is not much more efficient than Io.

Through careful design and comparison, you can achieve a transmission speed of Mbps.

This copyArticleThe purpose is to discuss the working method and feasibility of this application layer interception. Because of the driver sourceCodeIt has not been strictly tested and is not suitable for commercial use. As a demonstration, it only blocks Ethernet packets. Therefore, the demonstrated driver will not contain the source code.

API description

Third-party development uses the CAP. h header file. capdll. dll contains the following functions:

Bool capinitialize ();

Void capuninitialize ();

Bool capstartcapture (pktproc packetproc, adapters_change_callback adaptchange );

Void capstopcapture ();

DWORD capgetadaptlist (padapt_info padaptinfo, DWORD buffersize );

Void capsetrule (handle adapter, ulong rule );

Bool capsendpacket (handle adapter, ulong opcode, ulong length, puchar data );

At the same time, the capdll. Lib file of the DLL is provided so that capdll. lib can be introduced into the VC project file to facilitate compilation and connection.

Description

The returned values of all functions do not indicate the cause of the error. The debug version can print the running information on the console and have the same output information in C: \ capture.txt.

Bool capinitialize ();

Note:

The middle layer driver of the notification interception report performs some necessary initialization work.

Parameters:

None.

Return Value:

False is returned for failure.

Void capuninitialize ();

Note:

Release the events, threads, and memory created by the driver.

Parameters:

None.

Return Value:

None.

Note:

Before calling this function, you should call capsetrule to set the driver interception rule to opcode_passthru to restore passthru behavior.

Bool capstartcapture (pktproc packetproc, adapters_change_callback adaptchange); Description:

Enable report capture. Capdll creates a thread that runs at the thread_priority_highest priority and waits for network events. When a driver receives a packet or sends a packet through the IP protocol, you can also find that the NIC is enabled, disabled, inserted, or removed, and the user will be notified through the callback function provided by the user.

Parameters:

Packetproc: User-provided message processing function;

Adaptchange: The network change notification function provided by the user;

Return Value:

Void capstopcapture ();

Note:

Stop intercept. Destroys the created thread.

Parameters:

None.

Return Value:

None.

DWORD capgetadaptlist (padapt_info padaptinfo, DWORD buffersize );

Note:

Obtain the network adapter list.

Parameters:

The padaptinfo adapt_info array provides sufficient space.

Buffersize buffer size.

Return Value:

Number of network adapter cards.

Void capsetrule (handle adapter, ulong rule );

Note:

Set the alarm rule.

Parameters:

Adapter: Specifies the intercepted Nic handle.

Rule: opcode_passthru: passthru behavior; opcode_snd: intercepts all sent packets; opcode_rcv: intercepts all received packets. You can use opcode_snd | opcode_rcv.

Return Value:

None.

Bool capsendpacket (handle adapter, ulong opcode, ulong length, puchar data );

Note:

Put the processed packets into the buffer zone. You can also construct packets by yourself. It can not only send packets, but also send the packets to the local IP address software.

Parameters:

Adapter: Specifies the NIC handle used.

Opcode: opcode_snd, which sends packets to the network. opcode_rcv transmits packets to the local software.

Length: the length of the message;

Data: Packet content;

Return Value:

True is returned for success, and false is returned for failure.

Sample

# Include "cap. H"

# Include <stdio. h>

// Global data.

Adapt_info adaptinfo [16];

Int adapternum;

Void packetproc (handle adapter, ulong opcode, ulong length, puchar data)

{

Capsendpacket (adapter, opcode, length, data );

}

Void adapterschangecallback ()

{

Adapternum = capgetadaptlist (adaptinfo, sizeof (adaptinfo ));

}

Int main (INT argc, char * argv [])

{

Bool Bret;

Char cmd [80];

Int I;

Bret = capinitialize ();

If (BRET)

{

Adapternum = capgetadaptlist (adaptinfo, sizeof (adaptinfo ));

For (I = 0; I <adapternum; I ++)

{

Capsetrule (adaptinfo [I]. adapter, opcode_snd | opcode_rcv );

}

Capstartcapture (packetproc, adapterschangecallback );

For (;;)

{

Gets (CMD );

If (strcmp (CMD, "quit") = 0)

{

Break;

}

}

For (I = 0; I <adapternum; I ++)

{

Capsetrule (adaptinfo [I]. adapter, opcode_passthru );

}

Capstopcapture ();

Capuninitialize ();

}

Return 0;

}

Application Example

the above Code performs a Passthru action.
as a bridge or Nat, you need to modify the Ethernet header or other behavior of the packet content in the message processing function as needed, and then send the packet from another suitable Nic;
for protocol conversion, such as IP/udp tunneling or complex IPSec, You can unbind or decrypt the packet content in the message processing function and regroup the packet, put the buffer to send the driver to the IP software.
use the firewall to discard undesirable packets according to rules. The normal packets are also Passthru;
perform intrusion monitoring and security audit (only local hosts can be protected), and passthru records network events at the same time.