Serial Communication Data Processing Algorithm

 

Keywords: circular buffer Serial Communication Data Processing Algorithm serial communication Serial Data Processing

1. Background Analysis

In many practical engineering applications, communication methods are usually RS232, RS485, I2C, and SPI. These interfaces share a common feature: communication by byte streams, that is, each interruption, indicates that a byte is successfully transmitted or received.

Some interfaces transmit data blocks, that is, one transmission or multiple bytes, such as can, USB, and Ethernet. Can and USB usually transmit dozens of bytes at a time, similar to word throttling. The difference is that we can take advantage of the CAN and USB interfaces to simplify communication protocols and improve reliability. For example, both can and USB have the hardware verification function, so we do not need to add the verification domain in the communication protocol. The CAN network has the conflict detection and automatic retransmission functions, so we can easily implement multi-point to multi-point communication. In contrast, RS485 is suitable for one-point to multi-point communication, to Implement Multi-Point-to-multi-point communication, it is difficult to solve problems such as automatic retransmission of conflict detection.

However, Ethernet can transmit more than 1 kb of data at a time. Since Ethernet has a dedicated TCP/IP protocol stack for processing, we will not discuss it here.

The interface for communication between small data blocks in byte mode (CAN and USB). The difference is that the length of data received at a time is different. To improve the versatility of algorithms, we should abstract the differences between these hardware interfaces, that is, we should shield the differences between different interfaces. For data processing programs, it can be considered that these interfaces are transmitted byte streams, that is, a string of data. The USB interface refers to the situation where the USB slave port of the embedded device can communicate with the host.

 

Common Format of 2-byte stream communication protocol

The byte stream communication protocol generally contains the following fields:

Forward code + frame length + frame number + data domain + Verification

The actual communication protocol needs to define the length of each domain and the exact meaning of each bit. Some communication protocols place the frame length behind the frame number.

 

3-byte stream Data Processing Algorithm Implementation

Let's first look at what a circular FIFO buffer is.

A fifo buffer is a buffer with the first-in-first-out function. The following struct can be used for definition:

Typedef struct buffer_t {

Int head;

Int tail;

Char data [buffer_len];

} Buffer;

This structure is very simple. The head records the buffer header, the tail records the tail of the buffer, and data is used to store data. When modifying head and tail, You Need To modulo buffer_len to prevent overflow. The valid data length of the buffer can be calculated as follows:

(Tail + buffer_len-1-head) % buffer_len.

Based on this struct, three functions of buffer initialization, writing, and reading are implemented in FIFO mode. Loop buffer refers to when the data is written to the last data element, and then the 0th elements should be written to the data, turning the linear buffer into a loop. When writing and reading data, the data moves in this circular buffer zone, shielding the details of internal operations.

Pay attention to the following points when implementing algorithms:

1. Condition for determining whether the buffer is empty or full: when the head and tail are equal, the buffer is empty, and when the buffer has written BUFFER_LEN-1 data, the buffer is full. Why not write buffer_len data? Because after data is written into buffer_len, the head and tail are equal, so you cannot determine whether the buffer is empty or full.

2. write and read policies

When reading or writing data to the buffer zone, check whether the data or space in the buffer zone is sufficient.

When reading, if there is not enough data, whether to read the existing data or not read any data, but when writing, if there is not enough space, whether to partially write or not write any data, it depends on your application. In general, you can do nothing when the space is insufficient. When the above situation occurs, it is left to the upper-Layer Program for processing. In practical applications, if the read and write programs are properly designed and the buffer size is appropriate, there is generally no write failure.

Another programming detail is that the array operations must be careful when the array is out of bounds. Strict check is required. Otherwise, it may take a long time for you to get depressed during debugging.

You may have been wondering why circular buffer is used? Don't worry, let's analyze it slowly.

I have learned the underlying details of byte stream devices, the General Protocol format of byte stream devices, and the principles and implementation methods of the cyclic FIFO buffer, the buffer zone has the following advantages:

1. for upper-layer functions, the Operation Details of hardware interfaces are abstracted, improving portability. To put it simply, the upper-layer functions and the underlying drivers (the functions actually complete interface operations) are connected through the cyclic FIFO buffer. When the underlying interface is changed from RS232 to can or USB, you do not need to change the algorithm on the upper layer of the communication protocol. You only need to change the underlying driver, put the received data in the buffer or send the data to be sent in the buffer.

2. The circular buffer can be easily used to search for the forward guide code and handle error packets.

Forward Code Search: The forward code is usually located at the front of a valid data packet. Therefore, a scan function is implemented for the circular buffer to browse the first few data in the receiving buffer, that is to say, it is just to view (copy out) and not read from the buffer zone. Assume that our forward guide code is 0xa5, 0x5a, 0xa5, 0x5a. You can search for the forward guide Code as follows: four bytes are scanned from the buffer each time. If the forward guide code is the same, the search is successful. if the search is different, read a byte from the cache and then scan the buffer until the search is successful.

Let's take a look at the general process of the Protocol:

The starting condition for data processing is that the valid data length in the buffer zone is greater than or equal to the minimum package length.

1. Search for the forward guide code: If successful, go to 2. If it fails, one byte is discarded and the search continues.

2. Check the frame length: scan the length field and pass the maximum and minimum packet length check that may occur in the Protocol. If the check is normal, go to 3. Otherwise, discard one byte, to 1.

3. Check the frame number: scan the output frame number to check whether the frame number is a valid frame number. If the frame number is valid, go to 4. Otherwise, discard a byte and go to 1.

4. checksum check: scan the data domain and verification domain after the length. Check whether the checksum is correct. If the error occurs, discard a byte and go to 1. If correct, read the complete frame and retrieve the frame number and data domain. Go to 5.

5. perform operations based on the frame number.

Uses the features of the cyclic FIFO buffer to provide data caching and flexible access to byte stream data, to strictly check various Protocol domains, it provides perfect Filtering for packets with errors in some domains and incomplete packages, as well as accurate extraction of correct packets from chaotic data. All these features are required by byte stream devices.

 

4. Algorithm Improvement

4.1 improvement of the buffer Data Structure

This data structure is the model that I first designed the buffer data structure design. It satisfies the needs of my project at that time. However, you may have thought of its own shortcomings.

Since my previous device only has one interface to use this buffer, it runs very well. When I am working on a new project, there are multiple interfaces that need such a buffer. The problem arises: My buffer length is a fixed value (buffer_len ), if the buffer required by multiple interfaces is different, this buffer cannot be well satisfied. That is to say, my buffer is a fixed value, you cannot flexibly change the buffer length of different interfaces. Therefore, make the following improvements:

Typedef struct buffer_t {

Int head;

Int tail;

Char * data;

Int Len;

Void * PSEM;

} Buffer;

During buffer initialization, A Global Array pointer and array length declared externally are transmitted to initialize data and Len. In this way, the length of the buffer can be freely changed. In a multi-task environment, PSEM is used for mutual exclusion.

It should be noted that data is a char-type pointer, and any data is operated according to the char type. For multi-byte data operations, pay special attention to the issue of byte order. The communication protocol should accurately specify the collation of Multi-byte domains to avoid confusion between devices due to the collation problem.

4.2 interface-oriented Buffer

As we know, communication interfaces generally have the receiving and sending functions. For one interface, we need two buffers, one for receiving and the other for sending. Therefore, the following structure can be defined:

Typedef struct com_buffer_t {

Buffer txbuf;

Buffer rxbuf;

} Com_buffer;

Based on this structure, you can read, write, and browse devices.

4.3 Use heap to increase memory usage

If our system has multiple byte stream devices, each byte stream device uses two global arrays for the buffer zone. In reality, most of our interfaces will not always work, but they will always occupy the memory. If the device needs to work, dynamically obtain the memory, and release the memory after use, this will reduce the memory usage time of the device and increase the memory usage.

Note that if you directly use the implemented malloc function in the library, memory fragments are very likely to occur. Therefore, special memory management functions should be used.

4.4 discard the forward guide code

We previously assume that the forward guide code is 0xa5, 0x5a, 0xa5, and 0x5a. In fact, when the forward guide code is successfully searched, We can discard two bytes at a time, because the first two bytes of the forward code are the same as the last two bytes. Similarly, if the forward code is "ABCA", three bytes can be dropped at a time. If the forward code is "ABCD", four bytes can be dropped at a time. Obviously, this can speed up the search. The same technique can be used for the search of the forward guide code.

4.5 other useful buffer operation functions

1. Buffer clearing function: used to clear the buffer and prepare to receive new data. It is best to clear all the data.

2. Buffer data discard function: Replace the function of reading data from the buffer to discard data, improving the program execution speed.

 

Disadvantages of 5-cycle FIFO buffer

Lao Tzu said: "The Treasure of happiness depends on the treasure of happiness ". Xianzhe of China is too great.

Everything in the world has two sides. The advantage is the disadvantage.

The cyclic FIFO buffer stores byte streams in an ordered data block in order for upper-layer functions to read and abstract interfaces. However, the first-in-first-out feature also indicates that the first-in-first-out command is executed first, and the second-out command is executed later. Urgent commands can only be executed after the preceding commands are executed. If there are many time-consuming commands before an emergency command, the real-time performance of the Emergency Command will be hard to be guaranteed.

In actual projects, this requirement may be rare. However, we still need to consider these situations to prevent accidents.

In fact, there is a solution to improve the real-time response of commands. The reference prototype is the software flow control mode of serial ports.

The implementation principle is as follows: Set an emergency command byte (urgent_char = 0xff, fill_char = 0x00). When the serial port receives the Emergency Command byte, clear the buffer and wait for the emergency command to arrive. In order to achieve transparent data transmission, it is clear that bytes must be filled when the buffer is sent, that is, when the user's data contains emergency command bytes, A fill_char should be filled behind it.

So far, the problem has been solved perfectly.

 

6. Solve the old problems and the new problems will emerge again.

Whether in a system with OS or a system without OS, functions operated during interruption must be reentrant. Be careful when operating global variables. The buffer clearing function is a combination of these two problems.

When the task reads the buffer, the Emergency Command byte arrives, and the buffer clearing function is executed, which will damage the data in the buffer. When an interrupt is returned to the application, the read buffer data is incorrect, which may result in an incorrect command.

If the command validation is correct, there will be no errors in clearing the buffer. Only when the command Check is correct and the data is read from the buffer zone is interrupted, clearing the cache area may result in incorrect and abnormal data reading.

It is a good solution in the OS system. It can kill data processing tasks and create emergency tasks to process Emergency commands.

In a system without OS, we can set an emergency flag. when the data is returned, check this flag. If an emergency command exists, the read failure is returned. However, this does not perfectly solve the problem of real-time command alone.

There are two bad cases:

1. When an emergency command appears after the emergency mark is determined, it means that the Emergency Command will wait for the execution of a command to complete.

2. Before an emergency command is sent, the program is processing a long time-consuming command, which means that the Emergency Command will wait until the execution of the command is complete.

 

7. Conclusion

In actual use, if real-time requirements are required, it is strongly recommended to use a real-time operating system. Systems without real-time requirements, weigh the advantages and disadvantages according to their actual situation, and implement the above policies selectively.

 

Http://hi.baidu.com/mikenoodle/item/556aa7ef53e286265b2d64f0