Buffering and application optimization

Buffering: A storage technology used to bridge the speed gap between different data processing rates.
Buffering is everywhere in daily life. There are many buffering examples in the human body, such as eating a mouth. Because of the limited throat size, the mouth will store the food first, chew to the size that can pass through the esophagus, and then evenly into the human body; while the stomach is the second buffer zone, in order to adapt to the absorption rate of the intestines, the stomach needs to fully grind the food, in order to make the intestines better absorb.
Our commonly used computer hard disks, despite the increasing channel technology, are embedded with many microprocessor chips, but the speed of Disk Writing is always a bottleneck, so we also need to use the memory chip as a buffer, this is disk buffer or disk cache.
Through the above example, we can better understand the buffer principle. In the field of Server Load balancer, can we use the buffer technology to optimize traffic?
 
1) TCP buffering

The server Load balancer device adopts the full-proxy working mode. When the client data transmission speed is different from the server data transmission speed (for example), the tcp buffering technology can make up for this difference, ensure smooth data transmission.
The benefits include:
• The Server Load balancer device uses TCP reuse and open connection to maintain the connection between the client and the server
• Provides server-side or client-side buffering to temporarily store data packets to prevent connections from being dropped due to slow client-side or server-side responses
• Data can be sent at the client or server link speed
 
2) http buffer cache
The two concepts of buffer and cache are actually different. Although both are stored data (units), cache divides some buckets to store frequently accessed data, reduce the number of direct interactions between the access source and the original data; buffer is more a buffer action before writing data to the hard disk. In Qos applications, the buffer and cache are often "combined. Raw data (images, music, videos, and other files) on the server is frequently accessed, and the server's response speed is limited, therefore, a part of the memory space is allocated to the Server Load balancer device as a buffer zone. The Cache Server frequently responds to the content. When a large number of clients initiate requests, the Server Load balancer device can directly respond to the request from the buffer zone.
1) multi-CPU and multi-core Parallel Processing
The new generation of Server Load balancer devices all adopt multi-CPU, multi-core parallel processing technology, which greatly improves their delivery capabilities. There is a mismatch between the CPU and I/O devices. I/O buffer capacity is limited. When I/O buffer is exhausted, whether the system buffer can be generated to buffer more data queues requires the Server Load balancer device to have an excellent operating system design. A10 ACOS is the leader in this kind of technology. In the following example, we can see that each core has a corresponding buffer:
# Buffers in Q0 cache: 2393 App: 15743405 TCPQ: 0 misc: 0
# Buffers in Q1 cache: 3332 App: 0 TCPQ: 4223160824 misc: 0
# Buffers in Q2 cache: 4630 App: 0 TCPQ: 71806473 misc: 0
Approximate # buffers in App 15743405
Approximate # buffers in App_cp-15743405
Approximate # buffers in Cache_cp 595
Approximate # buffers in Cache 10355
Approximate # buffers in Queue 10149
Approximate # buffers in misc 0
Approximate # Buffers' free 110568
Approximate # buffers avail from HW 81255
Through OS allocation, 1. Improve the speed mismatch between the CPU and I/O devices;
2. It can reduce the number of CPU interruptions on the I/O device and relax the requirement on the CPU interrupt response time.
 
These are some of the applications of the buffer technology in Server Load balancer. Of course, not all application delivery products can provide complete Buffer Optimization, we hope that you can fully consider the hardware and software architecture and make the right choices when selecting products.
(ZJM)
This article is from the "ADC technology blog ".