[DPDK] Read Development Guide (1)

This document is an increasing reading note along with the progress of reading for the document. The main content is mainly outline, as well as records to help remember content.

In the later practical application, also does not follow the unceasing thorough understanding, gradually enriches each outline the content.

1. Pre-Preparation: Set up two environment variables.

Export rte_sdk=/home/user/dpdkexport rte_target=X86_64-NATIVE-LINUXAPP-GCC

2. DPDK provides an abstraction layer of the environment:

• DPDK loading and launching DPDK are linked to an executable program along with the application.
• Support for multi-process and multi-thread execution types
• Core affinity/assignment Procedures provides CPU binding capability.
• The system memory Allocation/de-allocation can reserve physical memories for device interaction.
• Atomic/lock operations libc not provided in the operation, spin lock, atomic count and so on.
• Time Reference
• PCI bus Access provides an interface to access the PCI physical space.
• Trace and debug functions provide debugging functions, logs, call stacks, panic, and so on.
• CPU feature identification runtime CPU feature awareness.
• Interrupt Handling
• Alarm operations
• Memory Management (malloc)

The 2.1 DPDK program runs in the user state while using the user-state PCI driver UIO, using MMAP to map the device memory into the user state.

2.2 Use large page memory.

Based on large-page memory, you can complete the application allocation and reservation of large pages of memory in your applications.

XEN DOM0 does not support large pages and needs to be used rte_dom0_mm See documentation

Technical details that require further detail: TSC (CPU time-stamp Counter), HPET kernel API, mmap

2.3 Initialization process, multi-core scheduling and processing.

The resource allocation during initialization is thread insecure. But once the initialization is complete, the resources can be used safely in each thread.

2.4 Supports multiple processes in addition to multi-threading.

The 2.5 provider log API, which can replace syslog and console logs for the program's log output.

Rte_panic for commissioning

2.6 User-configured NIC interrupt response

Suddenly a nameless DPDK PMD. What is PMD?

2.7 Memory segments and memory Zones

Allocate contiguous memory space for each file descriptor and align to cache line size or hugepage.

2.8 Multi-Threading

Each core binds only one pthread

Control power Management to turn on maximum performance.

Controls multi-threaded context switching within a single core.

2.9 Cgroup Control do not understand Ah, learn ah

Example of the original document:

mkdir/sys/fs/cgroup/cpu//sys/fs/cgroup/cpuset/>/sys/fs/cgroup/cpuset/>/sys/fs /cgroup/cpu/pkt_io/>/sys/fs/cgroup/cpuset/pkt_io/>/sys/fs/cgroup/cpu/pkt_io/  >/sys/fs/cgroup/cpuset/pkt_io//sys/fs/cgroup/cpu/100000 > pkt_io/cpu.cfs_ Period_usecho  50000 > Pkt_io/cpu.cfs_quota_us

2.10 Malloc

DPDK provides a similar system call to the malloc function Rte_malloc (), which is typically much slower than pool-based malloc and has locks during program operation. It can be used in configuration-related code, rather than in a single code for data processing.

Memory size using a power of 2 will support automatic alignment and will be allocated in the corresponding node based on NUMA optimizations. but it's not guaranteed to be on this node?!?!?! (not understood, please refer to the original document, and welcome to contact me to guide me)

2.10.1 Internal data Structure malloc_heap

Very simply, a linked list points to all unallocated memory. The used memory is re-linked to the linked list when free ().

2.10.2 Internal data Structure Malloc_elem

Use scenario one: As a header for all memory blocks (allocated/unassigned).

Use Scenario two: padding header

Use scenario Three: end of memseg marker

3. Core components:

3.1 Librte_ring A multi-read, multi-write, lock-free queue structure. Can be used by Librte_mempool.

There is no priority concept, and any queued (out-of-team) operation has no higher priority than the other Enqueue (outbound) operations.

3.2 Librte_mempool Memory Object pool, provides CPU cache alignment, memory alignment and other capabilities.

3.3 Librte_mbuf Memory request Release Management.

3.4 Librte_timer is a timer.

3.5 Network card driver, on the polling mode, do not use asynchronous interrupts.

3.6 packet forwarding algorithm, Hash LPM (longest profix Match)

3.7 librte_net protocol stack, Baotou structure and so on.

4. RING LIBRARY

• Fifo
• Maximum size is fixed, the pointers was stored in a table
• Lockless implementation
• Multi-consumer or Single-consumer dequeue
• Multi-producer or Single-producer Enqueue
• Bulk Dequeue-dequeues The specified count of objects if successful; Otherwise fails
• Bulk Enqueue-enqueues The specified count of objects if successful; Otherwise fails
• Burst Dequeue-dequeue The maximum available objects if the specified count cannot be fulfilled
• Burst Enqueue-enqueue The maximum available objects if the specified count cannot be fulfilled

The advantages of this data structure over a linked list queue is as follows:

• Faster; Only requires a single compare-and-swap instruction of sizeof (void *) instead of several Double-compare-and-swap instructi Ons.
• Simpler than a full lockless queue.
• Adapted to bulk enqueue/dequeue operations. As pointers is stored in a table, a dequeue of several objects would not produce as many cache misses as in a linked queue . Also, a bulk dequeue of many objects does not cost more than a dequeue of a simple object.

The disadvantages:

• Size is fixed
• Have many rings costs more in terms of memory than a linked list queue. An empty ring contains at least N pointers.

Other Features:

1. Use a unique name designation.

2. The water level cordon, exceeding the warning line, the caller will be informed.

3. Debug Debugging. Based on the single core count information.

Usage scenarios:

1. Inter-application communication interaction.

2. By Mempool Allocate.

Operating principle (no lock principle):

Slightly. There is a book << dpdk>>, read, supplemented the CPU and other knowledge, remember to come back to see.

Supplementary reading:

Http://lwn.net/Articles/340400/

5. Mempool Library

A memory pool is an allocator of a fixed-sized object. By default, Rte_ring is used, and the implication can be different. Additional features are available: A Per-core object cache and an alignment helper to ensure that objects is padded to spread them equally on all DR AM or DDR3 channels.

5.1 supports the debug feature and provides a core-based count.

5.2 Memory Alignment

command line specify channel number, rank number?

5.3 Local Cache

Compare-and-set (CAS) operation. CPU instructions?

5.4 Mempool Handlers

Use an external memory management system instead of DPDK.

6. Mbuf Library

Provides memory for alloc and free capabilities.

6.1 Packet Buffers

A memory data structure that has been designed to store network packets.

6.2 Meta Information

Information about the various package contents is stored in memory. It is worth mentioning when the contract, you can delegate a portion of the content to the network card for hardware calculation.

6.3 Direct and Indirect buffers

A simple understanding of indirect buffer is a reference to DIERCT buffer. The creation and delivery of references through a specific API.

6.4 Debug

MBUF related operations will open strict inspection, strict type, dirty data and so on.

(not to be continued ...) ）

[DPDK] Read Development Guide (1)