Some ideas and opinions on performance testing of network products.

The performance indicators of network products are mainly proposed in rfc2544 and rfc3511.
It mainly includes throughput, latency, back-to-back, packet loss rate, new link per second, transaction processing per second, valid throughput (HTTP throughput), and maximum concurrency.

Throughput, latency, back-to-back, and packet loss rate are defined in rfc2544 and belong to 2-3 tests. The UDP data type is used in the tests. The other tests are defined in rfc3511 and belong to Layer 4-7 tests. The test focuses on the TCP data type.

Rfc2544 defines the most basic test item in 2-3 tests. Customers can also improve or expand their testing methods as needed, such as hybrid packet throughput, Nat throughput, and L2TP testing.

Rfc3511 is also the most basic test item for layer-4-7 testing. The customer can also improve or expand the test method based on their own needs, especially the test at the application layer. For example, FTP throughput, SMTP establishment rate, and telnet establishment rate.

However, with the development of network products, the indicators defined in rfc2544 and rfc3511 cannot fully reflect the performance of network products.

I personally think. Measure the performance of a network product. It should be comprehensive. In addition to the most basic performance indicators, the functional performance of the product should also be tested.

For example. If a product has a VPN function. In addition to the above indicators, we should also test the performance of VPN.
Including the VPN throughput, the number of new tunnels per second and the maximum number of VPN tunnels.

If a product supports dynamic routing. The performance of dynamic routing should also be tested. Including route table capacity, route convergence time, and latency under jitter.

If this product has IPS or UTM functions, the performance of virus detection, deep analysis, content filtering, and email detection should also be considered ..

There are many network products, such as gateway products, routing products, and exchange products. The test focus varies with the product positioning. In my opinion, apart from basic test items, the test items should also be changed due to different product positioning.

Gateway products should focus on attack defense. The routing product should focus on the routing performance, and the switch product should focus on 2-3 tests.

According to the current network products, almost all vendors can easily achieve the rfc2544 testing indicators. Therefore, we should focus on testing at the network layer or application layer.

 

---------------------------------------

 

In the application layer test, the test method is flexible. Although testing the same indicator, You can compile different testing methods based on different concerns. Here is a small example.

 

Test Project effective throughput (HTTP throughput)

 

Test Instrument ixia400t alm_8t_1g Board

First, if we focus on the performance indicators of effective throughput, we can use a link for testing.

Second, if we focus on the performance indicators of the effective throughput under a certain link, we need to create a certain number of valid links and each link, both tests the effective throughput.

In the first test method, it is the standard test method defined in rfc3511. This test method uses only one link. The second test method is extended based on rfc3511. This test method uses multiple links for concurrent data transmission. This test method imposes a relatively high pressure on the device. However, there is a certain relationship between the number of links and the performance of throughput. The more connections, the smaller the throughput. And vice versa. How to control the number of links is defined in the test requirements. Define testing methods based on your own needs to better reflect the performance between software and hardware of network products.

These ideas are put forward. The reason is that the performance test on Layer 4-7 is flexible. Although it is the same test project, the test results vary according to the test notes and methods. In other words, it is impossible to reflect the performance of network products. Therefore, I propose that when measuring the performance of a network product, especially for Layer 4-7 testing, you should not only view the test results, but also understand the testing focus and testing methods. Only in this way can we understand the performance of our products in 4-7.

The above is Xiao Di Yu Jian. If any, please correct them. Thank you very much!