SIGCOMM17
Summary
Includes traffic monitoring, data collection, and prevention of a range of network attacks in existing network measurement tasks. The existing sketch-based measurement algorithms have serious performance loss, large computational overhead, and inadequate measurement accuracy, while the hardware-based optimization method is not suitable for sketch. In order to accomplish these tasks, a network measurement framework based on pure package forwarding is designed, and two algorithms are proposed to improve the existing algorithms. This network measurement framework features high performance (line-rate), high accuracy, versatility (for multiple sketch algorithms), and automatic (self-regulating load). The framework includes data plane and control plane, each software switch corresponds to a data plane, each data plane includes nomal path and fast path. Once the traffic is overloaded, the sketchvisor redirects the overloaded traffic to fast path to maintain high performance and high accuracy (albeit with slight loss), where the TOP-K algorithm is designed in fast path. There is a total control plane, the design of the compressing sensing algorithm, the distribution of the data provided by the switch to integrate, restore the entire network data. Finally, the experiment proves that Sketchvisor makes a series of sketch-based measurement methods achieve high performance and high precision.
Backgroundteminopogy
- epoches: One or multiple time periods.
- traffic statistics can be either flow-based, identified by 5-tuples, or host-based (identified by IP addresses); or either volume-based (measured by byte counts) or connectivity-based (measured by distinct flow/host counts).
- Sketch: At a high level, a Sketch is a compact data structure comprising a set of buckets, each of which is associated with one or multiple counters. It maps each packet to a subset of buckets with independent hash functions, and updates the counters of those buckets. Network operators can query the counter values to recover traffic statistics.
Measurement Tasks
- Network Measurement Tasks includes:monitor Traffid and Colect traffic statistics, and some network attack.
- Attack: Heavy hitter,heavy changer,ddos,superspreader,cardinality,flow size distribution,entropy.
Performance Flawsobservation
- Sketches is primitives that cannot is directly used for network measurement
- In order to collect meaningful traffic statistics, we must add extensions to sketches to make them reversible, me Aning that sketches not only store traffic statistics, but also efficiently answer queries on the statistics
- Although sketches is efficiently designed, applying them in network measurement inevitably incurs heavy computational Overhead.
- Sketches is compact data structures that can summarize traffic statistics of all packets with fixed-size memory, while in curring only bounded errors
Microbenchmark
- Providing comparation of exsited methods.
Problem
- Existing sketch-based Measurement solutions suffer from severe performance drops under high traffic load.
- Heavy Computational Overhead: Existing Representative sketch-based solutions in software actually consume Substantial CPU resources additional extensions or components of that often incur heavy computations.
- Optimizing specific functions (e.g., using hardware-based hash
Computations) May is not the work of the all sketch-based solutions.
Design goals
- Performance:it processes packets at high speed and aims to fulfill
The line-rate requirement of the underlying packet processing
Pipeline.
- Resource efficiency:it efficiently utilizes CPU for packet processing
and memory for data structures.
- accuracy: It preserves high measurement accuracy of sketches.
- Generality:it supports a wide range of sketch-based measurement
Tasks.
- Simplicity:it automatically mitigates the processing burdens of
sketch-based measurement tasks under high traffic load, without
Requiring manual per-host configurations and result aggregations
by network operators.
Solution
Implementaechitechture
- The Sketchvisor comprises Data Plane and Control Plane.
Data plane
- Each host possess a data plane, data plane can choose monitor ingress or egress traffic in case Duplicated count.
- Data plane has both path, one is Normal path and another are Fast path , when buffer is Full, the sketchvisor instructs the software switch to redirect overflowed packets to the fast path.
- They don ' t consider any proactive approach that examines packets and deciedes which packets should be Dispatched into either the normal path or the fast path,as it would incur non-trival overhead .
- the Fast path is less accurate than the Normal path.
- the fasy path should satisfy : fast enough to absorb all redirected traffic;highly accurate although SL ightly degrade from original sketch-base measurement;general for various traffic statics because each statics probably red Irect into the Fast path.
Control plane
- The Control plane collects each switch's results and merges them to providenetwork-wide measurement.
- The Control plane should satisfy: Eliminate the extra errors due to fast path (the error shoule only come from Ske Tches themselves.); Must is general to accommodate various measurement tasks.
Skechvisor
- Algorithmic Solutions, one builds on counter-based algorithms while the second builds on compressive sens ing to design a network-wide recovery algorithm.
Fast Path
- To avoid the measurement failed and keep accuracy, sketchvisor redirects overflow traffic into Fast Path.
- Design top-k algorithm which builds on misra-gries ' s top-k algorithm for fast path.
- First, in order-kick out a small flow and add a (potentially) large flow, it performs O1ko operations to update K count ERs in a hash table; The overhead becomes significant when there is many small flows to kick out.
- Second, it has loose bounds on the estimated values of the top-k flows. To overcome both limitations, we combine the idea of probabilistic lossy counting (PLC), a probabilistic algorithm that I Mproves accuracy for tracking skewed data, with
Misra-gries ' s algorithm.
- Specifically, we kick out multiple small flows each time, obviating the need of performing O1ko counter update operations For kicking out all flow (i.e., we amortize the operations over multiple kick-outs).
- Also, instead of using one
Counter per flow, we carefully associate three counters with each
Flow to provide tight per-flow lower and upper bounds.
Compressive sensing
- Use compressive sensing to recover network-wide statistics.
Related work
- sampling: Widely used in software-defined measurement for low measurement overhead, but inherently misses Informa tion and supports only coarse-grained measurement.
- Sketches: Many architechtures employ sketches as primitives to Chieve fine-grained measurement for various MEASRU Rement tasks, but incurs high computational overhead.
- TCAM: Can is used to Acheieve high-performance network measurement.
- Rule matching: Selectively processes only packets of interest, thereby reducing measurement overhead,but hash-tab Le incurs much higher memory overhead than sketched-based overhead.
- Recover missing information: a matrix
Interpolation problem to enable the control plane to recover missing
Information via compressive sensing
Advantages
- High throughput and high accuracy
- Fine grained
- Accurately reason about the behavior of high traffic load
- Resource-efficient
- Recovers Network-wide
Conclusion
- Design and implement Sketchvisor, a robust network-wide measurement architecture for software packet processing, with a PR Imary goal of preserving performance and accuracy guarantees even under high traffic load. Sketchvisor employs sketches as basic measurement primitives, and achieves high data plane performance with a fast path to Offload sketch-based measurement under high traffic load. It further leverages compressive sensing to achieve accurate network-wide measurement. Experiments demonstrate that sketchvisor achieves high performance and high accuracy for a rich set of sketch-based Soluti Ons.
Reading sketchvisor Robust Network measurement for sofeware Packet processing
