Big Data DDoS detection--ddos attack is essentially time series data, t+1 time data characteristics and T time strong correlation, so using hmm or CRF to do detection is inevitable! And a sentence of the word segmentation algorithm CRF no difference!

DDoS attacks are essentially time-series data, and the data characteristics of t+1 moments are strongly correlated with T-moments, so it is necessary to use HMM or CRF for detection! --and a sentence of the word segmentation algorithm CRF no difference!
Note: Traditional DDoS detection is directly based on the IP data sent traffic to identify, through the hardware firewall. Big data scenarios are done for slow DDoS attacks.
Difficulty: In the attack, the attack packet is disguised, the source IP address is also forged, so it is difficult to determine the address of the attack, in the search is also difficult. This results in a distributed denial of service attack that is difficult to verify. Field knowledge See: http://blog.csdn.net/eric_sunah/article/details/72782224
There are also summaries of: http://cleanbugs.com/item/ddos-attack-learning-414049.html

Paper: http://www.jos.org.cn/ch/reader/create_pdf.aspx?file_no=3960 can be downloaded, referring to the TCP flood,udp FLOOD,ICMP flood experiment

Excerpt from: http://wap.cnki.net/lunwen-1013353778.html
Research on DDoS attack detection technology based on spectral analysis and statistical machine learning
Chen Shiwen

  According to the research requirements of the National 863 Project "high credible Network Service control system" and "unified security control network for three networks", this paper studies the DDoS attack detection technology in accordance with the general idea of "distributed detection, hierarchical blocking and concentration situation perception". This paper presents a flood attack and low rate denial service (low-rate denial of service, LDoS) attack perception method based on IP stream sequence spectrum analysis from two angles of macro attack influenza knowledge and microscopic detection method, based on perceived attack. This paper transforms DDoS attack detection into two classification problem of machine learning, using hidden Markov model, twin support vector machine and three machine learning model of conditional random field to realize probabilistic point detection, classification hyper-plane detection and the conditional random-field detection method with multi-feature processing advantage. Aiming at the problem of macroscopic perception, this paper presents a flood DDoS attack perception method based on the fast fractional order Fourier transform to estimate Hurst, and uses the influence of DDoS attack on the self-similarity of network traffic, and by monitoring the Hurst index change threshold to determine whether there is DDoS attack, compared with the method of wavelet analysis. , the method has the advantages of low computational complexity, high precision of Hurst, and higher accuracy for low-rate denial-of-service LDOs attacks, and proposes a sensing method based on the power spectral estimation of Baxter, which is better than the rectangular window and triangular window method, and the performance of the Power spectral estimator is good. Low rate denial of service LDOs attack detection rate is high. Aiming at the problem of specific attack feature detection, this paper proposes an attack detection strategy based on hidden Markov model, twin support vector machine and three kinds of statistical machine learning methods based on conditional random field. Firstly, a DDoS attack detection method based on multi-feature parallel hidden Markov model (multi-feature Parallel Hidden Markov model, MFP-HMM) is proposed from the point of view of probability point discrimination. This method uses the correspondence of HMM hidden state sequence and feature observation sequence to transform the multi-dimensional characteristic anomaly caused by attack into discrete random variable, and describes the deviation degree of the current sliding window sequence and normal behavior contour by probability calculation. MFP-HMM Model architecture adopts multidimensional feature parallel processing mode, which facilitates the expansion of new feature modules. The characteristic sequence is formed by the sliding window to form the observation sequence into the Hmm, which can be realized by the hardware, which provides the condition for reconfigurable design and distributed deployment. The experimental results show that the method based on MFP-HMM is superior to the standard Hmm, such as machine learning, high accuracy and low false alarm rate. Secondly, based on the classification of the super-plane discriminant, a DDoS attack classification hyper-plane detection method is proposed, which uses the least squares twin support vector machines (Least Square Twin supported vector machine, LSTSVM), which is solved by the optimization method.Machine learning problem, using support vector machine Model good non-linear processing ability and generalization ability, using IP packet five tuple entropy, IP identifier, TCP head flag and packet rate as LSTSVM model of multidimensional detection feature vector, to reflect the existence of DDoS attack flow distribution characteristics. Experiments based on DARPA2000 data set and Tfn2k attack  show that this method is better than the standard SVM (support vector machine, SVM) and other machine learning methods, which have high accuracy and low false alarm rate for normal burst traffic and DDoS attack traffic detection. Finally, a new method for detecting DDoS attacks with multiple discriminant rules is proposed. The method does not require that each characteristic quantity must satisfy the hypothesis of independent distribution, and the method based on feature matching and anomaly detection is effectively unified to realize high detection rate and low false alarm rate on the basis of taking full advantage of the multi-feature advantage of the comprehensive treatment of the condition with the airport. DARPA2000 Data set experiment shows that the method based on conditional random field is better than traditional SVM, the accuracy rate is higher than 99.5%, false alarm rate FPR is less than 0.6%, and the anti-background noise ability is strong and the robustness is good. ......
[keywords]:ddos attack; self-similarity; fractional-order Fourier transform; Bartlett spectral estimation; hidden Markov model; twin support vector machine; condition with airport

Excerpt from: http://cdmd.cnki.com.cn/Article/CDMD-90002-2007140546.htm
Research on detection method of distributed denial-of-service attack based on machine learning
Sun Yongqiang
Abstract: In recent years, the detection and defense technology of Distributed denial of service (distributed denial of Service:ddos) attacks has become one of the research hotspots in information security field. The distributed nature of DDoS attacks makes such attacks more powerful and more destructive than traditional denial-of-service attacks (denial of service:dos), and more difficult to prevent. At present, due to the limitations of the existing intrusion detection technology, DDoS attacks have posed a great threat to Internet security operation, making the demand for new generation of DDoS detection and defense technology more urgent. Based on the analysis of the principle of DDoS and the current research situation of the detection and defense technology, this paper studies the method of detecting DDoS attack based on machine learning, and focuses on the hidden Markov model (Hidden) based on the existing problems of the detection methods and the related theory progress of machine learning. Markov model:hmm) A new DDoS detection model and research on the distributed collaborative detection mechanism based on adaptive learning. The main research work and innovation points include: 1. Based on the theory of Hmm, this paper proposes a method of DDoS attack detection on the basis of HMM and source IP address monitoring. The method uses the source IP address information in the network data stream to express the characteristic of the network traffic state. First, the common source IP address library is studied according to the normal data flow, and then the statistical modeling of the dynamic IP address sequence of the network data stream is made by using the hidden Markov model. Through the HMM model learning through the IP address sequence of normal traffic, the real-time anomaly detection based on the dynamic source IP address sequence is carried out to the unknown network traffic, and the common source IP address library keeps the online learning update. 2. A distributed collaborative detection method based on adaptive learning is proposed for the problems existing in DDoS distributed detection. In the framework of distributed collaborative detection, the method of data fusion is used to detect, combined with a return-based adaptive learning algorithm, in order to ensure the accuracy of detection, the system reduces the traffic between the detection nodes, and improves the system operating efficiency. 3. Design and implement the prototype system of DDoS detection experiment based on machine learning, including the single point detection module based on HMM and the distributed cooperative detection mechanism based on adaptive learning. In the context of LAN, this paper simulates and detects DDoS attacks with the above experimental prototype system, and verifies the feasibility and effectiveness of the proposed method. The research content of this paper is an important part of the National Natural Science Foundation "adaptive intrusion detection method based on reinforcement learning". Compared with other methods, the proposed method has the advantages of high detection accuracy, strong real-time, easy to respond and easy to deploy, and has a better application prospect.
"keywords": Distributed denial of service attack machine learning hidden Markov model intrusion detection

Big Data DDoS detection--ddos attack is essentially time series data, t+1 time data characteristics and T time strong correlation, so using hmm or CRF to do detection is inevitable! And a sentence of the word segmentation algorithm CRF no difference!