Mr Niu rejected cloud acceleration architecture sharing SACC speech

stand on the internet+High-availability architecture design, storage technology architecture,CDNand network architecture optimization, front-end technology development, mobile application architecture design,ITArchitecture New thinking, operation and maintenance2.0and other technologies of -China's system architecture, the national Normal University, will usher in a milestone. How to meet high-security, high-reliability and high-reliance industry architecture at the same time?how to deal with massive real-time large-volume, high-concurrency, high-response, high-real-time Internet architecture requirements? Mr Niu rejected cloud acceleration for this contribution and exploration is undoubtedly worthy of everyone's applause. Today, I will bring you the great God of this conference.Kindlefully prepared dry goods with you to share, I hope to attend the meeting of you to deepen memories, but also to the friends who have not attended some reference, so that the Mr Niu rejected cloud to accelerate in-depth understanding of the same time add trust and support. Let's get down to the chase.

Introduction talk about security.

There are a number of detection and protection systems around web security vulnerabilities that are often used by vulnerable scanners, WAF firewalls, and code audit products. Vulnerability scanners are often used to crawl site URLs to exploit vulnerabilities, and often we only need to fill in the target domain to help us discover a number of obvious web security vulnerabilities. However, because most of the vulnerability scanners need to crawl Web site connection parameters, and then the vulnerability test, such a pattern will have some crawler depth, form and JS Interactive ability challenges, resulting in the request address and parameter fetching incomplete, and the lack of encryption data testing problems, resulting in some of the security vulnerabilities were omitted, There will also be business growth that creates new vulnerabilities that are difficult to monitor in real time, and most of them are regularly scanned, so that there will be a time window in which vulnerabilities are exploited to complete an attack. A WAF firewall is a system that discovers malicious traffic and intercepts it by analyzing the HTTP protocol data. Traditional WAF equipment needs to be procured, expensive, needs to be adjusted for the existing business network topology, and not be able to follow the business development flexibility to expand nodes, such as the addition of several IDC will need to re-purchase equipment, and deployment is more cumbersome. Code audit products can be found in the solution Code of the vulnerability, fundamentally solve the program vulnerabilities, but the Web site is not only run web security issues, there will be web container security issues, configuration of non-compliance or vulnerability. For example, Nginx,iis and Apache have been the analysis of vulnerabilities, many enterprises or individuals have caused huge losses. DDoS hard-defense prices are more expensive, and DDoS defenses are not hard-to-protect, but they also need to address huge bandwidth issues. These security issues, we are difficult to solve through a system, one configuration or one scan.

Ideal Solution

can be To ensure the safe, fast and stable operation of the site at all times, and to facilitate user use, preferably zero deployment, 0 maintenance of sustainable improvement of the solution.

Mr Niu rejected cloud acceleration architecture resolution

Introduction of dispatch system

Mr Niu rejected cloud acceleration scheduling system using our company's brother Products Cloudxns (hereinafter referred to as XNS), XNS based on the DPDK independent research and development, single-machine performance up to tens of thousands of QPS intelligent DNS system; The line is widely distributed, there are more than 30 points near hundred sets of million/dual million gigabit server. and traditional AXFR, IXFR and other distribution methods, XNS independent research and Development Update module, can be achieved in half a second level effective. XNS also extends the DNS protocol by deriving link, AX, Cnamex Records in addition to all common record types. The API is simple and comprehensive and well suited as a CDN gslb service. Because it is self-developed, it is not affected by open-source software-level vulnerabilities, such as BIND's TKey vulnerability throughout the year Mr Niu rejected cloud acceleration is not affected.

Massive Configuration Management

Mr Niu rejected cloud accelerates every day with tens of thousands of domain name security, accelerated policy change operations, the configuration changes are very frequent. Because our service node business layer is based on nginx two times development, and can be based on the Nginx version of the seamless upgrade. Originally in the configuration management this piece before we use the ETCD+CONFD scheme, CONFD every two minutes to monitor ETCD configuration information, according to the template to generate Nginx configuration file, so that we are very convenient to complete the service of automatic discovery and configuration generation, With the development of the business we found more configuration content, and change frequently, reload time workers to the configuration effective time and affect performance, some special cases once the configuration is loaded incorrectly, will affect after the normal reload. In order to solve these problems, we have adopted the NGX_LUA+REDIS scheme, which increases the memory lock and the use of outdated data in the process of failure. The result is a perfect 5s complete network configuration change without the need for reload to take effect in real time, and each domain is configured to walk independently of each other.

In addition, the CDN return source needs to use the Inerdns, and Inerdns each configuration synchronization and the real-time effect is a very difficult problem. The BIND+MYSQL scheme is inefficient. Mr Niu rejected cloud accelerated through the Golang+redis re-developed the INERDNS system, borrowing such as Shanghai volume configuration management ideas. Dynamic changes and real-time effect are also completed. and on each node of the machine deployed a very lightweight dnsserver, the way also to avoid network query slow and timeout problems.

Cloud WAF

Cattle Shield cloud-accelerated cloud WAF is highly focused on the active and processing performance of the security guard. Capable of bidirectional processing of data stream, docking the received data for self-decoding, through the semantic analysis of SQL injection detection mode and the remaining few regular, the introduction of the concept of the rule chain so that we in the processing of many complex variant attacks more accurate, to avoid the strategy rough caused by the false interception. Post different types of data format is parsed, can get the file name, file contents, parameters, parameter names and other information to do in-depth detection and analysis. Enables users to customize security policies within the platform and dynamically manage all of the above security policies. In addition, our users ' websites will identify and assign the best security policy to the user when accessing Mr Niu rejected cloud acceleration. Focus on the latest defense and attack technology and Vulnerability Update security policy, when updated we combine the back-end real-time log Analysis System (ERK) to adapt the strategy. In addition, the data analysis system can be used to monitor false positives and false negatives, which will be introduced in the subsequent Data Analysis section.

CC Protection

A cc attack is a type of DDoS attack that is usually initiated by a host of HTTP requests from a zombie master. The traditional CC defense has the IP frequency, the Ip+uri frequency, the verification code, 302 and so on the scheme, however these scheme final processing way either the IP, either is easy to be bypassed, or the user experience is not good. The Mr Niu rejected cloud acceleration default policy determines whether there is a suspicious behavior of a CC attack by the client's request frequency with the Site resource dimension feature, and then the client is implanted with a specific code to detect if the client can bring it back to normal access, and if not, the request is blocked. And Mr Niu rejected cloud acceleration will recognize the attack fingerprint, full-node real-time synchronization, complete the entire defense. This ensures that the back-end business is not affected by the attack. In the face of more complex cc attacks, Mr Niu rejected cloud acceleration will perform real-time anomaly analysis of the access log and intelligently control the defense strategy. At the same time, we also provide customers with a variety of defense strategies in the platform can be freely selected, but also support the request speed limit function and other functions. The entire defense strategy is simply blocking the client requests from hackers and not being affected by the normal user.

DDOS Defense

Cattle Shield Cloud Acceleration provides 400G of anti-D bandwidth. A full range of self-developed CC and Synflood defense mechanisms have been deployed. Built five large cleaning centers, bypassing the deployment of DPI-based DDoS attack detection equipment and cleaning equipment, can do packet depth detection, second-level attack response time delay, so as to ensure the flow cleaning effect, and support attack fingerprint message matching discarded, dynamic fingerprint learning mechanism. When the attack traffic reaches a certain proportion of a single node, we use XNS to dispatch the attack traffic to our cleaning center to complete the traffic cleaning. and anti-DDoS and hard-to-block bandwidth is not good, attack traffic to fill the bandwidth to do? To solve this problem, the Mr Niu rejected cloud accelerates with the operator to use near-source suppression for super-large DDoS attacks, making black holes before the attack traffic reaches the target room.

SSL Support

Ssl and CDN in the cloud acceleration market has been treated as a fish and bear cake can not be both, There are only a few two or three support for HTTPS acceleration, and the Mr Niu rejected cloud-accelerated HTTPS service is more pro-people than the high threshold of an accelerated Enterprise Edition. The reason for this situation, the small part of the thinking is that HTTPS is not yet popular, on the other hand is to solve the performance and quality problems of the technical maturity. Mr Niu rejected cloud acceleration has long been a forward-looking, in-depth research, well deserved to be the pioneer model. First of all, the technical problem, the early SSL handshake process, there will be no host information, so the server side usually returns the first available certificate in the configuration. Therefore, once a multi-domain name is required to configure the certificate, the return is always the same. SNI (Server Name Indication) is a technique for improving SSL/TLS and is enabled in SSLV3/TLSV1. It allows the client to submit the host information of the request when initiating the SSL handshake request (specifically, the ClientHello phase of the client issuing the SSL request), allowing the server to switch to the correct domain and return the corresponding certificate. So Mr Niu rejected cloud accelerated the development of the SNI-based HTTPS service through Ngx_lua, as long as the user provides HTTPS key files can be accelerated, and even no certificate support has been included in the development process, will soon be available. In addition, the same HTTPS site, the gap can be very large, many sites just add TLS certificate is all right, such as 12306, there is no problem that HTTPS has a huge optimization space, Mr Niu rejected cloud accelerated in the quality of the certificate won praise customers. The world-renowned SSL security and Performance Research Institute Qualys SSL Labs evaluation shows that Mr Niu rejected cloud acceleration supports SSL, the SSLAB standard inspection score can reach a, if the force of HTTPS when the HSTs value amplification some can get a A +, which is already the highest evaluation.

CDN Acceleration  

In the CDN Acceleration link Mr Niu rejected cloud acceleration is divided into the following four aspects:

• Cache optimizations: Cache of nearby nodes, custom cache policies, and support for millions directory cache refreshes

• Network optimization: TCP stack Optimization (CLOUDTCP), keepalive optimization

• Content optimization: Content transfer compression, image content optimization, whitespace removal characters and annotations

• Parent-level back Source: bgp/multi-wire room deploy parent-level nodes, automatically assigned to the fastest parent of the source station, providing centralized cache back to source

< Strong> Data analysis System

         first in the Weblog acquisition link we directly modify the Access_log part of the code, so that it is compatible with the old configuration of the premise, Adds a new option to make the log log into a JSON format. There's a hole in this place. It is important to note that there are many weblog on the web based on the Log_format configuration to generate JSON format, where improper handling of JSON data format can cause a lot of data loss. Second, fluentd,flume,scribe and other log collection tools are based on regular tuples to extract JSON formatted data, in which because of the matching of regular, it is difficult to avoid two problems: the first problem is the performance of the regular, the second problem is that the regular match is not caused by data loss. So we made this change on nginx. We use the Rsyslog V8 version for a full line of Nginxlog to collect the transmission to the Kafka cluster, using the FLUENTD cluster to perform real-time reading of the Kafka cluster's data into the pre-processing output to Es,hdfs, and log merge operations. HDFs data is given to Hadoop to do the offline calculation of platform reports, because it is the second time that the results are overwritten for the first time, so we store the results of each statistic in Redis, which is processed by the platform program. The data given to ES are part of a full-volume acceslog data, provided to our operations and security partners for real-time data analysis and quality testing, and another part of the security attack log index that is dropped to ES, which is used by our customers for analysis. The FLUENTD cluster also stores the logs of our customers for download. In addition, we use storm to conduct a real-time security analysis of the log, here do a lot of interesting things, such as security policy mining, through storm we can detect the attack by other manufacturers in real time, can also find some of our missing attack data, so as to improve our security policy, and help other manufacturers to improve. There is no corner of the Web vulnerability scanning and Webshell analysis of interesting things, here for the moment not to do too much elaboration, have the opportunity to share with you

Stability Assurance

Whether it's safe or accelerating, the business is ultimately based on stability. Stability is the most important, so the Mr Niu rejected cloud to accelerate the adoption of internal and external repair method, in a good self-stability under the premise also help users to enhance the stability of the source station.

Internal:

• Four-layer and seven-layer load balancing

Through load balancing to solve the increasing business volume of traffic load capacity (shunt), but also a good solution to the dynamic expansion of resources, access using four-layer load balancing, dual-entry hot standby, the business layer through seven layers of business separation, and with the help of load balancing, eliminate to the parent service single point of failure.

• Monitoring System Scheduling

Through the monitoring platform, we monitor the quality of the edge to the parent layer, the link quality of the parent layer and the source station, and carry out the automatic dispatching ability.

the Outside:

• Downtime switch
  Support to fill in multiple IPs, after the main IP is hung off can automatically help users to cut to the standby IP to complete the failover.

• Permanent Online
  This is the advantage of the CDN itself, when the site's original server down, users can still access the site's cached data (the style of this page can be customized by the customer), so as to resolve the server failure to bring visitors to the bad experience problem, and does not affect the search engine collection.

• 24x7 Technical Support
  Senior operations and technical support staff are on hand to provide 7x24 hours of technical support to customers and respond quickly to user issues.

Mr Niu rejected cloud acceleration architecture sharing SACC speech