Replay! 2017 O & M personnel who understand these Linux O & M technologies have been given a raise !, 2017 linux

Replay! 2017 O & M personnel who understand these Linux O & M technologies have been given a raise !, 2017 linux

In recent years, the software field has undergone dramatic changes. From underlying infrastructure such as operating systems and databases to basic fields such as distributed systems, big data, cloud computing, and machine learning, from single applications, MVC, and servitization, to application development modes such as microservices, O & M technologies (especially the O & M of Large-Scale Complex Distributed Systems) from IaaS, PaaS, and CaaS to FaaS become increasingly important, IT has become the core of IT enterprises to improve productivity.

As O & M receives more and more attention, the O & M system is gradually enriched, and DevOps and other concepts have emerged to Connect R & D, testing, O & M and other processes. The container technology restructured the O & M from the underlying layer and connects the entire process of development, testing, deployment, operation, and monitoring, it further promotes the gradual migration of the O & M system from tool to platform, automation, and intelligence.

This article will thoroughly sort out and check the O & M technology from the bottom layer to the top layer.

Microservices

Microservice is a concept proposed in recent years. It decouples applications into multiple services to improve its modularization, making it easier to understand, develop, test, and deploy, it is more suitable for quick iterative collaborative development by small teams. At the same time, each service can also adopt different technologies to facilitate continuous evolution. Leading Internet companies in the industry have built microservice frameworks (such as open-source projects based on Spring Boot/Spring Cloud) to deal with their business complexity and efficiency issues during fast iteration. Recently, more and more attention has been paid to microservice configuration management, containerized deployment, automated testing, microservice governance, microservice monitoring, security, and fault tolerance.

SRE

SRE (Site Reliability Engineering, website Reliability Engineering) is a best practice from Google. It is used for service capacity planning and implementation, to ensure service Reliability and performance, more automated tools are built at the software infrastructure level to replace manual operations, so as to better meet the complex and changing business needs.

DevOps & CI/CD

DevOps has gradually become the mainstream of software development, and containers have rapidly grown into the core of DevOps over the past two years. They have also been paid more and more attention to continuous integration, continuous deployment, and continuous release. With the emergence of new DevOps automation tools, the maturity of containers and their related ecosystems (especially container orchestration tools and their support for stateful services), and the wide application of microservices, more and more related tools will be integrated in the continuous integration process, and automated continuous testing will become more popular, in this way, you can effectively control quality, ensure security, reduce costs, control risks, and improve efficiency, and more efficiently support complex large-scale distributed applications.

Container optimization and practices

Over the past few years, Docker-centered container technology has continued to evolve and has become a key technology in IT infrastructure with its simplicity of construction, distribution, and deployment. Container technology connects application R & D, testing, and O & M through standardized operating environments. It is simple, lightweight, highly portable, and can use resources more efficiently. It can also effectively solve software dependency problems, improve R & D efficiency, and reduce R & D costs, therefore, the industry continues to optimize its software release process through containers to containerize its Legacy programs.

However, container technology also faces many challenges. In the future, there will be much room for improvement in terms of container standardization, container security, container network, container storage, especially support for stateful services such as databases, the manageability and ease of use of containers also need to be further improved.

Container orchestration and Management

With the wide application of Docker and other container technologies, container orchestration and management have attracted more and more attention, the emergence of Kubernetes, Apache Mesos, Docker Swarm Mode and other excellent open source ecosystems and solutions. They try to transition the current resource-centric management mode to the application-centric management mode, and try to make up the basic components (such as configuration, service, and load balancing) of the application) to achieve better manageability. With the development of CaaS, the number of private or public container clouds is also increasing, and the user experience is getting better and better, thus significantly reducing migration costs.

However, in large-scale practice, gray release, resource scheduling, isolation, O & M monitoring, logs, and other aspects still need to be further developed and standardized, and cross-Data Center application management is in place, hybrid cloud environment support, cross-cloud service migration, and security still face difficulties and challenges.

Automated O & M

With the advent of virtualization and containerized technologies, the complexity and difficulty of O & M management have greatly increased. Therefore, O & M automation must be achieved through professional, standardized, and streamlined means. Many automation tools have emerged in the industry to improve efficiency, such as Puppet, Chef, Ansible, and Saltstack. Major Internet companies have gradually evolved from automated tools to an all-in-one automated O & M management platform, enabling one-stop processing of deployment, configuration, monitoring, and alarms, it standardizes and standardizes resources and processes, and visually manages application running states to improve O & M quality and reduce O & M costs.

Intelligent O & M

With the continuous expansion of the monitoring scope, the data generated by it has the characteristics of diversity, multi-dimensional and non-structured, and may be related to business data. The traditional manual analysis and processing method is inefficient and costly. With the rise of big data and artificial intelligence, more and more intelligent analysis algorithms are also applied to the O & M field. They analyze the massive data volumes owned and generated by the O & M system, it plays a major role in problem locating, traffic prediction, decision-making assistance, intelligent alarm, and automatic fault recovery, thus further reducing O & M costs.

Basic O & M Architecture

The O & M infrastructure covers the management of networks, machines, data centers, racks, and storage, it covers basic resources, rack design and delivery, network architecture design, data architecture planning, operating systems, system software, Environmental delivery, and replacement of decommission machines. The industry has constructed CMDB to support service delivery processes and corresponding management processes, and has also built tools such as initialization, deployment, operation, monitoring, and log. With the development of virtualization, containerization, and cloud computing, the O & M infrastructure also changes from providing resources to providing capabilities, improving the transparency of applications on the infrastructure, and thus improving the flexibility of the infrastructure.

Database O & M

Database O & M involves a series of services such as database deployment architecture, capacity planning, performance tuning, data backup and recovery, data migration, database monitoring and auditing, database O & M management, and troubleshooting.

With the wider use of the Internet, database O & M has also taken a new form. In recent years, a series of tools have been used to minimize the overall unavailable time of the database in the deployment mode, online table mode change, massive data migration, and troubleshooting of multiple active standbys in different regions, to minimize the impact on users. At the same time, to simplify the deployment and management of databases, it has gradually become one of the hot topics to manage and schedule databases in a containerized manner. Finally, in terms of intelligent O & M, through the analysis and mining of various database indicators, intelligent diagnosis solutions are provided to predict and control risks in advance, and the overall stability of the processing efficiency system is improved.

Big Data O & M

With the rapid growth of data, the Hadoop-based ecosystem also plays an increasingly important role. It covers offline computing, stream computing, ad hoc queries, and other usage methods, there are also outstanding open source projects such as Hadoop, Spark, Kafka, Hbase, Storm and Phoenix. In the O & M of the big data platform, the O & M of big data becomes extremely complex due to issues such as distributed architecture, storage of multi-source heterogeneous massive data, and more diversified and complicated data processing frameworks.

The main goal of big data O & M is to improve resource utilization, reduce the O & M complexity of big data systems, and improve user friendliness. Among them, the unified management and scheduling capabilities of computing resources, the hybrid deployment capability of various types of big data systems based on containers, fast elastic resizing capability, and cross-Data Center disaster tolerance capability, big Data Application monitoring and fast and flexible Fault Locating become more and more important.

O & M monitoring

Monitoring is an important part of IT system O & M to ensure the stability and availability of core services. IT covers network, host, business, application, and performance, and involves fast Fault Notification, precise Fault Locating and performance analysis and diagnosis. Currently, open-source monitoring software is widely used in the industry, including Nagios, Cacti, Zabbix, and Ganglia.

With the rapid expansion of application scale and the rapid development of DevOps, microservices, containers and other technologies, monitoring has also emerged. The industry has evolved from a Nagios-style monitoring method to a streaming style. It uses stream processing of massive monitoring metrics and a visual platform to display monitoring metrics in real time. In addition, as the infrastructure becomes more dynamic, monitoring not only cares about the running status of a single node, but also cares about the health status of the entire business system. Full-link tracing and other technologies have also been widely used.

O & M Security

In the context of Internet and mobile applications, applications are gradually migrated to the cloud, traditional boundaries become increasingly blurred, and security has a new development trend. In the past, security technologies used traditional firewalls and intrusion defense systems. Now, in addition to strengthening traditional security measures, some security practices will be introduced in the development process, including threat modeling, automatic security scanning, and security functional testing, this reduces security risks and shortens the feedback cycle of security issues. At the same time, security is also engaged in prevention first, turning to continuous detection and rapid response. Through continuous detection of attack behavior, security events are quickly responded to, thus greatly reducing losses.

Game Development and O & M

In recent years, online games have grown rapidly, and the trend of using universal frameworks and engines for game development has become increasingly apparent. In terms of game O & M, in addition to conventional O & M methods, the game also has its own characteristics. First, client games, web games, and mobile games have different forms in terms of networking methods, distribution channels, and lifecycles, therefore, O & M poses challenges to network access, multi-channel distribution, capacity planning, network latency, and highly reliable access to archival data.

Second, due to unpredictable user growth, game O & M must be capable of rapid scaling up and down. The hybrid cloud or public cloud technical architecture is used to maximize scalability. Finally, in the case of large-scale DDOS abnormal traffic attacks, game O & M should have a multi-level traffic cleaning protection mechanism and the ability to downgrade services to ensure availability as much as possible.

Internet Finance and O & M

In recent years, internet finance has experienced explosive growth, and Fintech has also injected technological innovation genes into it. Microservices, containerization, big data, and cloud computing provide the foundation for rapid iteration of internet finance. However, compared with the current application O & M, the internet finance industry has its own characteristics, it has strong demands and even mandatory financial supervision in terms of data retention, security compliance, attack prevention capabilities, payment and liquidation, financial supervision, data security, big data risk control, and high-level security protection. specification, it also poses a higher challenge to the O & M of internet finance.

Free Linux cloud computing courses are booming,5 days freeO & M dry goods content free of charge,Click the end of the article"Read original"Free O & M courses!Speed to grab ~~~~

PS: Check out the free gift package provided by xiaobian ~

Free | more than 1000 sets of resume templates are offered free of charge, and a resume creation tutorial is provided!

Free of charge | the "Shell script 100 cases" ebook is free of charge and must be used for O & M ~

Click 【Read original], Free to listen to five days of Linux O & M Dry Goods sharing courses, hot discussion, come to grab!

• This article has been included in the following columns: