Automated enterprise O & M

From http://bbs.chinaunix.net/thread-3779206-1-1.html

System installation is handed over to IDC personnel to minimize installation standards
Nagios and cacti are most used for production
Controltier, zabbix, and puppet -- these have been tested and tested.
Currently, batch password modification is based on scripts.
Unified host management and login are a third-party product bought
I developed an O & M management system using Python internally.
Php development database backup Check Tool

++
1. Do not deploy (or do not deliberately deploy) the agent
Ssh, ssh + ipvt for automatic response, other ssh-based (Serial running, parallel running, etc)
Advantage: convenient, no need to deploy (for example, sshd is generally installed by default and daemon is enabled)
Disadvantage: slow

2.
Puppet, cfengine, self-written (such as Python, Perl, etc., shell does not implement socket-level seemingly)
Advantage: fast
Disadvantages: Of course, deployment is required, and it is not easy to get started, and it cannot be done if you develop it yourself (I am a little white, I don't have this capability)

I can think of so many errors.

++

I. Application categories of O & M automation tools
I understand these categories.
1. system installation Initialization
2. Environment deployment, configuration change, process monitoring, and log monitoring
3. Code Distribution

Ii. Common O & M automation tools
Bacula cobbler zabbix cfengine swatch... the most important thing is the script.

Iii. Briefly introduce the functions of the current automated tools
Currently, cobbler is used to automatically install the system and initialize the system. cfengine is used to modify the configuration, and SVN + hook is used to automatically release the code.
Swatch is also used for Application log monitoring to automatically restart the service, and bacula is used for centralized data backup management.


Understanding of automated O & M
In fact, for O & M engineers, do not manually do anything that can be done automatically. The tasks that can be handed over to the program are the motivation for automated O & M. The existing tools are just
Automated O & M is more convenient. Many times, adding a shell script to a scheduled task can meet the requirements of automated O & M.
The vast majority of machines in various Internet companies are automated. However, these are core technologies that involve security and will not be released for small and medium-sized enterprises.
Enterprises do not have dedicated O & M developers. They only rely on O & M engineers to think more and try more. However, O & M engineers generally have poor development capabilities, which makes it difficult to make too many things.


Experience
The best tool is a tool suitable for your own business. A set of automated O & M tools are also composed of a small module. There is also the need to determine the accuracy of automated O & M before automation
Do not think that there is no problem when you do a good job of testing. After running the test for a while, you will find that you have not achieved the expected purpose and even caused business impact and loss. Manual check and confirmation are necessary for some work.

++

In fact, there are only two types of automated O & M tools.
1. Monitoring and alarms
This is a lot of open source, and many companies develop their own. However, I think it is better to select an open-source player, and there is no need to create wheels repeatedly. Taking plotting as an example, it is estimated that few self-developed graph monitoring systems have powerful cacti functions. Cacti is sufficient for small and medium-sized enterprises. If the enterprise is large, there are indeed too many machines, you can consider distributed monitoring, you can consider using multiple cacti monitoring by function or by region. In other words, there are always methods, depending on how you balance them.


2. Batch Management
Chifeng, puppet, CT, bulk shell, and many self-developed tools.
In fact, I think the batch management tool can meet three requirements.
(1) tool security cannot be the entrance to hacker attacks
(2) function verification: checks and filters the Issued commands. If an incorrect or dangerous command is found, the execution is ignored to avoid damage to the system.
(3. features are not too powerful. Generally, passwords, host names, software installation, and configuration files are modified in batches.
For batch management, it is almost the same for small and medium-sized enterprises to be able to play well with shell or Python.

++

1. Use Linux + KVM on your own virtualization platform. proxmox is recommended. Although it is not very stable in some small places, the overall function is normal, including migration, backup, and recovery;
2. Use Nagios monitoring internally and configure SMS alarms. Basically, you don't have to worry about accidents. You can also write a simple monitoring script or something based on your actual situation;
3. There are a lot of systems, but they belong to multiple systems and are not allowed to operate in batches.

++

1. Your understanding of automated O & M
Use scripts or tools that are periodically executed to replace manual commands to complete a job,
Advantage: quantifiable and queryable, reducing repetitive work and reducing the risk of human error ....


2. Share your experience in using these automated tools
System deployment: Cobbler
System update and Application Deployment: puppet
Version Control: git
Remote Logon: VNC netsupport (this is chargeable)
Monitoring: Nagios, cacti
Virtualization: KVM xen VMware (which is used by the company now)
Backup: tar and find write scripts

++


++

Automated enterprise O & M