Managing Linux servers in bulk with ansible: Configuring Inventory and Batch execution commands

Ansible is a relatively new automated operations tools, based on Python development, the collection of many operations tools (puppet, Cfengine, Chef, func, fabric) The advantages of the implementation of batch system configuration, batch program deployment, batch Run command and other functions.
Ansible is a module-based operation and does not have the capacity to deploy in bulk. The real batch deployment is the module that Ansible runs, and Ansible just provides a framework. Mainly include:
(1), connection plug-in connection plugins: responsible for and be monitored to achieve communication;
(2), host Inventory: Specifies the operation of the host, is a configuration file inside the definition of monitoring host;
(3), various modules: Core module, command module, custom module;
(4), with the help of the plug-in to complete log mail and other functions;
(5), playbook: When a script performs multiple tasks, it is not necessary to allow the node to run multiple tasks at once.
Installation and deployment of ansible I'm not going to talk about it here, so you can refer to the official documentation. Here I will be divided into three parts to explain: 1, configuration inventory,2, Batch execution command, 3, playbook job.
System environment:

Server	IP Address	Operating System	Required Software
Ansible Host	192.168.2.203	Centos 7 64-bit	Ansible
Remote machine 1	192.168.2.205	Centos 7 64-bit	httpd
Remote Machine 2	192.168.2.208	Centos 7 64-bit	httpd

First, the ansible commonly used five commands

1, Ansible
Ansible is the core part of the instruction, which is mainly used to execute the AD-HOC command, a single command. Command format: ansible Example: Viewing the date information for a host
ansible 192.168.2.205 -a "date"
2, Ansible-doc
This instruction is used to view the module information, the common parameters are two-L and-s,-l is to view all modules that Ansible has installed.
Example: viewing information for a cron module
ansible-doc -s cron
3, Ansible-galaxy
This directive is used to download third-party extensions from the https://galaxy.ansible.com/site, equivalent to the Yum, Pip, Easy_install, and other commands.
Example: Installing a third-party extension module
ansible-galaxy install <模块名>
In real-world applications, you can also specify TXT or yml files for downloading and installing multiple components, as described in the official documentation.
4, Ansible-config
View, edit, and manage ansible profiles
Example: Show only changed configurations
ansible-config dump --only-changed
5, Ansible-playbook
This directive is very important, by reading the playbook job file, executed on the node host, the equivalent of executing a shell script function, ansible automation is indispensable, but also I want to focus on the following.

Second, create SSH key authorization

Ansible is a remote server managed via SSH, so you first need to get through the SSH channel. When we manage large-volume servers, it is best to use the SSHD service's secure key authentication method in order to avoid the need to repeatedly enter a password for remote operation.
With Ssh-keygen to manage the key, even if the password is set when the key is created, a new session is created after the Ssh-add is executed, and each subsequent connection is not required to enter the key password. It is recommended to use SSH key authorization, and do not set the key password, so that it can be better to achieve automation.

ssh-keygen   # 创建公钥和私钥文件ssh-copy-id 192.168.2.205   # 将公钥文件copy到远程主机上ssh-copy-id 192.168.2.208ssh-agent bash   # 通过ssh-agent来管理密钥ssh-add ~/.ssh/id_rsa  # 将私钥交给ssh-agent管理，避免每次连接都需要输入密钥密码

Third, configuration inventory file

The default inventory file is/etc/ansible/hosts, similar in format to the INI configuration file for Windows
1, by IP address to configure
For example: I now have 2 hosts with IP addresses of 192.168.2.205 and 192.168.2.208, and both hosts are Web servers, so you can define a web_server group as follows:

[web_server]192.168.2.205192.168.2.208

If you modify the SSH port number, you can define it as follows:

[web_server]192.168.2.205:5210192.168.2.208:5210

If I have a host IP for 192.168.2.203 that does not belong to any group, you can define it as follows:

192.168.2.203[web_server]192.168.2.205192.168.2.208

Set an alias for the host, in this case, the "WebServer1" Alias will connect 192.168.2.205.

Webserver1 ansible_ssh_port=5210 ansible_ssh_host=192.168.2.205Webserver2 ansible_ssh_port=5210 ansible_ssh_host=192.168.2.208

You can set the connection user name and connection type individually:

[web_server]192.168.2.203       ansible_connection=local192.168.2.205:5210   ansible_connection=ssh   ansible_ssh_user=xuad192.168.2.208:5210   ansible_connection=ssh   ansible_ssh_user=andyxu

2. Configure by host name
by hostname configuration and by IP address configuration, just change IP to host name, for example:

[web_server]web1.comweb2.com[db_server]db1.com:5210db2.com:5210[targets]ansible.com       ansible_connection=localother1:5210   ansible_connection=ssh   ansible_ssh_user=xuadOther2:5210   ansible_connection=ssh   ansible_ssh_user=andyxu

For example, there is a set of DB servers, db1.xuad.com to db50.xuad.com, of 50 DB servers

[db_server]db[1:50].xuad.com:5210

If dba.xuad.com to dbf.xuad.com, a total of 6 DB servers

[db_server]db[a:f].xuad.com:5210

3. Set the variables for the host
Assigning variables to hosts is simple, and these variables can be used in playbooks after they are defined.

[web_server]web1.com  http_port=80  pathname=/etc/httpd/web2.com  http_port=8081  pathname=/etc/nginx/

4. Assigning variables to groups
The following example defines two variables for the Web_server group, Http_port and pathname.

[web_server]web1.comweb2.com[web_server:vars]http_port=80pathname=/etc/httpd/

5. Use one group as a child of another group
The following example assigns the Web_server and db_server two groups to the Some_server group as subgroups of the Some_server group, assigns variables to the Some_server group, and assigns the Some_server group to the User_ The server group, as a subgroup of the User_server group.

[web_server]192.168.2.205192.168.2.208[db_server]db1.com:5210db2.com:5210[some_server:children]web_serverdb_server[some_server:vars]http_port=80nginx_port=8080pathname=/etc/ansible/xuad=foo.xuad.com[user_server:children]some_serverother_serverNorthserver

6, sub-file definition host and group variables
Suppose you have a host of 192.168.2.205, belonging to the Web_server group, and now define variables for the Web_server group and the 192.168.2.205 host, the Hosts file adds the following:

/etc/ansible/group_vars/web_server/etc/ansible/host_vars/192.168.2.205

Creating Group_vars and Host_vars directories

mkdir /etc/ansible/group_varsmkdir /etc/ansible/host_vars

Then edit the Web_server file, you can define the variable as follows, note: There is a space after the number.
Vim/etc/ansible/group_vars/web_server

http_port: 80http_path: /data/httpd/ntp_server: ntp.xuad.comdb_server: db1.com

Edit the 192.168.2.205 file again to define the variable as follows
vim/etc/ansible/host_vars/192.168.2.205

nginx_port: 8081nginx_path: /data/nginx/db_server: db2.com

There are further applications, you can create a directory for a host or a group, directory name is the host name or group name, the directory can create multiple files, the variables in the file will be read to the main machine or group variables, the host file content is as follows:

/etc/ansible/group_vars/web_server/192.168.2.205/etc/ansible/group_vars/db_server/db1.com

The parameters of the inventory file can be viewed in the Ansible official documentation, below we look at an example of a host file in the official documentation and what each line means.

#给some_host主机定义了ssh连接的端口号和用户名some_host         ansible_ssh_port=2222     ansible_ssh_user=manager#连接aws_host主机时将通过以下定义的私钥文件进行验证aws_host          ansible_ssh_private_key_file=/home/example/.ssh/aws.pem#给freebasd_host主机指定了python的路径freebsd_host      ansible_python_interpreter=/usr/local/bin/python#给ruby_module_host主机指定了ruby运行的路径ruby_module_host  ansible_ruby_interpreter=/usr/bin/ruby.1.9.3

Ansible can also obtain inventory configuration information from the outside, in the following ways:
1. Pull inventory from the cloud
2. Get configuration from LDAP
3. Get configuration from Cobbler
4. Get the configuration from the CMDB
How to get the inventory configuration from the outside I'm not going to talk about it here. You can view official documents.

Iv. ansible Batch Execution command

First define a host group in the inventory configuration file/etc/ansible/hosts, as follows:
Vim/etc/ansible/hosts

(1) Let's ping first and see how the two hosts are connected.
ansible web_server -m ping

(2) See which hosts are under the Web_server group
ansible web_server --list

(3) View memory usage of a host computer
Command format: Ansible < hosts or host group >-a "shell command"
Note: Execute a command using the-a parameter followed by the shell command
ansible 192.168.2.205 -a "free -h"
(4) Batch Create a regular user in the Web_server group
ansible web_server -a "useradd andy"
(5) Restart the HTTPD service for all hosts
Command format: Ansible < host group >-m < module name >-a < Specify execution parameters >
Note: The all parameter is used to execute commands on all hosts configured in the Hosts file
ansible all -m service -a "name=httpd state=restarted"
You can also use the ansible all-a "systemctl restart httpd" statement, the results are the same, but the output information is different, you can test to see.
(6) View the UID and GID of the specified user for each host under multiple groups
Command format: Ansible < host group 1>:< host group 2>:< host group 3>-a "shell command"
ansible http1:http2 -a "id andy"

(7) to exclude a specific group, create a new file under the/tmp directory of the hosts that are not part of the HTTP1 group under the Web_server group
Command format: Ansible ' < host group 1>:!< host group 2> '-a ' shell command '
Note: The host that executes the command belongs to the Webserver group, but does not belong to the HTTP1 group
ansible ‘web_server:!http1‘ -a "touch /tmp/test.txt"

At this point, Ansible will give a warning message, meaning that Ansible has a file module built-in, it is recommended to use the files module to create documents, if you want to suppress the warning message, you can modify the configuration file Ansible.cfg, the Command_ If the warnings parameter is changed to False, no warning message will be given. In fact, the creation of the file can use the following command, the result is the same, about the use of the module will be described later.
ansible ‘web_server:!http1‘ -m file -a "dest=/tmp/test.txt state=touch"
(8) Specify the intersection of two groups to see the load of hosts belonging to the Web_server group and belonging to the HTTP1 group
NOTE:! Represents groups that belong to the previous group but do not belong to the following group;& represents the group that belongs to the previous group.
ansible ‘web_server:&http1‘ -a "uptime"

(9) A host that belongs to the Web_server group and belongs to the HTTP2 group but does not belong to the HTTP1 group Delete/tmp/test.txt file contents
ansible ‘web_server:!http1:&http2‘ -a "rm -f /tmp/test.txt"
(10) wildcard use to view the character encoding of the host for all groups beginning with HTTP
ansible http* -a "cat /etc/locale.conf"
(11) View disk usage for all hosts with all groups and web_server groups that begin with HTTP
ansible http*:web_server -a "df -h"
(12) View the hostname of the first host of the Web_server group
ansible web_server[0] -a "hostname"
(13) View the current date and time of the first 5 hosts in the Web_server group
ansible web_server[0:4] -a "date ‘+%Y-%m-%d %H:%M:%S‘"
(14) Regular expression used to view the process PID of the HTTPD service for a host of groups starting with Web or HTTP
ansible ‘~(web|http)*‘ -a "pidof httpd"
(15) Examples of other regular expressions

ansible ‘~(web|http)2‘ -a "pidof sshd"  # web2或者http2主机或者组ansible ‘~(^web)*‘ -a "pidof sshd"  # 以web开头的主机或者组ansible ‘~(web|db).*\.example\.com‘ -a "pidof sshd"  # web.*.example.com或者db.*.example.com主机或者组

(16) If you do not want to use ssh-agent and want to use SSH in a password-authenticated manner, you can use the--ask-pass or-K parameter when executing the ansible command.

ansible web_server -a "shell命令" --ask-passansible web_server -a "shell命令" -k

(17) Batch Restart the server, plus the-F 10 option to fork out 10 sub-processes (bash) to execute the reboot command in parallel, that is, each restart 10
ansible web_server -a "/sbin/reboot" -f 10
(18) When executing the ansible command, the default is to execute as the current user, if you want to execute the command with the specified user, you need to add the-u username option

ansible web_server -a "shell命令" -u username  # ssh安全密钥验证方式ansible web_server -a "shell命令" -u username -k  # ssh账号密码验证方式

(19) Execute commands with sudo
If sudo is not required to enter a password, it does not add--ask-sudo-pass, and if sudo needs to enter a password, it must be added--ask-sudo-pass parameter. The recommended setting to sudo does not require a password to be entered, which makes it easier to automate.

ansible web_server -a "shell命令" -u username --sudoansible web_server -a "shell命令" -u username --sudo --ask-sudo-pass

(20) You can also use sudo to switch to other users, not the root user, via the-sudo-user or-u option.
ansible web_server -a "shell命令" -u username -U otheruser [--ask-sudo-pass]
Ansible has a number of modules, the default is "command", that is, the commands module, which we executed before the statements are executed with the command module, see an example below.
(21) Statistics of the number of users who cannot log on to the system, i.e. Nologin users
ansible web_server -a "grep ‘/sbin/nologin‘ /etc/passwd | wc -l"
The results will be error-not get the result we want

This is because the command module does not support shell command variables, pipe characters, etc., if you want to use shell-related things, you can use the Shell module, with the-m parameter to specify the module to run.
ansible web_server -m shell -a "grep ‘/sbin/nologin‘ /etc/passwd | wc -l"

(22) View the Help information of the module
ansible-doc shell
(23) View the IP addresses of each host in the Web_server group
ansible web_server -m shell -a "/sbin/ifconfig | grep ‘inet ‘ | awk ‘{print \$2}‘ | sed -e ‘/127\.0\.0\.1/d‘"
Note: The shell module needs to use the escape character when using the awk command *
(24) View the contents of the path variable for each host in the Web_server group
ansible web_server -m shell -a ‘echo $PATH‘
Note: If you use the double quotation mark "Echo $PATH", you will find the value of the PATH variable in the current system, which involves the rules of the shell quotes. *
In the We see clearly that the result of double and single quotation marks is not the same, the single quotation marks are the result of the remote machine, and the double quotation marks are actually the result of the ansible local machine.

Ansible is able to concurrently SCP a large number of files to multiple machines, copy the local hosts file to the/tmp directory of all hosts in the Web_server group
Command format: Ansible < hosts or host group >-M copy-a "src=< local directory or file > dest=< remote directory or File >"
ansible web_server -m copy -a "src=/etc/hosts dest=/tmp/"

(26) Use the file module to modify the owner and permissions of the files, which can be replaced by the Copy module, is equivalent
Command format: Ansible < hosts or host group >-M file-a "dest=< remote directory or File > mode=< permissions > owner=< user > group=< user group >"

ansible web_server -m file -a "dest=/tmp/hosts mode=600"ansible web_server -m file -a "dest=/tmp/hosts mode=600 owner=xuad group=xuad"ansible web_server -a "ls -lh /tmp/hosts"

(27) Creating a directory with the file module, similar to performing a mkdir-p effect
ansible web_server -m file -a "dest=/tmp/test mode=755 owner=xuad group=xuad state=directory"
(28) creating files using the file module, similar to performing touch effects
ansible web_server -m file -a "dest=/tmp/test.txt mode=755 owner=xuad group=xuad state=touch"
(29) Create a soft link file using the file module
ansible web_server -m file -a "src=/root/test.txt dest=/tmp/test.txt state=link"
(30) Delete directory (recursive delete) and delete file, absent means delete

ansible web_server -m file -a "dest=/tmp/test state=absent"ansible web_server -m file -a "dest=/tmp/test.txt state=absent"

Ansible provides support for Yum and apt, here's an example of Yum
(31) Confirm that a package is installed, but do not upgrade it
ansible web_server -m yum -a "name=wget state=present"

(32) Upgrade a package to the latest version
ansible web_server -m yum -a "name=wget state=latest"

(33) Uninstalling a package, uninstalling the wget
ansible web_server -m yum -a "name=wget state=removed"

(34) Verify that a package is not installed and that wget is no longer available.
ansible web_server -m yum -a "name=wget state=absent"

(35) Install a software package, install wget by yum mode
ansible web_server -m yum -a "name=wget state=installed"

(36) Delete a user, delete Andy user
ansible web_server -m user -a "name=andy state=absent"
(37) Verify that a user exists and does not exist create this user
ansible web_server -m user -a "name=andy state=present"
(38) Create a new user and specify the user group and home directory
ansible web_server -m user -a "name=andy groups=xuad home=/home/andy1"
(39) Create a new user and set the password
ansible web_server -m user -a ‘name=andy password="<crypted password here>"‘
Note: <crypted password here> is a sha-512 encryption algorithm after the value of encryption, need to install Passlib, password value to be enclosed in double quotation marks, outside with single quotation marks
Generate the sha-512 algorithm password in the following ways

pip install passlibpython -c "from passlib.hash import sha512_crypt; import getpass; print sha512_crypt.encrypt(getpass.getpass())"

(40) We sometimes need to download a software project using Git, which can use the Git module
ansible web_server -m git -a "repo=foo.example.org/repo.git dest=/srv/myapp version=HEAD"
(41) service start, restart, stop, enabled=yes means to join the boot start

ansible web_server -m service -a "name=httpd state=started"ansible web_server -m service -a "name=httpd enabled=yes state=started"ansible web_server -m service -a "name=httpd state=restarted"ansible web_server -m service -a "name=httpd state=stopped"

Long-running operations can be performed in the background,--do-stuff parameters are required, ansible checks the status of the task, and the same job ID is assigned to the same task performed on the host.
(42) Background Execution command, up to 3,600 seconds,-B indicates the maximum time for background execution
ansible all -B 3600 -a "/usr/bin/long_running_operation --do-stuff"
(43) Check the status of the task run, using the Async_status module, the above executes the background command will return a job ID, the ID passed to the Async_status module
ansible all -m async_status -a "jid=488359623657.1326"
(44) Background Execution command, up to 1800 seconds, that is, 30 minutes,-p means check the status every 60 seconds, the default 15 seconds
ansible all -B 1800 -P 60 -a "/usr/bin/long_running_operation --do-stuff"
(45) View system information for a host computer
ansible 192.168.2.205 -m setup
Note: These system information variables can be called directly inside the playbook
(46) View the date and time information for a single host
ansible 192.168.2.205 -m setup -a ‘filter=ansible_date_time‘
(47) To view memory-related information for a single host
ansible 192.168.2.205 -m setup -a ‘filter=ansible_*_mb‘
(48) View the network card information of a host computer
ansible 192.168.2.205 -m setup -a ‘filter=ansible_ens3[0-5]‘
(49) Schedule Task module Cron, synchronize time every 6 hours
ansible web_server -m cron -a ‘name="modify ntp server" minute=0 hour="*/6" job="/usr/sbin/ntpdate ntp.xuadup.net"‘

The Ansible cron module is actually creating a crontab on the remote host (automatic scheduled Task), we go to the remote host to execute CRONTAB-E look at

(50) automatic execution of a script after system restart
ansible web_server -m cron -a ‘name="a job for reboot" special_time=reboot job="/root/xuad.sh"‘
(51) Delete a scheduled task
ansible web_server -m cron -a ‘name="a job for reboot" state=absent‘
(52) Mount Partition
ansible web_server -m mount -a ‘name=/data src=/dev/sdb1 fstype=ext4 opts=rw state=mounted‘
(53) Unmount partition
ansible web_server -m mount -a ‘name=/data state=unmounted‘
(54) Ensure that a partition is mounted and mounted if it is not
ansible web_server -m mount -a ‘name=/data src=/dev/sdb1 fstype=ext4 state=present‘
For more information on playbook, please continue to follow my post, thank you!

Managing Linux servers in bulk with ansible: Configuring Inventory and Batch execution commands