Quickly run commands across all IBM AIX servers using scripts and utilities

Brief introduction

In most companies, system administrators are required to do more with fewer resources. With virtualization technologies such as IBM PowerVM, the number of IBM AIX instances that system administrators must manage has grown significantly. PowerVM supports running dozens of or even hundreds of IBM AIX logical Partitions (LPARs) on a single hardware. Many years ago, a system administrator might be responsible for managing only more than 20 instances of AIX, and in today's environment, it is common for system administrators to manage hundreds of Aix lpars.

If you are in a large AIX environment with many lpars, it is impractical to log on to each server when you need to change or need to gather information from each server. In these large environments, the best choice is to use the technology that supports the fast running of commands or scripts across many servers. This article describes a variety of options for running commands and scripts across a large number of servers.

SSH Key Overview

All of the options discussed in this article require an SSH key to be set between a central management Server and the rest of the servers.

The first step is to pick a central management server to use. The server you pick should be able to access all other servers in your environment as securely as possible. One option is to create a new LPAR for this purpose and lock it as much as possible. Other potential options include network installation Manager (NIM) server or System Monitor server.

The next step is to pick a user account to use. Many of the tasks that you want to automate can require root user privileges. However, the direct use of the root account is not recommended because of security issues with prefixes. A good alternative would be to create a normal user account and use a utility (such as sudo) to give it access to the root user.

After you determine which server to use as the central Management Server and determine a user account, the next step is to generate an SSH key. There are many tutorials that introduce SSH keys, and this article does not discuss the topic in detail. Basically, you need to use the Ssh-keygen command to create a private key and public key on the central management Server. The private key is used only on the central Management Server, and the public key can be replicated to all other servers.

A new user account needs to be established on each server to create a ". Ssh/authorized_keys" file in the user's home directory and put the public key in it. If you plan to run commands with root user-level access, you should also set up sudo or similar utilities to grant additional access to the user. If you use sudo, you can use the "nopasswd" option so sudo will no longer prompt the user for a password.

To verify that the SSH key is valid, you can try using SSH to run a remote command on a server, such as hostname. For example, if the server name is "Server1", then you should be able to type SSH server1 hostname, and the response should be server1. When you use sudo, you should be able to run the SSH Server1 sudo su-c whoami command to verify this and finally return to root.

Run a single command on multiple servers using a simple for loop

One way to run a command remotely is to use a simple one-line loop command. The name of the server on which to run the command will be read from a file. Each row of the file has a server name. Listing 1 shows an example of running the For loop command line on multiple servers.

Listing 1. An example of an SSH for loop

For server in ' cat serverlist '; Do printf "%-20s" $server;

Ssh-q-O "Batchmode yes" $server oslevel-s 2>&1; Echo Done | Grep-v ^$

Let's analyze This example in detail:

For server in ' cat serverlist '; do– creates a loop where each server in the ServerList file is processed by the loop, and the server name is set to $server variable.

printf "%-20s" $server; – Display the name of the server and leave-align by filling in a space. This will result in the server name being neatly exported.

Ssh-q-O "Batchmode yes" $server oslevel-s 2>&1; – Use two options to invoke the SSH command. The-q option prevents the login window from appearing. The-o "batchmode yes" option causes SSH to run in a batch mode without prompting the user to enter any information. $server will change the name of the current server through a for loop. In this example, Oslevel-s is the command we need to run on a remote server. The 2>&1 option is used to redirect standard errors to standard output. Without this option, the output may be scrambled because there is no buffer to hold the standard error.

Echo – Make sure that the cursor moves to the next line, regardless of whether the command generates output, so that the next server name in the loop is arranged in the correct position. To do so, a carriage return is used at the end of each line, and the correct format is set if no output is returned after the command is executed. If the output is generated after the command is executed, an extra blank line is generated, but the rows will be deleted in the next step.

Done | Grep-v ^$– ends the for loop, and then filters and deletes all the blank lines in the output.

Figure 1 shows an example of this command line and output. The file serverlist has a list of server names (one per line), and the command to run on each server is oslevel-s.

Figure 1. A For loop command line and its output example