Tutorials for writing command-line tools for UNIX-like systems using Python

Introduction

Can you write command-line tools? Maybe you can, but can you write a really useful command-line tool? This article discusses using Python to create a robust command-line tool with built-in Help menus, error handling, and option handling. For some strange reason, many people don't know Python. The standard library has all the tools you need to make the most powerful *nix command-line tool.

As you can say, Python is the best language for making *nix command-line tools because it works in the philosophical Way of "batteries-included" and emphasizes the provision of highly readable code. But just as a reminder, when you discover how simple it is to create command-line tools using Python, these ideas are dangerous and your life can be messed up. As far as I know, there are no articles detailing the use of Python to create command-line tools, so I hope you enjoy this article.
Set up

The Optparse module in the Python Standard library can do most of the trivial work of creating command-line tools. Optparse is included in Python 2.3, so the module will be included in many *nix operating systems. If for some reason you are using an operating system that does not contain the required modules, it is fortunate that the latest version of Python has been tested and compiled into almost any *nix operating system. The systems supported by Python include IBM? AIX, HP-UX, Solaris, free BSD, Red Hat Linux, Ubuntu, OS X, IRIX, and even several Nokia phones.
Create the Hello World command-line tool

The first step in writing good command-line tools is to define the problem to be solved. This is critical to the success of your tools. This is equally important for solving problems in the simplest possible way. The Guidelines for KISS (Keep It simple Stupid) are explicitly adopted here. adding options and adding additional functionality is only possible after implementing and testing the planned features.

Let's start with creating the Hello World command-line tool. As suggested above, we use the simplest possible terminology to define the problem.

Problem definition: I want to create a command-line tool that prints Hello world by default and provides the option to print out the names of people who do not pass.

Based on the instructions above, you can provide a solution that contains a small amount of code.
Hello World Command Line Interface (CLI)

 #!/usr/bin/env python
     import optparse
     
     def main():
      p = optparse.OptionParser()
      p.add_option('--person', '-p', default="world")
      options, arguments = p.parse_args()
      print 'Hello %s' % options.person
     
     if __name__ == '__main__':
      main()

If you run this code, the expected output is as follows:

Hello World

But we can do much more than that with a small amount of code. We can get the auto-generated Help menu:

python hello_cli.py --help  
     Usage: hello_cli.py [options]
     
     Options:
     -h, --help      show this help message and exit
     -p PERSON, --person=PERSON


  

	
   


  

As you can see from the Help menu, there are two ways to change the output of Hello world:

Python hello_cli.py-p Guidohello Guido

We also implemented auto-generated error handling:

python hello_cli.py --name matz
Usage: hello_cli.py [options]
     
hello_cli.py: error: no such option: --name


  

	
   


  

If you haven't used the Python optparse module yet, you might have been surprised and wondered about all these incredible tools that python can write. If you're just beginning to touch python, you might be surprised that Python makes it all the easier. The "XKCD" website has published a very interesting comic book on the topic "Python is so Simple", which is included in the resources.
Creating useful command-line tools

Now that we've made the groundwork, we can continue to create tools to solve specific problems. For this example, we will use Python's network library named Scapy and interactive tools. Scapy can work on most *nix systems, send packets on layers 2nd and 3rd, and allow you to create very complex tools with only a few lines of Python code. If you want to start from scratch, make sure you have the necessary software installed correctly.

Let's first define the new problem to be solved.

Problem: I want to create a command-line tool that uses an IP address or subnet as a parameter, and return the MAC address or MAC address list and their respective IP addresses to standard output.

Now that we have a clear definition of the problem, let me try to break it down into the simplest possible part, and then resolve the parts individually. I have seen two separate parts for this issue. The first part is writing a function that receives an IP address or subnet range and returns a list of MAC addresses or MAC addresses. We can then consider integrating it into the command-line tool after solving the problem.
Solution Part 1th: Create a Python function that determines the MAC address by IP address

arping

from scapy import srp,Ether,ARP,conf

conf.verb=0
ans,unans=srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst="10.0.1.1"),
timeout=2)

for snd, rcv in ans:
print rcv.sprintf(r"%Ether.src% %ARP.psrc%")

The output of this command is:

    sudo python arping.py    00:00:00:00:00:01 10.0.1.1

Note that using scapy to perform actions requires elevated permissions, so we must use sudo. For the purposes of this article, I also changed the actual output to include a pseudo MAC address. We have confirmed that we can find the MAC address by IP address. We need to defragment this code to accept the IP address or subnet and return the MAC address and IP address pair.
arping function

 #!/usr/bin/env python
    
    from scapy import srp,Ether,ARP,conf
    
    def arping(iprange="10.0.1.0/24"):
     conf.verb=0
     ans,unans=srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst=iprange),
               timeout=2)
    
     collection = []
     for snd, rcv in ans:
      result = rcv.sprintf(r"%ARP.psrc% %Ether.src%").split()
      collection.append(result)
     return collection
    
    #Print results
    values = arping()
    
    for ip,mac in values:
    print ip,mac

As you can see, we have written a function that accepts an IP address or network and returns a nested list of Ip/mac addresses. We are now ready for the second part to create a command-line interface for our tools.
Solution Part 2nd: Creating command-line tools from our arping functions

In this example, we synthesize the idea in the earlier part of this article to create a complete command-line tool that solves our initial problem.
Arping CLI

 #!/usr/bin/env python
  
  import optparse
  from scapy import srp,Ether,ARP,conf
  
  def arping(iprange="10.0.1.0/24"):
   """Arping function takes IP Address or Network, returns nested mac/ip list"""
  
   conf.verb=0
   ans,unans=srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst=iprange),
        timeout=2)
  
   collection = []
   for snd, rcv in ans:
     result = rcv.sprintf(r"%ARP.psrc% %Ether.src%").split()
     collection.append(result)
   return collection
  
  def main():
   """Runs program and handles command line options"""
  
   p = optparse.OptionParser(description=' Finds MAC Address of IP address(es)',
                  prog='pyarping',
                  version='pyarping 0.1',
                  usage='%prog [10.0.1.1 or 10.0.1.0/24]')
  
  options, arguments = p.parse_args()
  if len(arguments) == 1:
   values = arping(iprange=arguments)
   for ip, mac in values:
    print ip, mac
  else:
   p.print_help()
  
  if __name__ == '__main__':
   main()

A few explanations of the above scripts will help us understand how optparse works.

First, you must create a optparse. An instance of Optionparser () and accepts an optional parameter as follows:

Copy the Code code as follows:

Description, prog, version, and usage

The meaning of these parameters is largely self-explanatory, but I would like to confirm that you should know that optparse, though powerful, is not omnipotent. It has clearly defined interfaces that can be used to quickly create command-line tools.

Second, in the following line:

Copy the Code code as follows:

Options, arguments = P.parse_args ()

The purpose of the row is to divide the options and parameters into different bits. In the above code, we expect to have exactly one parameter, so I specify that there must be only one parameter value and pass that value to the Arping function.

   If len (arguments) = = 1:      values = arping (iprange=arguments)

For further clarification, let's run the following command to understand how it works:

     sudo python arping.py 10.0.1.1          10.0.1.1 00:00:00:00:00:01

In the above example, the parameter is 10.0.1.1 Because there is only one parameter as I specified in the conditional statement, the parameter is passed to the Arping function. If there is an option, they will be passed to the options in the options, arguments = P.parse_args () method. Let's take a look at what happens when we break down the expected use case of the command-line tool and give two parameters to the use case:

  sudo python arping.py 10.0.1.1 10.0.1.3
     Usage: pyarping [10.0.1.1 or 10.0.1.0/24]
     
     Finds MAC Address or IP address(es)
     
     Options:
     --version  show program's version number and exit
     -h, --help show this help message and exit

Based on the structure of the conditional statement that I built for the parameter, if the number of arguments is not 1, it automatically opens the Help menu:

   if len(arguments) == 1:
      values = arping(iprange=arguments)
      for ip, mac in values:
       print ip, mac
     else:
      p.print_help()


  

	
   


  

This is an important way to control how a tool works, because you can use the number of parameters or the name of a particular option as a mechanism for controlling the flow of command-line tools. Because we covered the creation of the option in the original Hello World example, we then added several options to our command-line tool by slightly changing the main function:
Arping CLI main function

def main():
 """Runs program and handles command line options"""

 p = optparse.OptionParser(description='Finds MAC Address of IP address(es)',
              prog='pyarping',
              version='pyarping 0.1',
              usage='%prog [10.0.1.1 or 10.0.1.0/24]')
 p.add_option('-m', '--mac', action ='store_true', help='returns only mac address')
 p.add_option('-v', '--verbose', action ='store_true', help='returns verbose output')

 options, arguments = p.parse_args()
 if len(arguments) == 1:
  values = arping(iprange=arguments)
  if options.mac:
   for ip, mac in values:
    print mac
  elif options.verbose:
   for ip, mac in values:
    print "IP: %s MAC: %s " % (ip, mac)
  else:
   for ip, mac in values:
    print ip, mac

 else:
  p.print_help()


  

	
   


  

The primary change is to create a conditional statement based on whether an option is specified. Note that unlike the Hello World command-line tool, we only use options as true/false signals for our tools. In the case of the rumor option, if this option is specified, our conditional statement elif will only print the MAC address.

The following is the output of the new option:
Arping output

  sudo python arping2.py 
     Password:
     Usage: pyarping [10.0.1.1 or 10.0.1.0/24]
     
     Finds MAC Address of IP address(es)
     
     Options:
     --version   show program's version number and exit
     -h, --help   show this help message and exit
     -m, --mac   returns only mac address
     -v, --verbose returns verbose output
     [ngift@M-6][H:11184][J:0]> sudo python arping2.py 10.0.1.1
     10.0.1.1 00:00:00:00:00:01
     [ngift@M-6][H:11185][J:0]> sudo python arping2.py -m 10.0.1.1
     00:00:00:00:00:01
     [ngift@M-6][H:11186][J:0]> sudo python arping2.py -v 10.0.1.1
     IP: 10.0.1.1 MAC: 00:00:00:00:00:01


  

	
   


  

In-depth learning to create command-line tools

Here are a few new ideas for in-depth learning. These ideas are thoroughly explored in a book I am working with on Python *nix Systems Management, which will be published in the middle of 2008.
Using the subprocess module in command-line tools

The Subprocess module, included in Python 2.4 or later, is a unified interface for handling system calls and processes. You can easily replace the arping function above to use the Arping tool for your particular *nix operating system. The following is a rough example of this idea:
Sub-process Arping

 import subprocess
  import re
  def arping(ipaddress="10.0.1.1"):
   """Arping function takes IP Address or Network, returns nested mac/ip list"""
  
   #Assuming use of arping on Red Hat Linux
   p = subprocess.Popen("/usr/sbin/arping -c 2 %s" % ipaddress, shell=True,
              stdout=subprocess.PIPE)
   out = p.stdout.read()
   result = out.split()
   pattern = re.compile(":")
   for item in result:
    if re.search(pattern, item):
     print item
  arping()

The following is the output of the function when it is run separately: [root@localhost]~# python pyarp.py [00:16:cb:c3:b4:10]

Note that you use subprocess to get the output of the arping command and to match the MAC address with the compiled regular expression. Note that if you are using Python 2.3, you can replace subprocess with the Popen module, which is available in Python 2.4 or later.
Use Object-relational mapper in command-line tools, such as SQLAlchemy or Storm with SQLite

Another possible option for the

command-line tool is to use an ORM (object-relational mapper) to store data records generated by command-line tools. There are quite a few ORM available for Python, but SQLAlchemy and Storm happen to be the two most commonly used. I decided to use storm as an example by tossing a coin:
Storm ORM arping

   #!/usr/bin/env python
      import optparse
      from storm.locals import *
      from scapy import srp,Ether,ARP,conf
      
      class NetworkRecord(object):
       __storm_table__ = "networkrecord"
       id = Int(primary=True)
       ip = RawStr()
       mac = RawStr()
       hostname = RawStr()
      
      def arping(iprange="10.0.1.0/24"):
       """Arping function takes IP Address or Network, 
       returns nested mac/ip list"""
      
       conf.verb=0
       ans,unans=srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst=iprange),
              timeout=2)
       collection = []
       for snd, rcv in ans:
        result = rcv.sprintf(r"%ARP.psrc% %Ether.src%").split()
        collection.append(result)
       return collection
      
      def main():
       """Runs program and handles command line options"""
      
       p = optparse.OptionParser()
       p = optparse.OptionParser(description='Finds MACAddr of IP address(es)',
                    prog='pyarping',
                    version='pyarping 0.1',
                    usage= '%prog [10.0.1.1 or 10.0.1.0/24]')
      
       options, arguments = p.parse_args()
       if len(arguments) == 1:
        database = create_database("sqlite:")
        store = Store(database)
        store.execute("CREATE TABLE networkrecord "
           "(id INTEGER PRIMARY KEY, ip VARCHAR,\ 
           mac VARCHAR, hostname VARCHAR)")
        values = arping(iprange=arguments)
        machine = NetworkRecord()
        store.add(machine)
        #Creates Records
        for ip, mac in values:
         machine.mac = mac
         machine.ip = ip
        #Flushes to database
        store.flush()
        #Prints Record
        print "Record Number: %r" % machine.id
        print "MAC Address: %r" % machine.mac
        print "IP Address: %r" % machine.ip
       else:
        p.print_help()
      
      if __name__ == '__main__':
       main()


 

	
  


 

The main concern in this example is to create a class named Networkrecord, which maps to an "in-memory" SQLite database. In the main function, I change the output of the Arping function to map to our record objects, update them to the database, and then retrieve them to print the results. This is obviously not a tool that can be used for production, but can serve as illustrative examples of the steps involved in using ORM in our tools.
Integrating the config file in the CLI
Python INI config syntax

   [AIX]
     MAC: 00:00:00:00:02
     IP: 10.0.1.2
     Hostname: aix.example.com
     [HPUX]
     MAC: 00:00:00:00:03
     IP: 10.0.1.3
     Hostname: hpux.example.com
     [SOLARIS]
     MAC: 00:00:00:00:04
     IP: 10.0.1.4
     Hostname: solaris.example.com
     [REDHAT]
     MAC: 00:00:00:00:05
     IP: 10.0.1.5
     Hostname: redhat.example.com
     [UBUNTU]
     MAC: 00:00:00:00:06
     IP: 10.0.1.6
     Hostname: ubuntu.example.com
     [OSX]
     MAC: 00:00:00:00:07
     IP: 10.0.1.7
     Hostname: osx.example.com


  

	
   


  

Next, we need to use the Configparser module to parse the above content:
Configparser function

     #!/usr/bin/env python
     import ConfigParser
     
     def readConfig(file="config.ini"):
      Config = ConfigParser.ConfigParser()
      Config.read(file)
      sections = Config.sections()
      for machine in sections:
       #uncomment line below to see how this config file is parsed
       #print Config.items(machine)
       macAddr = Config.items(machine)[0][1]
       print machine, macAddr
     readConfig()

The output of this function is as follows:

OSX 00:00:00:00:07
    SOLARIS 00:00:00:00:04
    AIX 00:00:00:00:02
    REDHAT 00:00:00:00:05
    UBUNTU 00:00:00:00:06
    HPUX 00:00:00:00:03

I leave the remaining questions as exercises for the reader to solve. What I'm going to do next is to integrate the config file into my script so that I can compare the machine inventory recorded in my config file to the actual inventory of MAC addresses that appear in the ARP cache. The IP address or hostname only works when tracked to the computer, but the tools we implement can be useful for tracking the hardware address of a computer that exists on the network and determining whether it was previously present on the network.

Conclusion

We first created a very simple but powerful Hello World command-line tool by writing a few lines of code. A complex network tool was then created using the Python network library. Finally, we continue to discuss some of the more advanced areas of research readers. In the Advanced Research section, we discussed the integration of the Subprocess module and the object-relational mapper, and finally discussed the configuration file.

Although not known to everyone, anyone with an IT background can easily create command-line tools using Python. I hope this article will motivate you to create new command line tools yourself.

