How to use Python to compile command line tools for UNIX-like systems

This article mainly introduces how to use Python to compile command line tools for UNIX-like systems. This article is from the IBM official website technical documentation. For more information, see Introduction

Can you write a command line tool? Maybe you can, but can you compile a really useful command line tool? This article discusses how to use Python to create a robust command line tool with built-in help menus, error handling, and option processing. For some strange reasons, many people do not know Python? The standard library has all the tools required to create * NIX command line tools with extremely powerful functions.

It can be said that Python is the best language for * NIX command line tools, because it works in the philosophical way of "batteries-supported" and emphasizes providing readable code. But as a reminder, these ideas are dangerous and your life may be messy when you discover how easy it is to create a command line tool using Python. As far as I know, I have not published a detailed article about using Python to create command line tools, so I hope you like this article.
Set

The optparse module in the Python standard library can complete most of the trivial work of creating the command line tool. Optparse is included in Python 2.3, so this module will be included in many * NIX operating systems. For some reason, the operating system you use does not contain the required modules. Fortunately, the latest version of Python has been tested and compiled into almost any * NIX operating system. The systems supported by Python include IBM? AIX? , HP-UX, Solaris, Free BSD, Red Hat Linux? Ubuntu, OS X, IRIX, and even several Nokia mobile phones.
Create a Hello World command line tool

The first step in writing excellent command line tools is to define the problems to be solved. This is critical to the success of your tool. This is equally important for solving the problem in the simplest way possible. The KISS (Keep It Simple Stupid, Keep Simple) principle is clearly adopted here. Options and other features are added only after the planned functions are implemented and tested.

First, we start by creating the Hello World command line tool. According to the above suggestions, we use the simplest possible terms to define the problem.

Problem definition: I want to create a command line tool that prints Hello World by default and provides the option to print the name of the person who cannot get through.

Based on the preceding instructions, you can provide a solution that contains a small amount of code.
Hello World command line interface (CLI)

 #!/usr/bin/env python     import optparse          def main():      p = optparse.OptionParser()      p.add_option('--person', '-p', default="world")      options, arguments = p.parse_args()      print 'Hello %s' % options.person          if __name__ == '__main__':      main()

If you run this code, the expected output is as follows:

Hello world

However, we can do more than this with a small amount of code. We can get the automatically generated Help menu:

python hello_cli.py --help       Usage: hello_cli.py [options]          Options:     -h, --help      show this help message and exit     -p PERSON, --person=PERSON

We can see from the Help menu that we can use two methods to change the output of Hello World:

python hello_cli.py -p guidoHello guido

We also implement automatic error handling:

python hello_cli.py --name matzUsage: hello_cli.py [options]     hello_cli.py: error: no such option: --name

If you have not used the optparse module of Python, you may be surprised just now and want to use all these incredible tools that can be compiled by Python. If you are new to Python, you may be surprised that Python makes everything so simple. The "XKCD" website published an interesting cartoon about the theme "Python is so simple", which has been included in references.
Create a useful command line tool

Now that we have laid the foundation, we can continue to create tools to solve specific problems. In this example, we will use the Python network Library named Scapy and interactive tools. Scapy can work normally on most * NIX systems. it can send data packets on layer 2nd and layer 3rd, and allows you to create very complex tools with only a few lines of Python code. If you want to start from scratch, make sure that you have installed the necessary software correctly.

First, we define the new problem to be solved.

Problem: I want to create a command line tool that uses IP addresses or subnets as parameters, and return the MAC address or MAC address list and their respective IP addresses to the standard output.

Now that we have clearly defined the problem, let me try to break it down into as simple as possible and then solve it one by one. For this problem, I see two independent parts. The first part is to write a function for receiving IP addresses or subnet ranges, and return the MAC address or MAC address list. After solving this problem, we can consider integrating it into the command line tool.
Solution Part 1: Create a Python function that uses an IP address to determine the MAC address

arpingfrom scapy import srp,Ether,ARP,confconf.verb=0ans,unans=srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst="10.0.1.1"),timeout=2)for snd, rcv in ans:print rcv.sprintf(r"%Ether.src% %ARP.psrc%")

The output of this command is:

    sudo python arping.py    00:00:00:00:00:01 10.0.1.1

Note that using scapy to perform operations requires higher permissions, so we must use sudo. Considering the purpose of this article, I also changed the actual output to include a pseudo MAC address. We have confirmed that we can find the MAC address through the IP address. We need to sort out this code to accept the IP address or subnet and return the MAC address and IP address pair.
Arping function

    #!/usr/bin/env python        from scapy import srp,Ether,ARP,conf        def arping(iprange="10.0.1.0/24"):     conf.verb=0     ans,unans=srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst=iprange),               timeout=2)         collection = []     for snd, rcv in ans:      result = rcv.sprintf(r"%ARP.psrc% %Ether.src%").split()      collection.append(result)     return collection        #Print results    values = arping()        for ip,mac in values:    print ip,mac

As you can see, we have compiled a function that accepts IP addresses or networks and returns a list of nested IP/MAC addresses. We are now ready for part 2 to create a command line interface for our tool.
Solution Part 1: Create a command line tool from our arping function

In this example, we create a complete command line tool based on the previous ideas in this article.
Arping CLI

 #!/usr/bin/env python    import optparse  from scapy import srp,Ether,ARP,conf    def arping(iprange="10.0.1.0/24"):   """Arping function takes IP Address or Network, returns nested mac/ip list"""     conf.verb=0   ans,unans=srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst=iprange),        timeout=2)     collection = []   for snd, rcv in ans:     result = rcv.sprintf(r"%ARP.psrc% %Ether.src%").split()     collection.append(result)   return collection    def main():   """Runs program and handles command line options"""     p = optparse.OptionParser(description=' Finds MAC Address of IP address(es)',                  prog='pyarping',                  version='pyarping 0.1',                  usage='%prog [10.0.1.1 or 10.0.1.0/24]')    options, arguments = p.parse_args()  if len(arguments) == 1:   values = arping(iprange=arguments)   for ip, mac in values:    print ip, mac  else:   p.print_help()    if __name__ == '__main__':   main()

The above script will help us understand how optparse works.

First, you must create an optparse. OptionParser () instance and accept the following optional parameters:

The code is as follows:

Description, prog, version, and usage

The meanings of these parameters are basically self-explanatory, but I want to confirm that you should understand that although optparse is powerful, it is not omnipotent. It has clearly defined interfaces and can be used to quickly create command line tools.

Next, in the following line:

The code is as follows:

Options, arguments = p. parse_args ()

This row divides options and parameters into different bits. In the above code, we expect a parameter to exist. Therefore, I must specify only one parameter value and pass it to the arping function.

   if len(arguments) == 1:      values = arping(iprange=arguments)

For further explanation, run the following command to learn how it works:

     sudo python arping.py 10.0.1.1          10.0.1.1 00:00:00:00:00:01

In the preceding example, the parameter is 10.0.1.1. because there is only one parameter as specified in the condition statement, this parameter is passed to the arping function. If options exist, they will be passed to options in options, arguments = p. parse_args () method. Let's take a look at what will happen when we break down the expected use case of the command line tool and assign two parameters to the use case:

     sudo python arping.py 10.0.1.1 10.0.1.3     Usage: pyarping [10.0.1.1 or 10.0.1.0/24]          Finds MAC Address or IP address(es)          Options:     --version  show program's version number and exit     -h, --help show this help message and exit

Based on the structure of the condition statement built for the parameter, if the number of parameters is not 1, it will automatically open the Help menu:

     if len(arguments) == 1:      values = arping(iprange=arguments)      for ip, mac in values:       print ip, mac     else:      p.print_help()

This is an important way to control the way the tool works, because you can use the number of parameters or the name of a specific option as a mechanism to control the process of the command line tool. Because the option creation is involved in the initial Hello World example, we will add several options to our command line tool by slightly changing the main function:
Arping CLI main function

def main(): """Runs program and handles command line options""" p = optparse.OptionParser(description='Finds MAC Address of IP address(es)',              prog='pyarping',              version='pyarping 0.1',              usage='%prog [10.0.1.1 or 10.0.1.0/24]') p.add_option('-m', '--mac', action ='store_true', help='returns only mac address') p.add_option('-v', '--verbose', action ='store_true', help='returns verbose output') options, arguments = p.parse_args() if len(arguments) == 1:  values = arping(iprange=arguments)  if options.mac:   for ip, mac in values:    print mac  elif options.verbose:   for ip, mac in values:    print "IP: %s MAC: %s " % (ip, mac)  else:   for ip, mac in values:    print ip, mac else:  p.print_help()

The main change is to create a conditional statement based on whether an option is specified. Note that, unlike the Hello World command line tool, we only use the option as the true/false signal of our tool. If this option is specified for the-MAC option, our conditional statement elif will print only the MAC address.

The output of the new options is as follows:
Arping output

     sudo python arping2.py      Password:     Usage: pyarping [10.0.1.1 or 10.0.1.0/24]          Finds MAC Address of IP address(es)          Options:     --version   show program's version number and exit     -h, --help   show this help message and exit     -m, --mac   returns only mac address     -v, --verbose returns verbose output     [ngift@M-6][H:11184][J:0]> sudo python arping2.py 10.0.1.1     10.0.1.1 00:00:00:00:00:01     [ngift@M-6][H:11185][J:0]> sudo python arping2.py -m 10.0.1.1     00:00:00:00:00:01     [ngift@M-6][H:11186][J:0]> sudo python arping2.py -v 10.0.1.1     IP: 10.0.1.1 MAC: 00:00:00:00:00:01

Learn how to create a command line tool

The following are some new ideas for deep learning. These ideas will be discussed in depth in my book with others on Python * NIX system management, which will be published in the middle of 2008.
Use the subprocess module in the command line tool

The subprocess module is included in Python 2.4 or later and is a unified interface for processing system calls and processes. You can easily replace the preceding arping function to use the arping tool for your specific * NIX operating system. The following is a rough example of the above ideas:
Sub-process arping

  import subprocess  import re  def arping(ipaddress="10.0.1.1"):   """Arping function takes IP Address or Network, returns nested mac/ip list"""     #Assuming use of arping on Red Hat Linux   p = subprocess.Popen("/usr/sbin/arping -c 2 %s" % ipaddress, shell=True,              stdout=subprocess.PIPE)   out = p.stdout.read()   result = out.split()   pattern = re.compile(":")   for item in result:    if re.search(pattern, item):     print item  arping()

The output of this function is as follows: [root @ localhost] ~ # Python pyarp. py [00: 16: CB: C3: B4: 10]

Note that subprocess is used to obtain the output of the arping command, and the compiled regular expression is used to match the MAC address. Note: If you are using Python 2.3, you can use the popen module to replace subprocess, which is provided in Python 2.4 or later.
Use an object link er in the command line tool, such as SQLAlchemy or Storm used in combination with SQLite

Another possible option of the command line tool is to use the ORM (Object Relational Mapper) to store data records generated by the command line tool. There are quite a few ORM types available for Python, but SQLAlchemy and Storm are exactly the two most commonly used. I decided to use Storm as an example by throwing a coin:
Storm ORM arping

      #!/usr/bin/env python      import optparse      from storm.locals import *      from scapy import srp,Ether,ARP,conf            class NetworkRecord(object):       __storm_table__ = "networkrecord"       id = Int(primary=True)       ip = RawStr()       mac = RawStr()       hostname = RawStr()            def arping(iprange="10.0.1.0/24"):       """Arping function takes IP Address or Network,        returns nested mac/ip list"""             conf.verb=0       ans,unans=srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst=iprange),              timeout=2)       collection = []       for snd, rcv in ans:        result = rcv.sprintf(r"%ARP.psrc% %Ether.src%").split()        collection.append(result)       return collection            def main():       """Runs program and handles command line options"""             p = optparse.OptionParser()       p = optparse.OptionParser(description='Finds MACAddr of IP address(es)',                    prog='pyarping',                    version='pyarping 0.1',                    usage= '%prog [10.0.1.1 or 10.0.1.0/24]')             options, arguments = p.parse_args()       if len(arguments) == 1:        database = create_database("sqlite:")        store = Store(database)        store.execute("CREATE TABLE networkrecord "           "(id INTEGER PRIMARY KEY, ip VARCHAR,\            mac VARCHAR, hostname VARCHAR)")        values = arping(iprange=arguments)        machine = NetworkRecord()        store.add(machine)        #Creates Records        for ip, mac in values:         machine.mac = mac         machine.ip = ip        #Flushes to database        store.flush()        #Prints Record        print "Record Number: %r" % machine.id        print "MAC Address: %r" % machine.mac        print "IP Address: %r" % machine.ip       else:        p.print_help()            if __name__ == '__main__':       main()

In this example, you need to pay attention to the creation of a class named NetworkRecord, which is mapped to the SQLite database in "memory. In the main function, I changed the output of the arping function to map to our record objects, update them to the database, and retrieve them to print the results. This is obviously not a tool that can be used for production, but can be used as an illustration of the steps related to using ORM in our tool.
Integrate the config file in CLI
Python INI config syntax

     [AIX]     MAC: 00:00:00:00:02     IP: 10.0.1.2     Hostname: aix.example.com     [HPUX]     MAC: 00:00:00:00:03     IP: 10.0.1.3     Hostname: hpux.example.com     [SOLARIS]     MAC: 00:00:00:00:04     IP: 10.0.1.4     Hostname: solaris.example.com     [REDHAT]     MAC: 00:00:00:00:05     IP: 10.0.1.5     Hostname: redhat.example.com     [UBUNTU]     MAC: 00:00:00:00:06     IP: 10.0.1.6     Hostname: ubuntu.example.com     [OSX]     MAC: 00:00:00:00:07     IP: 10.0.1.7     Hostname: osx.example.com

Next, we need to use the ConfigParser module to parse the above content:
ConfigParser function

     #!/usr/bin/env python     import ConfigParser          def readConfig(file="config.ini"):      Config = ConfigParser.ConfigParser()      Config.read(file)      sections = Config.sections()      for machine in sections:       #uncomment line below to see how this config file is parsed       #print Config.items(machine)       macAddr = Config.items(machine)[0][1]       print machine, macAddr     readConfig()

The output of this function is as follows:

    OSX 00:00:00:00:07    SOLARIS 00:00:00:00:04    AIX 00:00:00:00:02    REDHAT 00:00:00:00:05    UBUNTU 00:00:00:00:06    HPUX 00:00:00:00:03

I leave the remaining questions to the readers as exercises. What I want to do next is to integrate the config file into my script, in this way, I can compare the machine inventory recorded in my config file with the actual inventory of the MAC address that appears in the ARP cache. The IP address or host name can only be used when it is tracked to a computer, however, the tool we implemented may be useful for tracking the hardware address of a computer on the network and determining whether it previously appeared on the network.

Conclusion

We first created a very simple but powerful Hello World command line tool by writing several lines of code. Then, a complex network tool is created using the Python network Library. Finally, we will continue to discuss more advanced research areas for readers. In the advanced research section, we discuss the integration of the subprocess module and object relationship ER, and finally the configuration file.

Although not everyone knows, any readers with IT background can use Python to easily create command line tools. I hope this article will inspire you to create a new command line tool.