Using Python and LIBNMAPD to extract Nmap scan results

Whenever I infiltrate the Intranet to face a large number of hosts and services, I am always accustomed to using automated methods to extract information from the NMAP scan results. This facilitates automated detection of different types of services, such as path blasting of Web services, testing of keys or protocols used by the SSL/TLS service, and other targeted testing.

I also often use Ipthon or *nix shells in penetration testing, which can be accessed through Python, whether it is used directly in scripts, in a REPL environment, or written to disk and accessed through a shell command.

Configuration
The first step in parsing Nmap scan results is to have an NMAP scan. I'm not going to focus too much on the details here, but you want to use the code in this article directly, you need to save the scan structure to an XML file (-ox or-oa) and perform service detection (-SV) and run related scripts (-SC) on the open port.
The commands in this document assume that you are executing in a Python REPL environment such as IPython and installing LIBNMAP modules (which can be installed using Easy_install or PIP).
Before starting, you need to set up the appropriate environment, first import the Nmapparser module and read into your XML scan results file (the instance named "Up_hosts_all_ports_fullscan.xml" is located in the current working directory)

From libnmap.parser Import nmapparsernmap_report = Nmapparser.parse_fromfile (' Up_hosts_all_ports_fullscan.xml ')

The remainder of this article contains a series of useful information extracted using a single line of code. All of the examples assume that the Nmap scan results are saved in a file as shown above. The following will give some basic sample code, if you want to run them directly in IPython, please run the above code first, so that it will be directly in the console output for your viewing. I usually do this first, so I can make sure the output is the same as expected.
You can then select a variable name and use "=" To assign the data to the variable so that you can call it directly in the subsequent code, or write it to disk for use by the shell command. If you have something you want to use multiple times, you can paste some code snippets into a Python script, or you want to add some more complex logic that might make the REPL environment difficult to handle, I'll explain in the last section how to do these things quickly.

Port information
Open host with specified port number
Displays all hosts that open a specified port number. Generates a list that contains the host address (string). Let's take port 443 as an example, you can change the value you need.

[A.address for A in nmap_report.hosts if (A.get_open_ports ()) and 443 in [b[0] for B in A.get_open_ports ()]]

Number of open ports

Displays the number of open ports for a series of hosts. Generates a list that contains the number of ports (int) and sorts them.

Sorted (Set ([B[0] for a under nmap_report.hosts for B in A.get_open_ports ()]), Key=int)

Services for Host open ports, grouped by port number

Displays the port numbers that are open for all hosts, grouped and sorted by port number. Generate a list that contains multiple lists (that is, each element of the list is also a list), where the first element of each member list is the port number (int) and the second element is a list that contains the open corresponding port host IP address (string).

[[A, [b.address for B in nmap_report.hosts for C in B.get_open_ports () if a==c[0]] "for a in sorted (set ([B[0] for a I n nmap_report.hosts for B in A.get_open_ports ()]), Key=int)]ssl/tls and Http/https

Hosts and ports that use SSL

Displays all hosts and ports that use SSL. This is done by finding out if there is a service that uses the "SSL" channel or the results of related script detection to include the PEM certificate. Generates a list containing a list of the host addresses (string) and port numbers (int) in each member list.

[[A.address, B.port] for a under nmap_report.hosts for B in a.services if b.tunnel== ' SSL ' or "' PEM '" in Str (b.scripts_result s)]

The following content contains the same information as above, but is not a list of included lists, but instead uses the Join function to create a list that contains the host: Port number (string).

[': '. Join ([A.address, str (b.port)]) for a in nmap_report.hosts for B in a.services if b.tunnel== ' SSL ' or ' PEM ' "in Str (b . Scripts_results)]

Hosts and ports that contain Web services

Displays all Web services and their corresponding port numbers and protocols (HTTP or HTTPS). This generates a list of multiple lists, each of which contains a protocol (string), an address (string), and a port number (int). But there's a problem here, nmap. When reporting sites that use HTTPS, there are times when the service is "https", and sometimes it appears as "http" using the "SSL" channel, so I've adjusted the data format for uniform output.

[[(B.service + b.tunnel). Replace (' SL ', '), A.address, B.port] for a in nmap_report.hosts for B in A.services if B.open () and B.service.startswith (' http ')]

This is still the same information, except that the URL (string) was added to the list that originally contained the Protocol, host, and port number.

[(B.service + b.tunnel). Replace (' SL ', ') + '://' + a.address + ': ' + str (b.port) + '/' for A and nmap_report.hosts for b I n a.services if B.open () and B.service.startswith (' http ')]

Other service information
Unknown service

Displays all services that NMAP does not recognize. Generates a list of multiple lists, each of which contains the address (string), the port number (int), and the Ports Thumbprint (string) of the Nmap scan. This information is generated primarily to facilitate the subsequent manual review of those specific services without participating in any automated process.

[[A.address, B.port, B.SERVICEFP] for a under nmap_report.hosts for B in a.services if (B.service = = ' unknown ' or b.service FP) and B.port in [c[0] for C in A.get_open_ports ()]]

The software identified by Nmap
Displays all software identified in the Nmap scan. Generate a list sorted by product alphabet.

Sorted (Set ([B.banner for a under nmap_report.hosts for B in a.services if ' product ' in B.banner]))

Software corresponding host and port number, grouped by product
Displays the host and port corresponding to the scanned software, grouped by product. Generates a list that contains multiple lists, where the first element of each member list is the name of the software (string), followed by another list containing the address (string) and the port number (int).

[[A, [[B.address, C.port] for B in Nmap_report.hosts-C in B.services if C.banner==a]] for a in sorted (set ([B.bann Er for a with nmap_report.hosts for B in a.services if ' product ' in B.banner])]

Ditto for the same information, except that the output is slightly different. It also generates a list with multiple lists, the first element of the member list is the name of the software (string), but the second is a list that contains the host: port number.

[[A, [': '] Join ([b.address, str (c.port)]) for B in nmap_report.hosts for C in B.services if C.banner==a]] "for a in sort Ed (Set ([B.banner for a under nmap_report.hosts for B in a.services if ' product ' in B.banner])]

Search for hosts and ports related to the specified keyword
Displays all the hosts and ports associated with the given keyword, looking for the product name, service name, and so forth from the original text of the NMAP scan results. The following is an example of "Oracle". Generates a list that contains multiple lists, each of which contains a host address (string) and a port number (int).

[[A.address, B.port] for a under nmap_report.hosts for B in A.services if B.open () and ' Oracle ' in str (b.get_dict ()) + STR (b . Scripts_results)]

The same approach, except that the stored information is modified to use lowercase to search (The following example is the lowercase "Oracle"), the output format or the same as above.

[[A.address, B.port] for a in nmap_report.hosts for B in A.services if B.open () and ' Oracle ' in (STR (b.get_dict ()) + STR ( b.scripts_results)). Lower ()]

Other things.

The same certificate name
Displays the SSL certificate found and the same portion of the certificate name that was parsed with Nmap script. This can help determine the host name of the system when you start scanning from an IP address and the reverse DNS fails. Generates a list of multiple lists, each of which contains the IP address (string) and the extracted host name (string).

[[A.address, c[' elements '] [' subject '] [' commonname ']] for a in nmap_report.hosts for B in a.services for C in B.scripts_r Esults if C.has_key (' elements ') and c[' elements '].has_key (' subject ')]

Methods for processing the above results

The above example, when you paste directly into the IPython REPL, simply prints the output to the screen. This is really good, because you can always look at the information you are interested in, but you may want to do more things. One of the great benefits of generating this information is that you can easily perform some automated actions based on the results.
If you are already familiar with Python and should be able to do it easily, then you can skip this section. But if you are unfamiliar, then this section will tell you some very basic knowledge of how to use the above code snippets.

Save to disk
If you want to save the output of the above snippet to a text file on disk, you need to convert the output list to the appropriate string format (depending on your needs) and then write the string to the file. In Python, you can use the Join function to consolidate these lists and write them to a file, which is just an example.
We want to extract the hosts and ports that support SSL from the generated list and save them in a new file so that you can use loops in bash to complete and test using command-line tools.
I usually use a line of code in IPython to do this, although a line of code will be convenient, but here for the convenience of reading and understanding, I will split the code to say.
Let's take a look at a list of "host: Ports" that was generated before, and note that we used the STR function to reload the port number from an integer type to a character type, so that it could also be stitched together with other strings using the Join function.

[': '. Join ([A.address, str (b.port)]) for a in nmap_report.hosts for B in a.services if b.tunnel== ' SSL ' or ' PEM ' "in Str (b . Scripts_results)]

Let's assign the result of the above code a variable named "ssl_services" to facilitate subsequent calls.

Ssl_services = [': '. Join ([A.address, str (b.port)]) for a in nmap_report.hosts for B in a.services if b.tunnel== ' SSL ' or ' ' Pem ' in str (b.scripts_results)]

Now, let's use the Join function to stitch up each element of the list and use (' \ n ') to wrap it, then assign it a variable named "Ssl_services_text."

Ssl_services_text = ' \ n '. Join (Ssl_services)

We can then create a new text named "Ssl_services_file.txt" under the current working directory and write the contents of the "ssl_services_text" variable.

Open (' Ssl_services_file.txt ', ' W '). Write (Ssl_services_text)

As simple as this, you can use the file content according to your own needs.

Use other Python code
Perhaps you would also like to use other Python code to do this work? As simple as this, here is another example, where we iterate through each of the Web services identified by Nmap and the results of their web page requests.
A list of URLs is generated below, and we assign a variable named "URLs" to it.

URLs = [(B.service + b.tunnel). Replace (' SL ', ') + '://' + a.address + ': ' + str (b.port) + '/' for a in nmap_report.hosts For b in A.services if B.open () and B.service.startswith (' http ')]

Next, let's do some preparatory work, import the requests module, then set up a simple getandsave function to make a Web request and save the returned result to disk, and the file name is automatically generated by URL. You may notice that in the following code, the "verify=false" option is used in the GET request, which ignores certificate validation errors when sending a request, which is often used when testing an internal machine, because the internal machine does not have an SSL certificate issued by a trusted certification authority.

Import requestsdef getandsave (URL): r = Requests.get (URL, verify=false) open (' _ '. Join (Url.split ('/') [2:]). Replace (': ') , '), ' WB '). Write (R.text.encode (' UTF8 '))

Now, let's add some code to traverse each URL, request the robots.txt file for each site, and save it locally for later use.

For a in Urls:getandsave (A + ' robots.txt ')

This will crawl each site's robots.txt file to the current working directory. This is just a very simple example.

Summarize

I hope that after reading this article, you can use Python to resolve the Nmap scan results.