Python implements script sharing for transferring pure IP database, pythonip

Python implements script sharing for transferring pure IP database, pythonip

Preface

I have written many articles about scanning scripts before, but I have never written my own scanning IP segments. Some of my friends often come to ask for some scanning experience, to be honest, I don't think this tool is actually technical, but it can improve work efficiency and share it with you ~

Speaking of scanning experience, I usually select different types of segments for different devices and applications.

For example, if I want to scan for a telecom optical cat, it will naturally select the Telecom IP segment. The optical cat is generally a home user. We will filter the Active IP segments of home users, in this way, we will be targeted.

For example, if I want to scan an enterprise routing device, I can select multiple segments of the enterprise.

Pure IP is really a good tool. I usually use it to retrieve IP segments in a region. However, this tool has a fatal drawback that it cannot be used together for queries, this is a headache, and every time you use this tool, you have to switch to the Windows operating system. For a Linux party like me, it is naturally intolerable, simply write a script that transfers a pure IP address to the mysql database, so that you do not have to go to win next to each query. You can also directly deploy it to a remote location, which makes query much easier, most importantly, multi-condition query is supported.

Requirement

It is not technically difficult to write this script in Python. I mainly use the MySQLdb library. I will not talk much about the installation of MySQLdb library. I have written related articles before my blog. Here we need to first analyze the data file structure of the pure IP database, and find that the structure of each row is fixed, so it is easy to write.

I will not elaborate on coding. A few simple methods can be easily implemented and the code will be pasted out.

#!/usr/bin/env python# coding=utf-8# kbdancer@92ez.comimport MySQLdbimport sysreload(sys)sys.setdefaultencoding('utf8')def save_data_to_mysql(mysql_object, ip_line): try:  begin = ip_line[0:16].replace(' ', '')  end = ip_line[16:32].replace(' ', '')  try:   location = line[32:].split(' ')[0]  except:   location = ''  try:   isp_type = line[32:].replace(' ', ' ').split(' ')[1].replace('\n', '').replace('\r', '')  except:   isp_type = ''  this_line_value = [begin + "-" + end, location, isp_type]  do_insert(mysql_object, this_line_value) except Exception, e:  print edef do_insert(mysql_object, row_data): try:  insert_sql = """INSERT INTO `ipdb` (`iprange`,`location`, `type`) VALUES ( %s, %s, %s )"""  mysql_object.insert(insert_sql, row_data) except Exception, e:  print row_data  print eclass Database: host = 'localhost' user = 'ipdb' password = '3u9whrpcEUBTnNNn' db = 'ipinfo' charset = 'utf8' def __init__(self):  self.connection = MySQLdb.connect(self.host, self.user, self.password, self.db, charset=self.charset)  self.cursor = self.connection.cursor() def insert(self, query, params):  try:   self.cursor.execute(query, params)   self.connection.commit()  except Exception, e:   print e   self.connection.rollback() def query(self, query, params):  cursor = self.connection.cursor(MySQLdb.cursors.DictCursor)  cursor.execute(query, params)  return cursor.fetchall() def __del__(self):  self.connection.close()if __name__ == '__main__': mysql = Database() ip_file = open(sys.path[0] + "/ip.txt") print 'Start save to mysql ...' for line in ip_file:  save_data_to_mysql(mysql, line) ip_file.close() print 'Save complete.'

Note:

There is a performance problem here, that is, when traversing all the data, you need to insert the database, the efficiency of single row insertion is very low, it is recommended to use multiple rows insert, such as writing a cache array, when the number of cached arrays reaches the specified number, for example, 100 entries are saved to the database at a time, this speed is much faster than that of a single entry. I have dug a hole here, and it is not difficult to change it because I hope my friends who use the script can modify it by themselves.

Because the txt files exported from the pure IP database are not standard without BOM UTF8 encoding, direct Parsing is definitely a failure. We recommend that you use Notepad ++ to transcode the files first.

Effect

Raw Data

Converted data

Use

First, export the pure real IP data library as a TXT file. Here I export it as ip.txt

And put it in the same directory as the Py script.

By the way, you must first have a mysql database

Then import the database structure, which is the SQL file.

Then you have to modify the mysql connection password in the script.

Finally, execute the Py script.

Description

All code is hosted on Github

Address https://github.com/kbdancer/myTools/tree/master/czip2mysql

Summary

The above is all the content of this article. I hope the content of this article will help you in your study or work. If you have any questions, please leave a message, thank you for your support.