Write a script in Python to extract data from the data file exported by Wireshark

The previous article builds the foundation of a UDP multicast program. The so-called Foundation is to look at it. I can write a simple multicast program and start working on it.

 

Where will the multicast content come from and what content will be broadcast? Haha, there is a device that does not have a communication protocol. It uses Wireshark to capture packets, analyze protocols, and program implementation. This is the task of this multicast.

 

Start Wireshark, capture data packets, export it as a text file, 3 MB file, ultraedit search, observation, eyes are straight, there are device communication data, and online browsing data, there are QQ data, MSN data, and some unknown data. I looked dizzy and felt that the structure of the Communication Data of the device was somewhat regular. I am a programmer. What's the problem? Can I solve it by programming? I can use python to extract data, which makes data analysis easier and can also be tested using extracted data.

 

Let's take a look at the wireshark export data format. Each data segment is "No. "At the beginning, the next line is the source IP address and the target IP address, followed by a data segment identified by" data: ", which is regular. I can use python to extract the data. If the rule is not obvious, I will be confused.

 

Functions to be completed by the python program:

L extract data from the specified IP address and save the data packet to file a. This facilitates observation of the data sent by the device and defines the relevant structure.

L then extract the data part from File A and save it as file B. I read this file B in the VC program and try to parse the data. If the parsing succeeds, my work is almost done.

 

Check the program.

 

# Coding = gb18030 </P> <p> Import fileinput <br/> # bug in this program: when exporting binary data, string Matching starts with "0000" <br/> #, which is the same as the time 0.000000 of the first package of Wireshark exported file <br/>, so first, you need to manually change 0.000000 to 0.000001. <Br/> # This program is used to export data for analysis, you don't have to go around for half a day to fix the above bug <br/> # </P> <p> # before introducing the features of this program, first understand the format of the wireshark file. <Br/> # Wireshark export file format, "no. "is the start flag. <br/> # The next line of the Start flag is the IP address and protocol information, followed by some output of its protocol analysis <br/> # The last is data. <Br/> # No. time source Destination Protocol info <br/> #16 58.5 192.168.0.66 234.5.6.7 UDP source port: 1024 Destination Port: synchronet-DB <br/> # This is the output of Wireshark analysis <br/> # data (40 bytes) <br/> #0000 E9 24 00 00 FF 01 00 02 00 00 00 D2 04 00. $ .............. <br/> #0010 14 00 00 00 00 42 03 01 00 00 00 00 05 00 00 00 .... B ........... <br/> #0020 02 00 00 00 33 00 00 00 .... 3... <br/> # </P> <p> # ----------------------------------------------------- <br/> # the function of this program is to read the export file of Wireshark, filter data of the specified IP address <br/> #1. print_data_text export from a "No. "To the next" no. "content <br/> #2. print_data_bin exports the content of "data" in the following format: <br/> # E9 24 00 00 FF 01 00 02 00 00 00 D2 04 00 00 <br/> #14 00 00 00 00 42 03 01 00 00 00 00 00 00 05 00 00 00 <br/> #02 00 00 00 33 00 00 00 <br/> # ---- <br/> # E9 24 00 00 00 FF 01 00 02 00 00 00 D2 04 00 <br/> #14 00 00 00 00 42 03 01 00 00 00 00 05 00 00 00 <br/> #02 00 00 00 00 33 00 00 00 <br/> # Insert the same "----", </P> <p> #3. print_ip_line exports only the rows containing IP addresses, that is, "No. "The following line </P> <p> # ------------------------------------------------------- <br/> # How to set conditions for data export: <br/> # conditions for data export <br/> # the content of the and field must all meet <br/> # or field content, at least one <br/> # The and field and the or field both meet <br/> # If the and field does not exist, the conditions are met. <br/> # If the or field does not exist, recognize <Br/> # No. time source Destination Protocol info <br/> #14 9.949685 192.168.0.66 234.5.6.7 UDP source port: onehome-remote destination port: synchronet-DB <br/> # example: <br/> # export data that meets the following conditions: <br/> # No. the following row of data, including "234.5.6.7" and "UDP ", <br/> # and include "192.168.0.202" and "192.168.0.22" <br/> # cond = [] <br/> # cond. append ({<br/> # "and": ["234.5.6.7", "UDP"], <br/> # "or": ["192.168.0.202 ", "192.168.0.22"], <br/> # }) <Br/> # ----------------------------------------------------- </P> <p> # export data that meets the conditions, in order to see data in the exported file from <br/> # The source file, the exported data can contain row number information, row number <br/> # Before each row of Data <br/> # filename: name of the original data file <br/> # saveto: name of the exported data storage file <br/> # cond: conditions required for data <br/> # with_ln_number: whether to export data with a row number <br/> # Only data with a row number can be used by the print_data_bin function <br/> def print_data_text (filename, saveto, Cond, with_ln_number = true ): <br/> F1 = open (SA Veto, 'W + ') </P> <p> Ln = [] <br/> myfile = fileinput. input (filename) <br/> for X in myfile: <br/> ln. append (X) <br/> start_flag = 0 <br/> start_ln = 0 <br/> end_ln = 0 <br/> end_flag = 0 <br/> find_count = 0 <br/> Print "" <br/> for I in range (0, len (LN): </P> <p> If ln [I]. find ("no. ") = 0 and start_flag = 1: <br/> start_flag = 0 <br/> If ln [I]. find ("no. ") = 0 and start_flag = 0: <br/> If check_e Xpr (Ln [I + 1], Cond) = true: <br/> start_ln = I <br/> start_flag = 1 <br/> find_count = find_count + 1 <br/> MSG = "" <br/> MSG = ("/ b/B % 02d ") % (find_count) <br/> Print MSG, </P> <p> If start_flag = 1: <br/> MSG = "" <br/> If with_ln_number = true: <br/> MSG = ("% 08d:/T % s ") % (I + 1, Ln [I]) <br/> else: <br/> MSG = ("% s") % (Ln [I]) </P> <p> f1.write (MSG) </P> <p> f1.close () </P> <p> # export the content of the data field in the data packet <br/> # Search for "0000" as the start flag <br/> # search for "data:" As the end flag <br/> def print_data_bin (filename, saveto, data_start, data_end, with_ln_number = true): <br/> F1 = open (saveto, 'W + ') <br/> Ln = [] <br/> myfile = fileinput. input (filename) <br/> for X in myfile: <br/> ln. append (X) <br/> start_flag = 0 <br/> start_ln = 0 <br/> end_ln = 0 <br/> end_flag = 0 <br/> find_count = 0 <br/> Print "" <br/> for I in range (0, len (L N): </P> <p> If start_flag = 0 and check_expr (Ln [I], data_start) = true: <br/> start_flag = 1 <br/> MSG = ("% s/n") % (Ln [I] [16: 63]) <br/> f1.write ("----/N") <br/> If start_flag = 1 and check_expr (Ln [I], data_end) = true: <br/> start_flag = 0 </P> <p> If start_flag = 1: <br/> MSG = "" <br/> If with_ln_number = true: <br/> MSG = ("% s/n") % (Ln [I] [16: 63]) <br/> else: <br/> MSG = ("% s/n") % (Ln [I] [6:53]) </P> <P> f1.write (MSG) <br/> # If I> 100: <br/> # Break <br/> f1.close () </P> <p> def check_expr (LN, expr): <br/> expr_flag = false <br/> for X in expr: <br/> and_flag = true <br/> or_flag = false <br/> If 'and' in X: <br/> for Y in X ["and"]: <br/> If ln. find (y) =-1: <br/> and_flag = false <br/> Break </P> <p> If 'or' in X: <br/> for Y in X ["or"]: <br/> If ln. find (y )! =-1: <br/> or_flag = true <br/> Break <br/> else: <br/> or_flag = true </P> <p> If and_flag = true and or_flag = true: <br/> expr_flag = true <br/> Break <br/> return expr_flag </P> <p >#< br/> # For ease of viewing the data network, only <br/> # print the row of data with an IP address <br/> # <br/> def print_ip_line (filename, saveto, Cond ): <br/> F1 = open (saveto, 'W + ') <br/> Ln = [] <br/> myfile = fileinput. input (filename) <br/> for X in myfile: <br/> ln. append (X) <br/> start_flag = 0 <br/> start_ln = 0 <br/> end_ln = 0 <br/> end_flag = 0 <br/> find_count = 0 <br/> Print "", <br/> for I in range (0, Len (LN): </P> <p> If ln [I]. find ("no. ") = 0: <br/> If check_expr (Ln [I + 1], Cond) = true: <br/> start_ln = I + 1 <br/> start_flag = 1 <br/> find_count = find_count + 1 <br/> MSG = "" <br/> MSG = ("/B % 02d ") % (find_count) <br/> Print MSG, </P> <p> If start_flag = 1: <br/> MSG = "" <br/> MSG = ("% 08d:/T % s") % (start_ln + 1, Ln [start_ln]) <br/> f1.write (MSG) <br/> start_flag = 0 </P> <p> f1.close () </P> <p> If _ name _ = '_ main _': </P> <p> src_data_file = "C: // ws.txt "<br/> txt_data_file_11 =" D: // re_11.txt "<br/> bin_data_file_11 =" D: // bin_11.txt "<br/> txt_data_file_66 =" D: // re_66.txt "<br/> bin_data_file_66 =" D: // bin_66.txt "<br/> # conditions for data export <br/> # the content of the and field must all meet <br/> # or field content, at least one <br/> # The and field and the or field both meet <br/> # If the and field does not exist, the conditions are met. <br/> # If the or field does not exist, <br/> # example: <br/> # No. time source Destination Protocol info <br/> #14 9.949685 192.168.0.202 234.5.6.7 UDP source port: onehome-remote destination port: synchronet-DB <br/> # export data that meets the following conditions: <br/> # No. the following row of data, including "234.5.6.7" and "UDP ", and contains "192.168.0.202" and "192.168.0.22" <br/> # cond = [] <br/> # cond. append ({<br/> # "and": ["234.5.6.7", "UDP"], <br/> # "or": ["192.168.0.202 ", "192.168.0.22"], <br/>#}) <br/>#</P> <p> cond = [] <br/> cond. append ({<br/> "and": ["234.5.6.7", "UDP"], <br/> "or": ["192.168.0.202", "192.168.0.22"], <br/>}) </P> <p> getdata_start = [] <br/> getdata_start.append ({<br/> "and": ["0000"], <br/>}) <br/> getdata_end = [] <br/> getdata_end.append ({<br/> "and": ["data:"], <br/>}) </P> <p> # print getdata_start <br/> # print_ip_line ('C: // re.txt ', Cond) <br/> filter_data_11 = [] <br/> filter_data_11.append ({<br/> "and": ["192.168.0.22", "234.5.6.7", "UDP"], <br/>}) <br/> filter_data_66 = [] <br/> filter_data_66.append ({<br/> "and": ["192.168.0.202", "234.5.6.7 ", "UDP"], <br/>}) </P> <p> print_data_text (src_data_file, bytes, filter_data_11) <br/> print_data_bin (bytes, bin_data_file_11, getdata_start, getdata_end) <br/> print_data_text (src_data_file, delimiter, filter_data_66) <br/> print_data_bin (delimiter, delimiter, getdata_start, getdata_end) <br/> print_data_bin (src_data_file, "D: // ttxx1.txt ", getdata_start, getdata_end, 0) <br/> print_ip_line (src_data_file," d: // ttxx2.txt ", Cond) <br/>