Mobile data capture and Wireshark tips

This article focuses on a very handy way to crawl Android and iphone network packets, as well as introducing Wireshark's most common tips

Grasping the Package tool introduction

(1). Web Capture Kit

Chrome browser Plugin

FireBug Plug-in

HttpWatch

Fiddler

In the browser Chrome and Firefox can use the plug-in crawl network package, HttpWatch will be installed in the form of plug-ins in IE and Firefox browser, the Web request to grab the packet. And fiddler is a separate program, the principle is to open the 127.0.0.1:8888 port after booting to listen, and the computer's browser proxy set to point to its listening port 8888, so when the browser requests the Web page, Fiddler listens on Port 8888, so that fiddler gets all the requested data and the data that the Web server replies to. Fiddler can automatically set the agent of IE and non-IE browser, after opening fiddler, click IE "Tools"-"Internet Options"-"Connection" tab – click on "LAN Settings" in the Pop-up dialog you will find the "Proxy server" setting in the " Use proxy server for LAN "This check box is ticked, click on the" Advanced "button below, you can see that the proxy server has been set to 127.0.0.1:8888.

Such as:

The settings for non-IE browsers are also automatic, as can be seen in the fiddler options

Click on the Fiddler menu "Tools"-"Fiddler Options ..." You can see such as:

Click "Copy browserproxy confiuration URL" in the figure to get a path, That is, the path of the browserpac.js, the file is fiddler used to set the non-IE browser proxy, as to what the principle, not very clear, but according to the JS comments, as well as the fiddler selected links to the Tools tips tips, It can be seen that the role is to set the non-IE browser proxy

This is the intelligence of the Fiddler software, each time the Fiddler software is turned on, he will automatically proxy settings for the browser installed in the system, and after turning off fiddler, he will restore the proxy settings of those browsers.

From this point of view, I think fiddler is more useful than httpwatch. The browser plug-in can also meet most of the Web Capture requirements.

(2). Network Grab Kit

Wireshark

Sniffer

The main principle of such tools is to monitor the data flowing through the network card through the underlying driver, which contains all the data from the link layer to the topmost application layer. This capture tool captures network packets that are the most comprehensive and can capture other packets except the HTTP protocol. For network card capture, Setup is not required.

Wireshark Mobile Grab Bag

From the network above the search data to see, to crawl the mobile phone app's network package has the following several ways:

(1). Port the tcpdump to the Android platform and start the tcpdump on the command line to grab the packet. The Tcpdump program can actually be seen as a command-line version of Wireshark, porting the program to the Android platform to grab the packet directly, which is the most straightforward way to grab packets and then upload the captured packet files from the phone to the Windows system with the Wireshark open for analysis , it seems that it can't be used on an Apple phone.

(2). Using Fiddler, open the Fiddler software on the Windows system, the software will turn our computer into an agent, then set up a WiFi network on the phone, designate the agent as the computer that opens fiddler, And the port is set to fiddler listening on the 8888 port, this time using the mobile phone access to the data, the proxy, in the Fiddler can see the HTTP packet. This method I tried for half a day how can not see the packet, do not know where the problem, according to the principle, this way to support the mobile phone through the proxy access to the network. So in principle it supports Android and Apple phones.

(3). Create a WiFi hotspot on your PC in various ways, then use Wireshark to monitor the WiFi hotspot on your PC and connect to the hotspot via your phone. This way, Wireshark will get all the packets that flow through the hotspot, and this applies to all mobile phones that can be accessed wirelessly, that is, all Android and Apple phones.

So how to build a WiFi hotspot on your PC, there are several ways:

(1). Win7 computer set up, you can set the network card as a WiFi hotspot, this method I used before, can be successful, but the steps cumbersome, and not necessarily successful, the other Windows system estimates will not be.

(2). Using connectify hotspot software, this method can also be used, this software I have also used, more useful, but there are many people on the network through this software unsuccessful case.

(3). Use the portable WiFi hardware. This can be said to be the most convenient method, and relatively stable. I bought a 360 of the portable WiFi (not advertising, I am not a 360 company, but his portable wifi does not do, colleagues have to buy millet WiFi, not too stable). I recommend using this method as long as you download the driver on the official website of 360 and plug it in with your portable WiFi directly.

The following highlights, through the use of portable WiFi to build hot spots, and then using Wireshark to grab the package, other ways to build hot-spot grab the same principle, is just the way to establish hotspots, the process of grasping the package is not different:

First insert the portable WiFi:

Use your phone to connect to the WiFi above, then open "Network and Sharing Center", and click on the Internet connection that we have built using the portable WiFi:

Inside can see the link of the network card physical address, as well as IPv6 address information, this should be a portable WiFi network card, the reason to view this information, because we want to determine wireshark grab the packet when the information of the network card, A virtual network card may also be generated on a computer with more than one network card or a virtual machine.

Open Wireshark, select the menu "Capture"-"Interface" in the popup dialog box has a column is the IP bar, you can see in addition to the local network connection, as if other network links this column shows the IPV6 address, Where we can find the IP address that matches the above network connection information:

You can also click the "Details" button to the right of the entry to open the Details dialog box, and in the "802.3 (Ethernet)" tab, you can see the physical address of the NIC, which corresponds to the physical address shown in the Network Connection dialog box above:

This can determine which network card we need to crawl data, determine, directly click on the entry of the "Start" button, the use of the WiFi on the phone casually access the network, you can crawl all the package, if it is too difficult to view the network card information, you can put these several display packets of network cards are caught to try, You should be able to see that the Wi-Fi is connected to the Internet.

On your phone, view the IP address information that is assigned to the WiFi connection:

In the Wireshark packet capture result, the address and HTTP protocol filtering out the addresses 172.24.160.3 related packets get the following results:

These data are all the packets that I have captured on my Xiaomi phone, through the UC browser, on the Borg Park computer page.

There is no difference in the capture process for iphone. This method can be captured so there are packets sent from the phone, including HTTP and all packets of non-HTTP.

Personally think this way is very simple and convenient, crawl the packet comprehensive, and through the portable WiFi set hotspot does not require any cumbersome configuration, just install a driver, in fact, the portable WiFi seems to be a new network card. We just use Wireshark to capture the data on the network card established by the portable WiFi. For those of you who are engaged in app development, it's worth 19 bucks to buy a WiFi, which can be very helpful for debugging, and for studying other people's agreements.

Wireshark Common Skills

This section mainly explains some of the common techniques used in Wireshark, in fact, the most frequent use of the process, the most frequently used to filter the rules of the package.

There are two types of filters in the Wireshark:

(1). Capture filter: Tell Wireshark we only need to capture packages that meet what conditions, while packages that do not meet the criteria do not need to be captured. Because capture filters are used in Wireshark during capture, the filter conditions of the capture filter are limited to the Transport Layer protocol, which means that rules can be specified via IP and port, while the higher layer application protocol cannot be used in the capture filter.

Grammar:Protocol  Direction  Host(s)   Value  LogicalOperations  Other  expression

Example: TCP DST 10.1.1.1 or TCP DST 10.2.2.2 3128

Explanation: Protocol represents the Protocol, direction represents the direction, host specifies the IP address, value specifies the port, and the logical operation can be used to join other expressions to generate the conforming expression. For example:

tcp dst port 8888

Capture destination TCP port 8888 Package

ip src host 10.1.1.1

Capturing a package with a source address of 10.1.1.1

host 10.1.1.1

A package that captures the purpose or source address of 10.1.1.1

not icmp

Capture all packages except ICMP packets

For more capture filters please refer to the Wireshark documentation

(2). Display Filter: The filter is in the captured package to filter out the packet you want to analyze, that is, the filter is done after the capture work has been completed, the data base is the captured packets, the filter supports the protocol is Wireshark can identify all the protocols, Because the filter is in the captured package, the conditional expression in the filter can support all upper-level protocols, and its filter criteria can be filtered according to the different parts of each protocol. The following is an example of an HTTP protocol:

View the packets for all HTTP requests (including Get,post and so on, as long as the Web request is counted)

Packets for all POST requests

All requested URLs contain the string ". jpg" Package

All packets with an HTTP response status code of 200

You can see that the HTTP protocol has many fields to provide filtering, and Wireshark also supports the matches operator for regular filtering

Find all HTTP request URLs that contain the/mvc/string and request a package with an ASPX page with parameters

The field documents for the various protocols can be consulted in the Wireshark documentation, and are described in detail, including the meaning of each field part of the protocol. The HTTP filter fields that are common for apps or web development are enough.

In addition, Wireshark can be filtered according to the content value of each field in the protocol, and can also be used to filter the binary data values of the bytes in the packet, which is generally useful in the private binary protocol of the socket, which is used to filter binary data in the packet. These protocols are generally privately defined and are binary-based protocols, such as what the first few bytes mean, Wireshark must not recognize these packages, so we can filter on our own binary data

The packet that filters out the IP source or destination address is 172.16.1.126 and the UDP port is 50798 and the value of the 2nd byte of the RTP packet is 0XE0, such that the package is a frame-end package for the RTP packet.

Of course, the above example can be filtered by the rtp.marker==1 of the RTP protocol field, and the example above is to show how to filter the binary data values in the package.

Wireshark has a very powerful capture and filtering function, this section also only lists the most commonly used filter rules, in the rules of the display filter Wireshark can be filtered according to the protocol of each layer, such as the network layer (IP,ARP,ICMP and other protocols), Transport layer (TCP,UDP) such as protocol, Application layer (HTTP,RTMP,RTSP, etc.), the fields of each layer protocol can be filtered by logic and (&&,and), or (| |,or), non-(|,not) and other operators to join as composite expressions.

More detailed Wireshark filter rules can refer to Wireshark's official documentation.

Suggestions for the blog park: Upload photos of the album If you can upload a lot of good, every time to write an article, upload more than 10 20 pictures very painful