One of Netanalyzer notes. Opening words

Source: Internet
Author: User

[Creation time: 2015-08-26-22:00:12]

Netanalyzer

The first time to write technology-related blogs, deficiencies also please magnanimous and inform.

Before starting, the first brief introduction of Netanalyzer, Netanalyzer is a set of network data acquisition, message Protocol analysis, statistics, network traffic monitoring in one of the network management tool software, You can directly think that Netanalyzer is the Chinese (Simplified) version of the Wrishark, haha, a little portentous, but do not care about these details.

Now take a look at my comment in the code:

1 /*2 * Project start time: 2011-09-173 * 2.0 Release date: 2011-10-04//sail4 * 2.1 Release date: 2011-10-20//growth Tour5 * 2.1.3.10 release time: 2011-12-06//software release6 * 2.2.0.11 Release date: 2011-12-23//Three site simultaneous release7 * 2.4 Extensions8 * 2.5 Practicality9 * 2.8 UniversalityTen * 2.9 Preliminary business level One * 2.10 Create a complete software environment A * Feng Tianwen -  * - */

Note that the project is starting from September 2011, and the version is starting from 2.0, and what is this for? And listen to me slowly, a long time ago ...

1 prior to Netanalyzer technology accumulation

Just learn Java program design, because the teacher's explanation is not very thorough, a lot of things need their own to find information learning, so they found some teaching video, through the video to learn Java, when talking about socket communication, the explanation of people on the way some network transmission data packet problems, although they are network professional, But I am only a freshman student, a lot of things feel very advanced, this is the first time to contact the concept of network grab bag. In the second half of 2010 began to enter the sophomore, began to have a bit of writing skills, every day are happy to write some small toys, but never made a thing worth showing off, this state lasted for a long time. Probably around September, a friend of the class accidentally saw me writing the program, so I joined their group to participate in a competition, he was looking for someone, I did not hesitate to agree. On the eve of the national Day, the college teacher gave a few topics, there are fractal graphic design, there is the internet of things, there are peer-to exchange technology, and of course, network protocol programming, because considering our own professional, so chose the network protocol programming, and to each team assigned a tutor, in the conversation with the teacher learned , we are temporarily going to do a simple network sniffer based on WINPACP.

1.1 The most primitive data acquisition program

Remember that I was very excited, because at that time sounded, if the production of such a software does have a great sense of accomplishment. However, the beginning of the difficult, although already sophomore, but has not opened about the network of specialized courses, so all about the concept of the network is zero, so carry forward the old tradition, in the Internet to find relevant information on their own study, the national Day pushed off all travel plans, early in the morning to start learning from the network, because the knowledge touched too wide , a lot of indefinitely, but I still insist, when the holiday is over, have a little clue, and learn to use some network protocol analysis tools such as: Wireshark,sniffer, and combined with the knowledge already learned, can do some simple protocol analysis.

Looks like everything can start, so began to try to see the development of WINPCAP documents, remember when I saw the feeling of a mess, the sample code is written in C, no matter the function name, or variable name are very awkward, but still can see clearly. First of all naturally from the start to obtain a network card, that is relatively short sample code, according to the document above the hint, took almost two days to configure the environment, finally the code debugging through, when the network card information displayed on the black and black console, especially excited.

Figure 1.1 Network data Acquisition Console program

Then the Data Acquisition console program is debugged. But the data displayed is all raw data, only the hexadecimal way to show it, if only to show it is not much significance, I need to save the data, and need to analyze the information, and immediately rewrite the program to the data in the form of text to get the network information written in the file. Then use C # to do a string parsing tool to analyze the data, the interpretation tool by starting the process, the console program to start the collection of data, and save it in a text file, and then load the file into the interpreter, extract a field to be analyzed separately. The interpreter has extracted data from three fields in Ethernet: Target Mac, source Mac, upper layer protocol type, and so on.

Figure 1.2 The Despised text Interpretation tool

When the completion of the tool, the enthusiasm of the teacher to see the time, but completely mercilessly despised a pass, but I stubbornly believe that this gadget is still meaningful, at least let oneself in the data collection design took a big step. Although a lot of data acquisition tools have been done, this is always kept in the Data Acquisition tool folder that I wrote. This gadget's collection of data is the only program written in C, and it really works in the way that you understand it. Now, however, it was a helpless move at the time. Because of the Java, I feel that the Windows desktop program is not very good, so self-taught C #, which can also explain why I use C # as an interpreter problem. In fact, the beginning is to use C + + through MFC to make, but if you re-learn a language, the introduction may not be difficult, but if you want to achieve a handy point but need to slowly accumulate, not false to the time is impossible, and insisted that the original of the old one must not be, the whole and into a dilemma.

The development of 1.2 sniffersharp

One day, hold the mood to try to find out on the internet if there is someone to provide relevant technical support. I discovered the WinPcap. NET class library Sharppcap, which is a winpcap-provided interface that can be applied directly in C # by leveraging. NET technology for placement.

Then all the development is transferred to. NET platform, and immediately found the developer's development documents, although all are English, but spent a few days of the whole can be understood, the document provides the relevant sample code, really Vista, a new round of programming immediately began, but with the last lesson, this time has some experience, and have the help of the document, do also have the emboldened, so after more than 10 days of construction finally the first can be called data capture software Sniffersharp v0.5 version finally completed, the software based on SharpPcap3.3 version, the software has been implemented

Figure 1.3 Sniffersharp v0.5 Software

The initial goal, but also increased the resolution of the Protocol, the software set up a dedicated structure to extract the Protocol field of the packet, because at that time its own knowledge level constraints, so only extracted a few simple fields for analysis, and display it in a text box, user-friendly, this kind of design way, Had a deep impact on later software. Because of the use of sniffer pro and Wrieshark, which has a hexadecimal display window, the software also opens up a hexadecimal display window, the program provides a filter expression configuration features to facilitate users to capture specific data. The various features listed on the software later in the software become the basic configuration.

Of course, there are a lot of unreasonable design in the software, the program uses a 2 million array to cache the captured data, which leads directly to the memory allocation is unreasonable, was actually aware of this, or because of the technical level of the problem, not using dynamic arrays. The other is because the UI configuration is executed directly between the data store, and after the data is stored, the packet information is extracted, but this problem is later in addition to NetAnalyzer2.0 and all subsequent versions. In the UI display, this takes time, and during this time the software will not be able to collect data, which results in a large packet loss rate, and the data extraction using a cyclic wait mode, resulting in a high CPU consumption. Another important problem is that the ListView control in WinForm, which is used to display the collected data, is re-returned whenever the data arrives, which is one of the important reasons for the loss of packets.

In the next few days, the Sniffersharp was completed immediately. v1.0 version, with the foreshadowing of the previous version, this time does not change anything on the basis of the original, just increased traffic monitoring function, in the software, added its own development of the Chart Component, in the netanalyzer2.x version still use the component, of course, this dashboard has been completely replaced, and the entire component is also from the original Of one, developed into 11 kinds, convenient for various applications. In the Sniffersharp v1.5 for the ListView to affect the data acquisition speed problem, the program is set to capture the packet is not displayed on the interface, when the click button stops, and then start in the UI display. This method is similar to how sniffer pro is handled. However, although the method is helpful to data collection, if the data collection is too large, it will have to wait for the data to be displayed slowly in the UI. Instead of the instant processing of data, users have to spend more time waiting for data to be resolved.

Figure 1.4 Sniffersharp v1.0 software

Figure 1.5 Sniffersharp v1.5 Software

Construction of 1.3 Csniffer

With the Sniffersharp series version of the design, the entire Network data Acquisition tool infrastructure has been presented, but the accompanying overall performance is unsatisfactory. Needs to be re-architected, including data storage, UI echoing, and many more issues that need to be addressed. So he gave up the plan to continue to develop sniffersharp, instead of building a new data acquisition software system, this is csniffer. This series of software only has a version, but it is very important in the development process of data acquisition software, because this kind of design, including design idea, UI design, can find it to shadow in Netanalyzer.

                                      Figure 1.6 Csniffer software a data acquisition interface      in order to tie in with the various features of this version, the Chart component has a great update, a pointer indicator used in the software, a flow chart, Data message charts are derived from this update. Of course, these are only superficial articles, the real change is its internal architecture, the use of dynamic arrays to store data, so that the software in memory allocation or expansion has a leap forward, and the use of data acquisition, protocol analysis, message statistics and other operations independent of the way the UI, more conducive to fast data processing, Thus, the efficiency of the software is improved and the usability of the software is improved.                                                   Figure 1.6 Csniffer Software B Protocol Analysis Interface                                                     Figure 1.6 Csniffer software C Message Statistics interface    

This time also increased the use of the layer load data recovery, in fact, this is also an experiment, but when the effect is, still a little achievement. This function, after several modifications have become the basic configuration, so far. In the later Netanalyzer development, there are traces to follow.

Although the development of this new architecture, but still can not get rid of those deep-rooted problems, data acquisition efficiency has improved, but the space is still very small. And in the design, still use the original software protocol analysis structure, so the message header parsing, as in the past, the overall performance is mediocre.

Another innovative point of this design is the addition of the message type distribution statistics function. However, because of the increase of this function, the efficiency of software data acquisition is more affected, but the function is still preserved. Because there's always a way to deal with it. This has become one of the problems that Netanalyzer to solve later.

At this point before the Netanalyzer all of their own production of network data collection software is complete, in fact, including testing and other data acquisition software more than these, because in the production process or modified or deleted, only the more typical several, is also a more typical several as the founder of the Netanalyzer Development Road. In fact, all the things in the world are from scratch, and from there to the classics, "Do not accumulate kuibu, no to thousands of miles, no small stream, no to become Jianghai." "Without the support of these softwares, I'm afraid I'm just going to have to design my own netanalyzer." From the beginning to feel inscrutable, to complete a program is not a program, from the original flawed, to a framework that can form a complete system, so that now I need to write the relevant instructions to assist the management of the entire system, this is not just the development of software, but also the upgrading of their own technology, understanding of the leap. Standing in the perspective of today, feel that the previous software does have a lot of flaws, and the whole program is not designed, and then look at the complete netanalyzer feeling is perfect. Perhaps one day to look back to see the software is now, perhaps, and now look at the same as the previous program. Perhaps this will appear very pessimistic, but only see the lack of software, is the real progress, but also the development of space.

2 The beginning of the Netanalzer

After completing the Csniffer, and ready to go on to the second version, the team needs a clutch software to complete the project. So it stopped the development of the Csniffer, and entered into a new stage of development. If according to the software engineering angle, such as oneself such a mess certainly is not possible, say is a new development phase, actually is to reconstruct a solution, simply finished the interface design hurriedly began the code, and did not carry on the complete design analysis, fortunately the whole process already pro, immediately completed the basic code, And the whole software is relatively simple, so with the use of less than a week to complete the design of the software, this design uses a large number of new technologies, and designed the Netanalyzer standard UI, the need to provide a window layout location, And the use of paging to display the main interface of the software and data processing interface, basically put the content of the four pages implemented in the Csniffer in two pages, increasing the utilization of the layout.

Figure 1.7 NetAnalyzer1.3 A main interface diagram 1.7 NetAnalyzer1.3 b Data Processing interface

Compared to UI design, this time more attention to performance improvement, the first natural consideration is the problem of network data acquisition efficiency, in the previous software, this is always an insurmountable gap. However, the software design to a large extent has solved the problem, this data acquisition by the original use of the circular monitoring mode into the event trigger notification mode, so that in the absence of data packets, the overall system overhead decreases, when there is data when, then notify the corresponding data storage structure to extract data, This makes the data save more flexible and does not have much effect when other threads invoke the data. In terms of interface display, the software abandons the ListView control, which has a very high frequency, and instead buffers a very small control. netanalyzer1.x version of the use of hierarchical approach, data collection in the analysis is not in the same interface, when the data collection is completed, it is submitted to the Protocol analysis interface, but later found that the interface display data at the same time, the main interface is also synchronized display data, and later found that because the value is the application type of problem, so in the second version The method was canceled.

Figure 1.8 NetAnalzyer1.3 Data Acquisition interface

Another feature is to improve the extraction of load data, to provide a different encoding method of conversion, considering the software applied to the network layer and the transport layer, so did not spend more time to deal with the application layer protocol data extraction function. This packet analysis still uses the initial analysis structure, but the structure is adjusted to make it more consistent with the distribution in the protocol.

The first version of Netanalyzer was made at the beginning of January 2011, and then a minor change was made, until September, when the version was used, and, in terms of functionality, almost minimal, in fact, it was intended to reconstruct netanalyzer but remained in mind. Later re-design idea is because the holiday in and the teacher to do the project, found some of their own software design methods, there are a lot of problems, such as: Eager to code, do not do pre-analysis, software level fuzzy, data abstraction ability is not enough. It also makes you realize the true meaning of software development.

In mid-September began the Netanalyzer second edition of the pre-design, in summary of the previous version, based on the planning of the architecture, this time using the component approach to deal with different tasks. Of course, this time still did not write the document, just do a little bit of planning to start, which is later the whole software is more and more difficult to maintain the reason. Later in the design of the gradual separation of some components, thereby reducing the occupancy rate of resources. And some components as a standalone tool for direct calls outside the system, such as traffic monitoring tools, encoding conversion tools, Application layer protocol configuration tools, and for some components need to be called in different tools or the main system, so this is also in the form of components, to provide different tools to use, such as: Filter Expression Configuration dialog box, exception Information Send dialog box.

Figure 1.9 NetAnalyzer2.2 Main interface

Figure 1.10 NetAnalyzer2.2 Data Analysis interface

In fact, at present, the good multi-function is not in the beginning of the design of the plan, however, with the accumulation of time, with their own analysis of the various needs, so constantly increased the relevant functions, in software engineering, in the beginning of the production of software is to do demand analysis, but because it does not use such conditions, Therefore, the added function is only the process of their own use and gradually discovered, in fact, in the production version of the user to add the message function to facilitate the user to return their ideas, but later in the consideration of security issues, then give up (will say later), of course, in the latest version has solved the security problem.

The second version of the Netanalyzer has been completely out of the concept of grasping software, according to the current construction structure, the whole can be called the system environment. Because this design in addition to the original goal of network data acquisition, protocol analysis, increase traffic monitoring, background data collection, code conversion, such as a series of auxiliary tools, for traffic monitoring, although once appeared, but, in this design, flow monitoring has become an independent system, with the flow of record files to join, The tool is ready to be used on its own.

Figure 1.11 Network traffic Monitoring Tool

By documenting the status of the network, a dedicated file interpreter is required to process the relevant data, because the interpreter and the traffic monitoring tool do not do so in the same way.

Another tool that can be used independently is the Code conversion tool, which gives the tool a tool that has been set up in the first place to handle the encoding of the conversion load data after the TCP reorganization of the main system. Later, many times, the use of a standalone tool is eventually provided, while in the main system the value is initiated by providing an interface. This makes it easy to use in different situations. The tool provides Base64, URL, HTML, encoding and decoding methods, and provides ASCII, GB2312, UTF-8, Unicode and other character encoding methods, in the latest version of the MD5 message Digest feature, you can separate the string and file summary information extraction.

Figure 1.12 Coding Tools

For some shared use of the component here is just a filter expression to configure the window description, the component can not be executed independently, it needs to be loaded on other tools, it only provides data communication interface, and the method of invocation, the configuration window provides two ways, direct input method and fill way, direct Input method, Suitable for people familiar with the WinPcap filter expression format, so that can write a complex filter expression, in the interface provides a link to the expression settings rules, for those who do not understand the user can also directly learn to use. and fill in the way is suitable for entry-level personnel, through a few options, input specified parameters, you can automatically generate expressions, easy to use, but can only generate simple is an expression, but can meet the general requirements. When you are done clicking the OK button, the system begins to automatically check that the expression conforms to the specification. The user is not prompted to reset, if it is in the desired location, and the expression is made history for the next use, and the expression input box to color some keywords, through different colors to mark the different primitives, to prevent errors and convenient error checking.

Figure 1.13 Filtering An expression configuration

The introduction to Netanalyzer ends here, but it is only a prologue. And we just touch the tip of the iceberg. In the above introduction, the main focus on the whole part of the system, and then introduced a number of tool components, but also just stay on the surface only, although now people pay more and more attention to user experience, but as a professional use of software, more should pay attention to efficiency and usability. In the following explanation, we will gradually display the implementation of these functions, so that the principle of deep metanalyzer.

Summarize

"Jade does not grind, does not build", the software development is more so, in the Network Data Acquisition tool development process, from the concept to an impact, then to a primitive entity, finally to a complete software, not a day, in about 1.5 of the time, their own dedication, continuous improvement, constantly looking for solutions, It may be slow, but at least every tool that is produced in your own hands is stronger than the last, and that is progress. In fact, this is not the hustled to write this book, I really do not know that they have tried so many difficulties, if not so high enthusiasm to put in or satisfied with the status quo, perhaps there is no follow-up version, but not the creation of the book.

Netanalyzer is now a 2.9 version, and the last release was 2014 September, the function compared to have been enriched a lot, the ability is much stronger, but the update is also more and more slow, and then from June 2013 after graduating from college only released two simple version, perhaps the work is busy, maybe it is just less crazy Hot Pursuit, so that now although there are many ideas, but only to stay in the imagination, and immediately 2015 years of September also came, specially sent a series to commemorate the enthusiasm of that year.

Netanalyzer

Netanalzyer AC Group: 39753670 (PS only provide communication platform, the group master basically do not speak ^_^)

[Reprint please retain the author Information von Astronomy website: http://www.cnblogs.com/twzy/p/4761889.html]

One of Netanalyzer notes. Opening words

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.