Log Parser: A Powerful Microsoft log analysis tool

Log parser is a log analysis tool developed by Microsoft. It is powerful and easy to use. It can analyze text-based log files, XML files, and CSV files (comma delimiter) file, as well as the operating system event log, registry, file system, active directory. It can query and analyze the data just like using SQL statements, and even display the analysis results in various charts.

The installation of log parser is very simple and there is nothing special. After installation, you can find the logparser. chm file in the installation directory. This file is the help file of logparser, which is in English language.

We will briefly introduce how to use Log parser in the command prompt.

Logparser.exe is an executable file, followed by a SQL statement parameter.

C: \ Program Files \ log parser 2.2> logparser "select top 10 * from system order by recordnumber DESC"

Select the first 10 records in reverse order by recordnumber from system logs (which can be found in Event Viewer. Recordnumber is a field in system.

We can also generate charts:

C: \ Program Files \ log parser 2.2> logparser "select eventid, count (*) as occurrences into chart.gif from system group by eventid"-charttype: pieexploded3d-groupsize: 500x350-charttitle: "event category"

The preceding Code also adds other parameters to determine the chart type and size:

Is it very simple and exciting, but these are not the focus of our introduction. Our focus is on how to use C # To execute log parser.

Com through. NET FrameworkInterOP(COM Interactive Operation) feature, you can easily use Log parser,. NET Framework COM in. Net ApplicationsInterOPIs implemented through runtime callable wrappers (RCW) for com operations, RCW is a class in. net.

We use Log parser and C # To build a website access statistical analysis system, that is, we use C # To call log parser and use Log parser to analyze IIS log files. Here, the log file is in W3C format.

Program

Create a. CS file and introduce:

Using logquery = maid. msutil. logqueryclassclass; using iisinputformat = InterOP. msutil. comiisw3cinputcontextclassclass; using logrecordset = InterOP. msutil. ilogrecordset;

The key code in the class is as follows:

Logquery ologquery = new logquery (); iisinputformat oiisinputformat = new iisinputformat (); string query = @ "select count (distinct C-IP) as hits from 'C: \ windows \ system32 \ logfiles \ w3svc1 \ ex07081_log 'where CS-Uri-stem like' %. ASP 'and SC-status = 200 "; logrecordset orecordset = ologquery. execute (query, oiisinputformat); If (! Orecordset. atend () {hits = (INT) orecordset. getrecord (). getvalue ("hits") ;}orecordset. Close ();

It can be seen that it is not much different from using the database. In the query statement, C-IP, CS-Uri-stem, SC-status, and so on are all log file fields, you can directly open the log file to find the content.

Compile

The program is complete, but it cannot be run directly. We need to compile it. Here we compile it into exe. Of course, you can also compile it into a DLL for ASP.. net.

Two steps are required. Step 1:

C: \ Program Files \ Microsoft Visual Studio 8 \ SDK \ V2.0 \ bin \ tlbimp.exe c: \ Program Files \ log parser 2.2 \ logparser. dll/out: e: \ InterOP. msutil. DLL

Download InterOP. msutil. dll directly.

Tlbimp stands for Type Library importer. It is used to create RCW, which is usually located in the bin directory of. NET Framework.

If you are using Visual Studio 2005, you can directly import it in the project without using a command prompt.

Step 2:

C: \ windows \ Microsoft. net \ framework \ v2.0.50727 \ CSC/Target: EXE/reference: E: \ InterOP. msutil. dll/out: e: \ loganal.exe E: \ loganal. CS

Loganal. CS is the program we wrote earlier. loganal.exe is the EXE file to be generated, and InterOP. msutil. dll is the DLL generated earlier.

Finished

It is so simple that there is nothing complicated. By changing the query statement, multiple types of statistics can be achieved.

 

1. Export the format logparser-I: EVT-O: CSV "select * from c: \ sec. EVT "> D: sec.csv logparser-I: EVT-O: CSV" select * from security "> D: sec.csv logparser-I: EVT-O: nat "select * into a.txt from security" logparser-I: EVT-O: CSV "select timegenerated, eventid, message from c: \ sec. EVT "> D: sec.csv logparser-I: EVT-O: TPL-TPL: eventlogs. TPL "select * into B .html from D: \ sec. EVT "Condition Statement: Select timegenerated, Eventtypename, sourcename from system where (sourcename = 'service control manager' and eventid >=7024) or (sourcename = 'w32time ') select * from security where message like '% logon % 'a. In the IIS log, search for the special link logparser-O: CSV "select * into a.csv from IIS. log where extract_extension (CS-Uri-stem) Like 'asp '"B. The most typical example is to merge the URLs in the log to collect statistics on logparser-O: CSV "select CS-Uri-stem, count (*) into a.csv from IIS. log Group By CS-Uri-stem "C. Count all logs logparser-O: CSV" select CS-Uri-stem, count (*) into a.csv from ex *. log group by CS-Uri-stem "logparser-I: iisw3c-O: CSV" select CS-Uri-stem, count (*) into a.csv from *. log group by CS-Uri-stem "D. rank the file suffix logparser-I: iisw3c-O: CSV" select extract_extension (CS-Uri-stem) as pagetype, count (*) into a.css V from *. log group by pagetype "E. obtain all the non-repeated links logparser-I: iisw3c-O: CSV" s Elect distinct CS-Uri-stem into a.csv from *. log "2. Generate percentage pie chart logparser" select eventid, count (*) as times into chart.gif from D: \ TMP \ sec. EVT group by eventid order by times DESC "-charttype: pieexploded3d-charttitle:" status codes "3. Http log logparser file: querytop. SQL-O: Chart-charttype: bar3d-charttitle: "Top 10 URL" querytop. SQL: Select top 10 CS-Uri-stem as URL, count (*) as hits into urls.gif from <1> group by URL order by hits DESC 4. Search for the keyword return the lines in an HTML document that contains links to other pages on the HTML page: logparser "select text from http://www.microsoft.adatum.com where text like '% href %'"-I: textline 5, MD5 hashes of system files logparser "select path, hashmd5_file (PATH) into a.txt from C: \ windows \ system32 \*. EXE "-I: FS-recurse: 0 6, print the 10 largest files on the C: Drive: Logparser "select top 10 path, name, size from c :\*. * order by size DESC "-I: FS 7. Obtain the logparser.exe-O: Nat" select resolve_sid (SID) as account from security where eventid in (540; 528) "8. Obtain the classification details of system logs logparser" select distinct sourcename, eventid, sourcename, message into Event _*. CSV from security "-I: EVT-O: CSV logparser" select distinct sourcename, eventid, sourcename, message into Event _*. CSV from System "-I: EVT-O: CSV logparser by ID" select distinct eventid, eventid, sourcename, message into Event _*. CSV from System "-I: EVT-O: CSV logparser" select distinct eventid, eventid, sourcename, message into Event _*. CSV from security "-I: EVT-O: CSV 9. Generate graphical interface log logparser" select 'event ID: ', eventid, system_timestamp (), message from security "-I: EVT-O: DataGrid 10. Generate a Web page logparser file: D: \ EV Entlogs. SQL? EventLog = Security-O: TPL-TPL: D: \ eventlogs. TPL logparser file: D: \ eventlogs. SQL? EventLog = system-O: TPL-TPL: D: \ eventlogs. TPL 11. view the returned code distribution pie chart logparser "select SC-status, count (*) as times into chart.gif from IIS in the IIS log. log group by SC-status order by times DESC "-charttype: pieexploded3d-charttitle:" status codes "12. logparser file: querytop ranked among the top 10 mobile phones in all logs. SQL-O: Chart-charttype: bar3d-charttitle: "Top 10 URL" querytop. SQL: Select top 10 CS-Uri-stem as URL, count (*) as hits into urls.gif from ex *. log group by URL order by hits DESC 13. Retrieve all information about all files in the directory. logparser "select * into a.csv from c: \ x-scan \*. * "-I: FS-O: CSV: Check the number of packages sent by each source IP address logparser" select srcip, count (*) into a.csv from. CAP group by srcip "-fmode: TCPIP-O: CSV view the number of packages on each source port logparser" select srcport, count (*) into a.csv from. CAP group by srcport "-fmode: TCPIP-O: CSV merges all the packages with the same srcip, dstip, and srcport, and obtains the number of logparser" select srcip, dstip, srcport, count (*) into a.csv from. CAP group by srcip, dstip, srcport "-fmode: TCPIP-O: CSV merge all tcpflags logparser" select srcip, srcport, dstip, dstport, tcpflags, count (*) into a.csv from. cap where tcpflags = 'af' group by srcip, srcport, dstip, dstport, tcpflags "-fmode: TCPIP-O: Distribution pie chart of csvtcpflags logparser" select tcpflags, count (*) into a.gif from. CAP group by tcpflags "-fmode: TCPIP-charttype: pieexploded3d-charttitle:" status codes "logparser" select tcpflags, count (*) into a.csv from. CAP group by tcpflags "-fmode: TCPIP-O: CSV

Log Parser: A Powerful Microsoft log analysis tool