LogParser's IIS Web Analytics technology

LogParser Introduction

LogParser is a free log analysis tool from Microsoft that supports many formats of file types. There are several main categories:

· IIS file format

Øiisw3c:iis generated by the expansion format

Øiis:iis proprietary format.

The BIN format of the Øbin:iis.

Øiisodbc:iis can store logs in the database. LogParser can parse the IIS logs that are stored in the database.

Øhttperr:http.sys log format.

Øurlscan:urlscan log format.

· General text Format

Øcsv: Comma delimiter file.

ØTSV: A file that is separated by a space or a tab character.

Øxml:xml format.

ØW3C:W3C format.

ØNCSA:NCSA format.

Øtextline: Normal file file. LogParser can return content by row

Øtextword:logparser can return the content by word.

· System Information

EVT: Event log file.

FS: File directory.

REG: Registry format.

Ads:ad information.

There are other formats, such as Netmon, ETW, and so on, which are not detailed here.

LogParser principle

The LogParser architecture looks like this:

Note: The picture is quoted from Http://e-mojo.net/wp-content/uploads/2009/11/logparser_architecture.gif

The core of LogParser is the class SQL engine, which is used primarily for LogParser SQL parsing. With LogParser we can convert log files to many formats, and then we can analyze them with our own handy tool.

It is important to note that if we want to output graphics format, we must install the Office Web Components.

IIS Advanced Log Analysis

This article mainly introduces some advanced IIS log analysis functions, the basic analysis function, please refer to these blogs:

Http://hi.baidu.com/tpxc/blog/item/b6e4561060f2ecf9c2ce7931.html

Http://www.cnblogs.com/yonglun/archive/2007/02/20/652929.html

number of user visits per day

LogParser "Select To_date (To_localtime (To_timestamp (Date))) as [Date1], c-ip as CIP into iishc_ Distinctclientipperday.csv from *.log GROUP by [Date1], CIP "-i:iisw3c-o:csv-recurse:-1

We summarize by date, mainly by using group by. Because the time in the IIS log is in UTC, we need to convert it to local time using To_date (To_localtime (To_timestamp (Date)).

The recurse:-1 parameter causes all files in the LogParser subdirectory to be counted.

In this example a CSV file is generated, and we can use Excel to work with the number of user curves per day (we can, of course, generate graphs directly with LogParser). Here is an example:

number of user visits per hour

LogParser "Select To_localtime (Quantize (To_timestamp (date, time), 3600)) as Hours, COUNT (*) as Hits into iishc_ Requestsperhour.csv from *.log where To_date (To_localtime (To_timestamp (Date, Time))) =timestamp (' 10/08/2010 ', ' MM/dd/ yyyy ') GROUP by Hours ORDER by Hours "-i:iisw3c-o:csv-recurse:-1

The main usage is similar to the previous example. But we need to specify a date. The specified date is implemented in the Where To_date (To_localtime (To_timestamp (Date))) =timestamp (' 10/08/2010 ', ' mm/dd/yyyy ').

Splitting files

By default, IIS produces a log file every day. For some particularly busy websites, IIS log files can be very large. I used to have a client with 2 grams per log file. Such a large file is very slow to analyze with logparser, and it is easy to make an error.

The solution is to split the log file into a number of small files for analysis separately. The following command can divide an IIS log into 24 files per hour:

LogParser "Select REPLACE_CHR (REPLACE_CHR (to_string (To_localtime (quantize (date, time), 3600), ' Yyyy-mm-dd hh:mm:ss '), ': ', ' _ '), ' ', ' _ '), date, time, C-ip, Cs-uri-stem, Cs-uri-query, Sc-status, Sc-substatus, sc-win3 2-status, Sc-bytes, Cs-bytes, Time-taken from Ex080114.log to Ex080114_*.log "-i:iisw3c-o:w3c

How about the tips of the side of the house to understand J?

Appendix

1. LogParser

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en

2. Office 2003 Web Component

http://www.microsoft.com/downloads/en/details.aspx?FamilyId=7287252C-402E-4F72-97A5-E0FD290D4B76&displaylang=en

3. Security Update for Microsoft Office Web components

http://www.microsoft.com/downloads/en/details.aspx?familyid=95c94c9a-6aca-42fb-9679-3234f06c72f7& displaylang=en

Robert

LogParser's IIS Web Analytics technology