IIS Log-a good helper for website operation

reading the table of contents What IIS logs contain the configuration of the IIS log how to analyze the IIS logs recommended log profiling methods exception records in IIS logs talk about scwin32status=64 looking for performance problems looking for an improved target program architecture for IIS log points Analysis of the impact of the process

For a site that needs long-term maintenance, how to keep the site stable running is a very meaningful thing. It is also normal that some problems that are not exposed during the development phase are likely to appear in the operational dimension. There are also times when we want to continually optimize the site to make it more responsive to user requests, all of which occur after the development of the operational dimension.

Unlike the development phase, the operational dimension can not allow you to debug the program, find all kinds of problems, we can only through a variety of system logs to analyze the health of the site, for the site deployed on IIS, the IIS log provides the most valuable information, we can analyze the response of the site, To determine whether there is a performance problem with the site, or where improvements are needed. Back to the top IIS log contains what information

I said earlier, "The IIS log provides the most valuable information," and what is this information? Take a look at this screenshot:

Here's a record:
1. At what point does the request take place,
2. Which client IP accesses the port of the server IP,
3. What type of client tools, what version,
4. What is the URL of the request and what is the query string parameter?
5. The way to request is get or post,
6. What is the result of the request processing: The HTTP status code, and the status code at the bottom of the operating system,
7. In the request process, the client uploads how much data, the server sends the data,
8. How long does the request occupy the server, and so on.

What is the use of this information in the analysis, I say later. It's OK to have an impression on it first. Back to the top IIS log configuration

By default, IIS produces log files, but some of the parameters are worth our attention. The Setup interface for IIS is as follows (this article takes the interface of IIS 8 for example).

In IIS Manager, select a Web site, double-click the log icon, and refer to the following figure:

At this point (the main part) the interface is as follows:

In the screenshot, the way the log is created is to generate a new file every day, generating the file name by date (this is the default).
Description: IIS uses UTC time, so I checked the bottom check box to tell IIS to use local time to generate the filename.

The following dialog box appears when you click the "Select Field" button:

Note:the "Number of fields sent" and "bytes Received" are not selected by default. It is recommended that they be checked.
As for the other fields, you can decide whether you want to check them as needed. Back to top how to analyze IIS logs

If you set the IIS logging parameters in the way I described earlier, IIS logs will be generated after the request is processed (after a period of time).
We can click "View Log File" in the right-hand area of the log interface to quickly navigate to the root of the IIS log, and then look for the corresponding log file in the directory (by default, the directory is differentiated according to the application pool ordinal).

For example: I found the log I needed:

This file is a lot of dense characters, now how do I analyze it.

A tool called log Parser can specifically parse the IIS log, which we can use to view the information in the log.
For example, I can run the following command line (note: In order not to affect the page width I wrap the command text):

"C:\Program files\log Parser 2.2\logparser.exe"-i:iisw3c-o:datagrid 
"select C-ip,cs-method,s-port,cs-uri-stem, Sc-status,sc-win32-status,
Sc-bytes,cs-bytes,time-taken from U_ex130615.log "

You can now read the IIS log in tabular form:

Description: I do not recommend this method for the analysis of IIS logs for two points:
1. Slow: When the log file is slightly larger, it is a waste of time to analyze it (especially when you need more than one statistic).
2. Inconvenient: It supports query syntax that is not rich enough to be as comprehensive as SQL Server for datasheet queries. Back to the top recommended IIS Log analysis method

Although log Parser supports the parsing of the IIS log in tabular form for people to read, but sometimes we need to do some more detailed analysis, may be different ways to do "multiple" query, for this requirement, if each query runs directly log Parser, you will waste a lot of time. Fortunately, Log parser supports the export of parsing results in multiple formats (the following screenshot of the Help document):

Here, I recommend that you select the output format as SQL.
NOTE: SQL Here does not refer to server, but to all databases that provide an ODBC access interface.
I can import IIS logs into SQL Server using the following command (note: I wrap the command text in order to not affect the page width):

"C:\Program files\log Parser 2.2\logparser.exe"  
"Select  *  from  ' D:\Temp\u_ex130615.log  ' to Mymvc_weblog "-i:iisw3c-o:sql 
-oconnstring:" Driver={sql server};server=localhost\sqlexpress;database= mytestdb;integrated Security=sspi " 
-createtable:on

Once the import is complete, we can use familiar SQL Server to do a variety of query and statistical analysis, such as the following query:

SELECT Cip,csmethod,sport,csuristem,scstatus,scwin32status,scbytes,csbytes,timetaken from 
dbo. Mymvc_weblog

If the following:

Attention:
1. When the IIS log exports the results to SQL Server, characters that do not conform to the identifier specification in the field names are deleted.
For example: C-ip will become CIP, S-port will become sport.
2. The time recorded in the IIS log is UTC, and the date and time are separated and two fields are generated when exported to SQL Server:

Date, time these two fields look very uncomfortable, right.
I am also disgusted with this result, the following two solutions to say:

1. Add a column to SQL Server and change the UTC time to the time in the local time zone, the T-SQL script is as follows:

ALTER TABLE Mymvc_weblog add requesttime datetime
go
update mymvc_weblog set Requesttime=dateadd (Hh,8,convert ( varchar (+), date,120) 
            + ' + convert (varchar, time,114))

2. Directly in the export of IIS log, the time to convert to change the command at this time:

"C:\Program files\log Parser 2.2\logparser.exe"  
"select To_localtime (ADD to_timestamp (date, ' Yyyy-mm-dd '), to_string (Time, ' Hh:mm:ss ')), 
' Yyyy-mm-dd hh:mm:ss ')-as RequestTime, *  from  ' D:\Temp\u_ Ex130615.log '  to  mymvc_weblog2 ' 
-i:iisw3c-o:sql 
-oconnstring: "Driver={sql server};server= localhost\sqlexpress;database=mytestdb;integrated Security=sspi "
-createtable:on

Look at these three columns again:

Select RequestTime, date, time from MYMVC_WEBLOG2

When you do this, you can delete the date, time, two columns (you can also ignore them when you export the IIS log, but specify each field name).

UTC time issue in IIS log here, I hope everyone understands. ~~~~~~~~~~~ back to top the exception record in the IIS log

Information about each request is recorded in the IIS log, including normal response requests and requests with exceptions.

"Exceptions" Here are not related to exceptions in the. NET Framework.
For a asp.net program, if an unhandled exception is thrown, it is logged to the IIS log (500), but the exception I am talking about is not limited to this.

The exception described in this article can be divided into four parts:
1. (asp.net) A program throws an unhandled exception, causing the server to produce a 500 response output.
There are no errors in the request resources, such as 2.404.
3. Server error greater than 500, for example: 502,503
4. System error or network transmission error.

The first three class exceptions can be obtained by using the following query:

Select Scstatus, COUNT (*) as Count, sum (Timetaken * 1.0)/1000.0 as Sum_timetaken_second from
Mymvc_weblog with (nolock Group BY Scstatus ORDER by
3 desc

There is a column in the IIS log: Sc-win32-status, which records system-level errors, such as network transmission errors, that occurred during the processing of the request.
Normally, 0 indicates normal, and a non-0 value means that an error has occurred. We can count such errors in this way:

declare @recCount bigint;
Select @recCount = count (*) from Mymvc_weblog with (nolock)
Select Scwin32status, COUNT (*) as Count, (COUNT (*) * 100.0 /@recCount) as [percent] from 
Mymvc_weblog with (nolock)
where scwin32status > 0
Group by scwin32status< C5/>order by 2 desc

The following table lists the more common network-related errors and explanations:

Scwin32status	Meaning
64	Client connection closed (or disconnected)
121	Transfer timeout
1236	Local network outage

All status codes can be obtained by using the following command to obtain the corresponding explanation:

D:\temp>net helpmsg

The specified network name is no longer available.

With regard to Scwin32status and scstatus, I would also like to add that they are not related.
Like requesting this address: http://www.abc.com/test.aspx
It is possible to scstatus=200, but scwin32status=64, which means that ASP.net has successfully processed the request, but the client's connection was disconnected when the result of the response was sent by IIS.
Another scenario is: scstatus=500, but scwin32status=0, which indicates that an unhandled exception occurred during the processing of the request, but the result of the exception was successfully sent to the client. Go back to the top and talk about Scwin32status=64 .

Remember before see scstatus=200,scwin32status=64 this situation is very not understand, so search the Internet, all kinds of answers have, and some even said that the web crawler-related. In order to verify the various answers, I did an experiment. I write a ashx file that simulates a long network transfer, and the code is as follows:

public class Test_iis_time_taken:ihttphandler {public
    
    void ProcessRequest (HttpContext context) {context
        . Response.ContentType = "Text/plain";

        System.Threading.Thread.Sleep (1000 * 2);
        
        Context. Response.Write (String. Format ("{0}, {1}\r\n", "Start", DateTime.Now));
        Context. Response.Flush ();
        
        System.Threading.Thread.Sleep (1000 * 2);