Web site Operations Tools use the IIS Log Analysis tool to analyze IIS logs (configuration of IIS logs)

We can only analyze the health of the site through a variety of system logs, and for Web sites deployed on IIS, IIS logs provide the most valuable information that we can use to analyze the response of the site to determine whether the site has performance issues or where improvements are needed.

For a long-term maintenance of the site, how to make the site long-term stable operation is a very meaningful thing. It is also normal that some problems that are not exposed during the development phase are likely to occur in the operations phase. There are also times when we want to constantly optimize our website and make it more responsive to user requests, all of which occur in the operational phase after development.

Unlike the development phase, the operations phase is not likely to allow you to debug the program, to find all kinds of problems, we can only through a variety of system logs to analyze the health of the site, for the site deployed on IIS, IIS logs provide the most valuable information, we can use it to analyze the response of the site, To determine whether the site has performance issues, or what needs to be improved.

What information is included in the IIS logs

I said earlier that "IIS logs provide the most valuable information", what are the information? Let's look at this:

Here's the record:
1. At what point the request occurred,
2. Which client IP has access to the port of the server IP,
3. What is the client tool type, what version,
4. The URL of the request and what the query string parameter is,
5. Whether the requested method is get or post,
6. What is the processing result of the request: the HTTP status code, and the underlying status code of the operating system,
7. How much data is uploaded by the client during the request and how much data is sent by the server?
8. How long the request takes up the server, and so on.

What is the use of this information in analysis, I'll say it later. Let's make an impression on it first.

Configuration of IIS Logs

By default, IIS generates log files, but there are some parameters that deserve our attention. The Setup interface for IIS is as follows (this article takes the IIS 8 interface as an example).

In IIS Manager, select a Web site, and double-click the log icon, refer to:

At this point (main section) the interface is as follows:

In, the log is created every day to generate a new file, and the file name is generated by date (this is the default value).
Description: IIS uses UTC time, so I ticked the bottom check box to tell IIS to generate the file name with local time.

Click the "Select Field" button and the following dialog box will appear:

Note:the number of fields sent and number of bytes received is not selected by default. It is recommended to check them.
As for the other fields, you can decide whether or not to tick them as needed.

How to parse IIS logs

If you set the IIS log parameters in the way that I described earlier, IIS logs are generated after the request is processed (after a period of time).
We can quickly navigate to the root of the IIS log by tapping "View log File" in the "Actions" section on the right area of the "Log Interface", and then look for the appropriate log file in the directory (by default, the directory is differentiated according to the application pool ordinal).

For example: I found the log I needed:

This file is a whole bunch of characters, how can I analyze it now?

A tool called log Parser can specifically parse the IIS log, which we can use to view the information in the log.
For example, I can run the following command line (note: In order not to affect the width of the page I wrap the command text):

Copy CodeThe code is as follows:
"C:\Program files\log Parser 2.2\logparser.exe"-i:iisw3c-o:datagrid
"Select C-ip,cs-method,s-port,cs-uri-stem,sc-status,sc-win32-status,
Sc-bytes,cs-bytes,time-taken from U_ex130615.log "

You can now read the IIS logs in tabular form:

Description: I do not recommend this method for parsing IIS logs for two reasons:
1. Slow: When the log file is slightly larger, it is a waste of time to analyze it (especially when multiple statistics are required).
2. Inconvenient: The query syntax it supports is not rich enough to be as comprehensive as SQL Server for data table queries.

Recommended methods for parsing IIS logs

Although log Parser supports the parsing of IIS logs in tabular form for people to read, sometimes we need to do some detailed analysis, may be different ways of "multiple" query, for this need, if each query directly run log Parser, you will waste a lot of time. Fortunately, Log parser supports exporting parsing results in multiple formats (Help documentation below):

Here, I recommend selecting the output format for SQL.
Note: The SQL here does not refer to SQL Server, but to all databases that provide an ODBC access interface.
I can use the following command to import the IIS logs into SQL Server (note: In order to not affect the width of the page I wrap the command text):

Copy CodeThe code is as follows:
"C:\Program files\log Parser 2.2\logparser.exe"
"SELECT * from ' D:\Temp\u_ex130615.log ' to Mymvc_weblog"-i:iisw3c-o:sql
-oconnstring: "Driver={sql server};server=localhost\sqlexpress;database=mytestdb;integrated Security=SSPI"
-createtable:on

After the import is complete, we can use the familiar SQL Server to do various queries and statistical analysis, such as the following query:

Copy CodeThe code is as follows:
SELECT Cip,csmethod,sport,csuristem,scstatus,scwin32status,scbytes,csbytes,timetaken
FROM dbo. Mymvc_weblog

If this is the case:

Attention:
1. IIS logs characters that do not conform to the identifier specification in the field name are deleted when the results are exported to SQL Server.
For example: C-IP will become CIP and S-port will become sport.
2. The time recorded in the IIS log is UTC time, and the date and time are separated, and when exported to SQL Server, two fields are generated:

Date, time these two fields look uncomfortable, right?
I am also disgusted with this result, the following two ways to solve:

1. Add a column to SQL Server, and then change the UTC time to the local time zone, as in the T-SQL script:

Copy CodeThe code is as follows:
ALTER TABLE Mymvc_weblog add RequestTime datetime
Go
Update Mymvc_weblog Set Requesttime=dateadd (hh,8,convert (varchar), date,120
+ ' + CONVERT (varchar (+), time,114))

2. When you export the IIS log, convert the time to change the command:

Copy CodeThe code is as follows:
"C:\Program files\log Parser 2.2\logparser.exe"
"Select To_localtime (To_timestamp (ADD (to_string (date, ' Yyyy-mm-dd '), to_string (Time, ' hh:mm:ss '),
' Yyyy-mm-dd hh:mm:ss ')) as RequestTime, * from ' D:\Temp\u_ex130615.log ' to Mymvc_weblog2 '
-i:iisw3c-o:sql
-oconnstring: "Driver={sql server};server=localhost\sqlexpress;database=mytestdb;integrated Security=SSPI"
-createtable:on

Look at these three columns again:

Copy CodeThe code is as follows:
Select RequestTime, date, time from MYMVC_WEBLOG2

Once this is done, you can simply delete the date, time two columns (you can also omit them when you export the IIS logs, but explicitly specify each field name).

UTC time in the IIS log the problem is here, I hope everyone understands ~~~~~~~~~~~

Exception records in the IIS log

Each requested information is logged in the IIS log, including a normal response request and an exception request.

The "exception" described here is not related to exceptions in the. NET Framework.
For an ASP, if an uncaught exception is thrown, it is logged to the IIS log (500), but the exception I'm talking about is not limited to this.

The anomalies mentioned in this article can be divided into four parts:
1. An uncaught exception thrown by the program (ASP. NET) causes the server to produce a 500 response output.
2. There are no errors for request resources such as 404.
3. Server error greater than 500, for example: 502,503
4. System error or network transmission error.

The first three types of exceptions can be obtained using the following query:

Copy CodeThe code is as follows:
Select Scstatus, COUNT (*) as Count, sum (Timetaken * 1.0)/1000.0 as Sum_timetaken_second
From Mymvc_weblog with (NOLOCK)
GROUP BY Scstatus
ORDER BY 3 Desc

There is a column in the IIS log: Sc-win32-status, which records system-level errors, such as network transmission errors, that occur during request processing.
Normally, 0 is normal, and a non-0 value means an error occurred. We can count this kind of error like this.

Copy CodeThe code is as follows:
DECLARE @recCount bigint;
Select @recCount = count (*) from Mymvc_weblog with (NOLOCK)
Select Scwin32status, COUNT (*) as Count, (COUNT (*) * 100.0/@recCount) as [percent]
From Mymvc_weblog with (NOLOCK)
where Scwin32status > 0
GROUP BY Scwin32status
ORDER BY 2 Desc

The following table lists the more common network-related errors and explanations:

Scwin32status	Meaning
64	Client connection is down (or disconnected)
121	Transfer timeout
1236	Local network outage

All status codes can be explained by the following command:

D:\temp>net HELPMSG 64 The specified network name is no longer available.

With regard to Scwin32status and scstatus, I would also like to add that they are not related.
For example, request this address: http://www.abc.com/test.aspx
It is possible that scstatus=200, but scwin32status=64, indicates that ASP. NET has successfully processed the request, but when IIS sends the response result, the client's connection is broken.
Another scenario is: scstatus=500, but scwin32status=0, which indicates that an uncaught exception occurred during the processing of the request, but the exception result was successfully sent to the client.

Talk about scwin32status=64 again.

Remember to see scstatus=200,scwin32status=64 this situation is not understand, so search the Internet, all kinds of answers have, and some even say with the web crawler related. In order to verify the various answers, I did an experiment. I write a ashx file that uses it to simulate long-time network transmissions with the following code:

Copy CodeThe code is as follows:
public class Test_iis_time_taken:ihttphandler {

public void ProcessRequest (HttpContext context) {
Context. Response.ContentType = "Text/plain";</p>< p> System.Threading.Thread.Sleep (1000 * 2);

Context. Response.Write (String. Format ("{0}, {1}\r\n", "Start", DateTime.Now));
Context. Response.Flush ();

System.Threading.Thread.Sleep (2);</p>< p> for (int i = 0; i <; i++) {
Context. Response.Write (String. Format ("{0}, {1}\r\n", I, DateTime.Now));
Context. Response.Flush ();
System.Threading.Thread.Sleep (1000 * 1);
}</p>< p> context. Response.Write ("End");
}

Section code is very simple, I do not want to do too much explanation, just want to say: I use Thread.Sleep and Response.Flush these two methods to simulate a long duration of the continuous transmission process.

We can see this output in the browser (show me when it's not completely over)

I have done this test 8 times, only 2 times is full display completed, the remaining 6 times I close the browser window early.
We then look at the contents of the IIS logs:

Depending on the IIS logs and in combination with my own actions, you can discover:
1. When I close the browser window early, I see scstatus=200,scwin32status=64
2. If the requested content is fully displayed, I will see scstatus=200,scwin32status=0
From this experiment we can also find: Timetaken contains the network transmission time.

Based on the results of this experiment, have you ever thought of a problem:
If there is a lot of scstatus=200,scwin32status=64 in the IIS logs of your site, and the request is initiated by the user's browser.
What is the cause of this?
My "guess" is that users have been reluctant to wait while they visit the site, and they have turned off the browser window.
In other words: You can see from the statistical results of scwin32status=64 whether the response speed of the website can make users satisfied.

Looking for performance issues

The IIS log has a column called: Timetaken, which shows what it means in the IIS interface: all the time.
This time is defined as the time at which the first byte of the request received from the server starts, until all responses have been sent out.
Microsoft's website has made a description of this field: http://support.microsoft.com/kb/944884

Once we know the definition of timetaken, we can use it to analyze the processing time of some requests, that is, performance analysis.

For example, I want to see the load of the slowest 20 pages, so you can query:

Select Top Csuristem,scstatus,scwin32status,scbytes,csbytes,timetakenfrom dbo. Mymvc_weblog with (NOLOCK) where Csuristem like '/pages/% ' ORDER by timetaken Desc

Or I would like to look at the slowest 20 of the response of the Ajax situation, you can query:

Copy CodeThe code is as follows:
Select Top Csuristem,scstatus,scwin32status,scbytes,csbytes,timetaken
FROM dbo. Mymvc_weblog with (NOLOCK)
where Csuristem like '/pages/% '
ORDER BY Timetaken Desc

Or I would like to look at the slowest 20 of the response of the Ajax situation, you can query:

Copy CodeThe code is as follows:
Select Top Csuristem,scstatus,scwin32status,scbytes,csbytes,timetaken
FROM dbo. Mymvc_weblog with (NOLOCK)
where Csuristem like '/ajax/% '
ORDER BY Timetaken Desc

In summary, the way to look for performance problems is to select the Timetaken field in the query and use it to sort in descending order.

Note:scbytes,csbytes These two fields are also worthy of our attention:
1. If the csbytes is too large, we need to analyze whether the form can be split because it contains too much useless data.
Csbytes There is another possibility: the cookie is too large, but it will behave as a large number of requests csbytes, so it is easy to differentiate.
2. Scbytes if it is too large, we have to check whether the page is not paged, or you can consider the load-on-demand way to implement.
Typically, these two values become larger when ViewState is used in a large number of cases. So we can find the ViewState abuse problem through IIS logs.
Another special case is: Uploading the download file will also cause these two values to become larger, why I do not explain.

Scbytes,csbytes, regardless of which value is large, will occupy the network transmission time, for the user, it will take longer waiting time.

I said three fields at a time, which one should I refer to when looking for performance problems?
I think the timetaken should be given priority because its value directly reflects the user's wait time (excluding the front-end rendering time).
If the timetaken is too large, it is necessary to check whether the scbytes,csbytes is too large,
If the latter is too large, then the direction of optimization is to reduce the amount of data transfer, otherwise the program processing takes up a lot of time, should consider optimizing program code.

Finding goals that can be improved

In addition to discovering performance issues from the IIS logs, you can use it to find targets that can be improved.
For example:
1. Are there 404 errors?
2. Is there a large number of 304 requests?
3. Is there a large number of duplicate requests?

When we find that there are 404 responses, we should analyze the reason for generating 404:
1. Is the user entering the wrong URL address?
2. Or does the developer refer to a resource file that does not exist?
If the latter, you should remove the invalid reference as soon as possible, because the 404 response is also a page response, and they also occupy the network transmission time, especially such a request can not be cached, it will always appear, wasting network resources.

Http://www.jb51.net/os/windows/Win2003/126035.html

Web site Operations Tools use the IIS Log Analysis tool to analyze IIS logs (configuration of IIS logs)