High CPU troubleshooting

Http://www.blogjava.net/hankchen/archive/2012/05/09/377735.html

An application consumes a high CPU, and in addition to being a compute-intensive application, it is often the result of a dead loop.

(Friendly tip: This blog article welcome reprint, but please specify the source: Hankchen,http://www.blogjava.net/hankchen)

Taking one of our recent practical failures as an example, this paper describes how to locate and solve such problems.

According to the top command, it is found that the Java process with PID 28555 consumes up to 200% CPU and fails.

Via PS aux | grep pid command, you can further determine that there is a problem with the Tomcat process. However, how to navigate to specific threads or code.

The list of threads is displayed first:

PS-MP Pid-o Thread,tid,time

Found the most time-consuming thread 28802, which takes up two hours of CPU time.

Next, convert the required thread ID to 16 binary format:

printf "%x\n" tid

last print thread's stack information:

Jstack PID |grep tid-a 30

Find the code that is having the problem.

Now to analyze the specific code: shortsocketio.readbytes (shortsocketio.java:106)

Shortsocketio is an application-encapsulated tool class with a short-connection socket communication. The code for the Readbytes function is as follows:

Public byte[] readbytes (int length) throws IOException {

if ((This.socket = = null) | | (!this.socket.isconnected ())) {

throw new IOException ("++++ attempting to the read from closed socket");

}

byte[] result = NULL;

Bytearrayoutputstream BOS = new Bytearrayoutputstream ();

if (this.recindex >= length) {

Bos.write (this.recbuf, 0, length);

byte[] Newbuf = new Byte[this.recbufsize];

if (This.recindex > Length) {

System.arraycopy (this.recbuf, length, newbuf, 0, this.recindex-length);

}

This.recbuf = Newbuf;

This.recindex-= length;

} else {

int totalread = length;

if (This.recindex > 0) {

Totalread-= This.recindex;

Bos.write (this.recbuf, 0, This.recindex);

This.recbuf = new Byte[this.recbufsize];

This.recindex = 0;

}

int readcount = 0;

while (Totalread > 0) {

if ((Readcount = This.in.read (this.recbuf)) > 0) {

if (Totalread > Readcount) {

Bos.write (this.recbuf, 0, Readcount);

This.recbuf = new Byte[this.recbufsize];

This.recindex = 0;

} else {

Bos.write (this.recbuf, 0, Totalread);

byte[] Newbuf = new Byte[this.recbufsize];

System.arraycopy (This.recbuf, Totalread, Newbuf, 0, Readcount-totalread);

This.recbuf = Newbuf;

This.recindex = (readcount-totalread);

}

Totalread-= Readcount;

}

}

}

The problem is in the Red Code section. If the data returned by This.in.read () is less than or equal to 0 o'clock, the loop continues. This situation can occur when the network is congested.

As for how to modify it, it depends on how the business logic should treat this particular situation.

Finally, summarize the methods and techniques for troubleshooting CPU failures:

1. Top command: Linux command. You can view real-time CPU usage. You can also view CPU usage for the most recent period of time.

2, PS command: Linux command. Powerful process status monitoring commands. You can view the current CPU usage of the process and threads in the process. The sampled data that belongs to the current state.

3, the Jstack:java provides the command. You can view the current thread stack run of a process. Depending on the output of this command, you can locate the current running state of all threads of a process, run code, deadlock, and so on.

4, Pstack:linux command. You can view the current thread stack run of a process.