Check tomcat memory usage by the program and view tomcat memory usage

Check tomcat memory usage by the program and view tomcat memory usage
Recently, the company's online tomcat often has no chance of downtime, and summarizes the problem locating methods for reference only:
Error message:

Maximum number of threads (200) created for connector with address null and port 9443# There is insufficient memory for the Java Runtime Environment to continue.# Cannot create GC thread. Out of system resources.

1. Check whether the current user thread and file handle count exceed the limit (1) display the current user process limit: ulimit-
Display result:

core file size          (blocks, -c) 0data seg size           (kbytes, -d) unlimitedscheduling priority             (-e) 0file size               (blocks, -f) unlimitedpending signals                 (-i) 256612max locked memory       (kbytes, -l) 64max memory size         (kbytes, -m) unlimitedopen files                      (-n) 102400pipe size            (512 bytes, -p) 8POSIX message queues     (bytes, -q) 819200real-time priority              (-r) 0stack size              (kbytes, -s) 10240cpu time               (seconds, -t) unlimitedmax user processes              (-u) 1024virtual memory          (kbytes, -v) unlimitedfile locks                      (-x) unlimited

(2) modify the environment variable files of all linux users:

vi /etc/profileulimit -u 10000ulimit -n 4096

Save and run # source/etc/profile to make it take effect
2. view the current port number process information and GC usage (1) display the port PID: lsof-I: Port
Example: lsof-I: 7074

COMMAND  PID   USER   FD   TYPE   DEVICE SIZE/OFF NODE NAMEjava    3195  ligang  34u  IPv4   37416693  0t0    TCP *:7074 (LISTEN)

(2) gc statistics: jstat-gcutil PID
Example: jstat-gcutil 3195

 S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT   12.63   0.00  52.03  78.63  99.13   4148   24.274   200   40.246   64.520

(3) number of output threads: ps-mp PID-o THREAD, tid, time | wc-l
Example: ps-mp 3195-o THREAD, tid, time | wc-l
43
3. view the process memory usage and locate the corresponding program (1) memory usage: top-p PID
Example: top 3195

top - 15:29:27 up 25 days, 20:05,  2 users,  load average: 0.01, 0.05, 0.01Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombieCpu(s):  0.0%us,  0.1%sy,  0.0%ni, 99.8%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%stMem:   8058868k total,  6821684k used,  1237184k free,   181936k buffersSwap:  2097144k total,   492300k used,  1604844k free,  1897320k cachedPID  USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                           3195 ligang    20   0 4862m 196m  10m S  0.0  2.5   7:57.48 java

(2) how to locate the specific thread or code after finding the process? First, display the thread list and sort it by the threads with high CPU usage:
Ps-mp PID-o THREAD, tid, time | sort-rn | head-10
Example: ps-mp PID-o THREAD, tid, time | sort-rn | head-10

USER     %CPU PRI SCNT WCHAN  USER SYSTEM   TID     TIMEligang    0.6   -    - -         -      -     - 00:07:58ligang    0.2  19    - futex_    -      -  3270 00:02:49ligang    0.0  19    - inet_c    -      -  3277 00:00:00ligang    0.0  19    - inet_c    -      -  3273 00:00:00ligang    0.0  19    - inet_c    -      -  3271 00:00:00ligang    0.0  19    - inet_c    -      -  3203 00:00:05ligang    0.0  19    - futex_    -      -  7644 00:00:00ligang    0.0  19    - futex_    -      -  3420 00:00:00ligang    0.0  19    - futex_    -      -  3288 00:00:06

(3) convert the required thread ID to the hexadecimal format: printf "% x \ n" TID
Example: printf "% x \ n" 3270
Cc6
(4) print the stack information of the thread: jstack PID | grep cc6-A 30
Example:
Jstack 2633 | grep e18-A 30

The result shows the code that causes the problem...