Linux-based WebSphere performance tuning and fault diagnosis

Source: Internet
Author: User
Tags apm solr knowledge base websphere application server ibm support

I. Configuration of issues such as the was data source

(1) What is the maximum and minimum configuration for the was data source connection pool? How to calculate?

(2) about the configuration of the JVM, the 64-bit system, 64-bit was, the minimum and maximum configuration of the optimal size? How to calculate?

(3) Application server thread pool, how to configure the optimal? How to calculate?

(4) What parameters does Linux have to configure when the was is completed on Linux and is the best performance? Does the performance impact be large if not configured?


Database connection pooling recommended initialization tuning 10-100. was connection pool parameter description and tuning v1.0:http://www.webspherechina.net/document/detail/tid/109707
JVM1024-3072,JVM, please refer to the documentation of the connection pool, the official knowledge base some documents, can be used as reference.
Thread pool 100-100.
Linux Max File open number 65536, open core, user Max process number 65536
These are generally adjusted, the specific need to combine the actual situation, in general, before the system on-line, will do stress testing, including concurrent testing, fatigue testing and so on. In the test, you will constantly adjust the amount of concurrency, the duration of the pressure. This time, according to the operation of was at each test and the result of the stress test, it will be adjusted continuously until a satisfactory value is obtained. I usually open the PMI before the stress test, the level is basic, and then each was instance memory has a minimum value of 2G and a maximum value of 4G, while opening the verbose GC of was. The minimum value of the data source is set to 10 and the maximum value is set to 50. The Web container is set to 200. Other tweaks, such as Linux, can be adjusted according to the recommendations in the WebSphere Application Server performance Cookbook. As the file is quite close to 10M, you can download the pdf file directly on the IBM website: https://publib.boulder.ibm.com/httpserv/cookbook/
After each stress test, according to the records of the verbose GC, PMI records the situation, and then make a targeted adjustment. If you find a problem that is not a was, you also need to optimize the business code with a developer. Linux parameter tuning:

Parameters

Parameter values

Description

Nofiles

10000

The number of files allowed to open. The default number of open File Settings (2000) is usually sufficient for most applications. If the value set for this parameter is too small, an error may occur when you open the file or make a connection. Because this value restricts the number of file descriptors that the server process can open (soft limit).

Nofiles_hard

10000

The number of files allowed to open. The default number of open File Settings (2000) is usually sufficient for most applications. If the value set for this parameter is too small, an error may occur when you open the file or make a connection. Because this value restricts the number of file descriptors (hard limits) that the server process can open.

Tcp_keepalive_interval

15

When the probe is not confirmed, the frequency of the probe is resent. The default is 75 seconds. The recommended setting is 15.

Tcp_keepalive_probes

5

How many TCP keepalive probe packets are sent before the connection is determined to fail. The default value is 9. This value, multiplied by TCP_KEEPALIVE_INTVL, determines how much time a connection is sent after KeepAlive has not responded to the recommended setting of 5.

Netdev_max_backlog

Somaxconn

3000

3000

Change the following parameters when the connection fails because the incoming connection request rate is too high:

echo >/proc/sys/net/core/netdev_max_backlog

echo >/proc/sys/net/core/somaxconn




Second, the JVM size optimization


First set an experience value, such as the minimum value of 2G, the maximum value of 4G, and open verbose GC. Then do a stress test, or run a business, and then set a reasonable JVM value after a period of time based on the GC log.
1. GC log is the best reference when setting the JVM size
2, I see a lot of production environment GC is open, if the pursuit of extreme performance must close the good
3, session replication whether open and business requirements related, performance is to ensure that the business is normal under the premise of further consideration of the issue

When the new system is deployed online, how is it reasonable to evaluate the size of the memory set?
This is a lot of considerations, for example, the number of concurrent users, your application is not eating large memory type? For example, to generate a large report, you would return a large database result set.
It is usually possible to estimate an approximate value with a stress test and then adjust it slowly.

Third, WAS7.0 cluster a node suspended animation


WAS7.0 Virtual machine Linux cluster F5 do load balancing, often in the busy period of the business encountered a certain node suspended animation, in was console to view the node is all right, F5 because the death of suspended animation node normal distribution load, resulting in distribution to the node of the access users can not use the application system, Is there a solution to a similar problem?
This problem can be addressed in two ways:
1, F5 detection of the way to be accurate. You can write an application and then let F5 to probe and return custom code such as 200/500/501 to decide whether to distribute it or not. The other one can be done directly with F5 + IHS + ODR + was, followed by a health management strategy.
2, back-end was since the busy business performance problems, in the busy time to catch a few javacore, found the cause of suspended animation.

Is there a better way to analyze Javacore, GC, and heapdump than to deal with such problems in a daily environment, there is no good documentation for reference.
IBM provides an integrated tool for Javacore, GC, and Heapdump, called IBM Support Assistant, and the version I am currently using is 4.1.2 this version.
Analyzing Javacore, GC, and Heapdump, actually dealing with the IBM JDK, the document I read is a diagnostic document for the IBM JDK, which is very helpful for troubleshooting.
Diag60.pdf:http://www.webspherechina.net/club/attachment.php?aid= Mje5mdd8odljnzy4yjn8mtq3nde4ntgxmhw5ztgwtw5iugruwld0adbhce83mzc4t3dlmxhdagrxnjhqdk1ibkvuyzbsqzdhsq%3d%3d

Ibmjava626.pdf:http://www.webspherechina.net/club/attachment.php?aid= Mje5mdl8zjviodi2ndv8mtq3nde4ntgxmhxkmdvhoupoujnlbk5iymd5zg9vetlcsdc3bk5kuu1da29cre5ycufps0lduwxnzw%3d%3d

Four, was cluster, realize session sharing problem


The WebSphere application Server nd version defaults to 2 session sharing methods, one based on the database JDBC type and one based on memory replication. In addition to the two built-in session sharing implementations, there is a websphere EXtreme scale implementation, but this will require additional purchases in the early WebSphere version.
For me, I use a lot of memory to replicate in less nodes. Memory-to-memory replication specific configuration steps, distributed environment settings v1.0.docx:http://www.webspherechina.net/club/attachment.php?aid= Mje4nzl8ndliztg0mwj8mtq3nde4nte5oxxizgqzl0srcjlitm0xstjondb6y3azbjlgy1rfs2rpsmq3dtvvdki5n2dwuzfnma%3d%3d

If the volume is small, memory-to-memory replication can be used.
If the volume is large, it can be done on the WebSphere EXtreme scale.
How to monitor the number of sessions. Enable the PMI,PMI data with the number of sessions.

V. How was the was monitored in place?

Currently, the best software for monitoring WebSphere is the APM category. The most popular monitoring tools for APM categories, such as Quest Foglight, BMC Patrol, and more. Take ONEAPM as an example, the monitoring content is as follows:

Using OneAPM Java probes to monitor Java applications running on WebSphere, OneAPM provides a deep, high-visibility application performance management solution that captures row-level code or SQL statement calls to WEB applications on WebSphere. OneAPM also provides a wealth of monitoring information such as JVM parsing, threading, and sessions to visually demonstrate JVM heap memory, non-heap memory (stack memory, method area, Chang, etc.), memory footprint of objects of each period, garbage collection activity details, and number of classes.

Monitoring key features of Java applications in WebSphere

Monitor cross-application tracking for distributed SOA applications

Topology analysis of key business operations for the main business track

Analysis method call stack trajectory with profiling

Monitor your JVM's health in depth


Memory graphs for the JVM in WebSphere:

Heap Memory Display Chart

Non-heap Memory display chart

Real-time display of garbage collection activities

Load and unload class count chart


Thread diagram for the JVM in WebSphere:

Number of active Threads chart

Activity and idle threads for the thread pool


Session diagram for HTTP Server in WebSphere:

Analyze the number of HTTP sessions per application, including: Active, expired, rejected sessions


WebSphere Application Server transactions:

WebSphere transaction management, active transaction monitoring

Monitor changes in the total number of transactions created by WebSphere application Server

Monitor changes in the total transaction

Vi. ISA healthcenterclient not connected to the agent


Q: Server was8.5, was SDK1.6 (was), ISA5 latest version (default SDK1.8), local PC jdk1.8
The 1972 Agent agent Port is turned on on the was node virtual machine
2016-8-16 9:51:50 Com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean startagent
Info:health Center agent started on port 1972.
Health Center Connection 1972 Port error:
The JVM is not detected on port 1972 on 193.28.6.8

Answer: Option 1: Use the Health Center client for JDK version 1.6
Method 2, you try to watch to make it bad:
${was_home}${your_java_home}jrelibhealthcenter.properties
From
Com.ibm.java.diagnostics.healthcenter.agent.transport=iiop
To:
Com.ibm.java.diagnostics.healthcenter.agent.transport=jrmp
Related instructions:
Com.ibm.java.diagnostics.healthcenter.agent.transport (Java application only)
Sets the transport protocol used for the connection between the client and the agent, either IIOP or JRMP. What if you put health Center V1.3.0 or later with WebSphere? Application Server for z/Os? is used together, set the property to JRMP. The default value is IIOP.

Q: There are performance Tuning toolkit and this ISA what is the difference and connection; PMI mode and the current JMX way to connect diagnostic tools; PTT should be the SOAP protocol that collects PMI data, right? Is Isa supposed to collect PMI data using the JMX protocol?
A: It is easy to understand that PTT is a tool for online monitoring of was and offline analysis of was monitoring data; Isa is a collection of offline analysis tools.


Q: I'm very annoyed now, why does the health Center client not detect the JVM on port 1972, and I don't know where the problem is?
1, my server-side Health Center agent has developed 1972 ports, and I found that the port has been listening to wait for the connection
2, Health center agent This agent is was8.5 sdk1.6 self-brought.
I'm guessing if I need to upgrade the server-side agent?


Answer: Com.ibm.java.diagnostics.healthcenter.agent.transportSets the transport protocol to use for the connection between the Client and the agent, either IIOP or JRMP. If you are using the health Center version 1.3.0 or later with WebSphere? Application Server for z/OS, set this property to JRMP. The default value is IIOP.
By changing the transfer protocol from IIOP to JRMP, you can connect. The high version of the Healthcenter client connects the Healthcenter Agent with the Low SDK.

Is there a default permsize? How big is the initial value of some words?

The IBM JDK, which is based on the Sun JDK, has permsize. For example, the following based on Oracle 7u79-b14, which is based on the sun JDK change. The operating system is primarily Windows, Linux, and the Solaris platform, trying to make no difference to the normal sun jdk. For example, 64-bit JDK, I usually set:

-xx:permsize=256m-xx:maxpermsize=512m

D:ibmjavajava1.7.0_64_sr9fp1bin>java-version

Java Version "1.7.0"

Java (TM) SE Runtime Environment (build pwa6470sr9fp1-20150602_01 (SR9 FP1))

IBM J9 VM (build 2.6, JRE 1.7.0 Windows 7 amd64-64 compressed References 20150406_242981 (JIT enable

D, AOT enabled)

j9vm-r26_java726_sr9_20150406_1443_b242981

jit-tr.r11_20150401_88894

Gc-r26_java726_sr9_20150406_1443_b242981_cmprss

j9cl-20150406_242981)

Viii. WebSphere Process Server: How to manually clean up garbage instances??

Operating the WebSphere Process Server environment, part 3rd: Setting up, configuring, and maintaining the WebSphere Process Server business Process choreographer database

Http://www.ibm.com/developerworks/cn/websphere/library/techarticles/0912_grundler/0912_grundler.html

The above article has the corresponding script, you need to tailor the script according to the actual situation of the project, in order to complete the manual cleanup work.

Nine, about the WCS8.0 two-node double SOLR node construction

Q: Recently I have been doing WebSphere Commerce maintenance work, recently a set of WCS two nodes, the business system in the Web Access, refresh once access to the random back-end app, after the discovery may be the SOLR cluster may be configured with a problem, Solrcluster,appcluster2 clusters are deployed in dmgr on both nodes, and are normally started. Change Solrconfig.xml, normal page access error, WCS background no problem, change Solrconfig.xml to the default Web page can be normal, Web Access will be free to cause abnormal page, ask specific was8.5 SOLR cluster deployment

A: The cluster implementation of Websphere commerce is complex and cumbersome, and requires a variety of personalization settings based on the project requirements. You can refer to the DW community's two targeted documents: WebSphere Commerce and the build of the Search Engine dual cluster environment: http://www.ibm.com/developerworks/cn/websphere/ tutorials/1111_mazq_commmers/

Some best practices for deploying SOLR search in WebSphere Commerce V7.0: http://www.ibm.com/developerworks/cn/websphere/library/techarticles /1306_liq_solr/1306_liq_solr.html

Commerce8.0 uses the automated deployment method, in the IBM Knowlege Center, there are a lot of watching the heel Commerce7.0 or there is a big difference.

Ten, 9080 port occupancy caused by was failed to start

Environment, performance appraisal System Linux6.2,was6.1nd Single instance was shut down, but the 9080 port was slow to release, using Netstate-an |grep 9080 found that some connections have been unable to release, then the problem is if the 9080 port can not be released, When you start was, the newspapers port binding fails, the boot fails, and there is no good way to get these TCPIP's knowledge to quickly release and reduce business downtime. Symptom logging:

After the was process does not exist, using the netstat command to view Port 9080, the following results are found:

#netstat-aon|grep 9080

TCP 0 153305:: ffff:20.100.99.40:9080:: ffff:20.101.37.14:63336 fin_wait1 probe (0.66/0/0)

Answer: Refer to Tuning

Http://www.ibm.com/support/knowledgecenter/SSEQTP_6.1.0/com.ibm.websphere.nd.doc/info/ae/ae/tprf_tunelinux.html

Added in sysctl.conf file:

Net.ipv4.tcp_fin_timeout = 30

Let the Linux operating system actively release

Xi. Book of was recommended

With regard to WebSphere technical knowledge, I mainly read the following e-books downloaded from the Redbook website:

redp4580 WebSphere application Server V7 Session management.pdf

redp4855 WebSphere application Server V8.5.5 Technical overview.pdf

redp4870 WebSphere Application Server New Features in V8.5.pdf

sg246798 WebSphere ApplicationServer V6 problem determination for distributed platforms.pdf

sg247461 WebSphere Application Server V6.1 problem determination IBM Redpaper collection.pdf

sg247584 IBM WebSphere Application Server V6.1 on the Solaris Operating system.pdf

sg247615 WebSphere application Server V7 Administration and Configuration Guide.pdf

sg247708 WebSphere Application Server V7.0 Concepts, planning, and Design.pdf

sg248022 WebSphere Application Server V8.5 Concepts, planning, and Design Guide.pdf

sg248043 Enterprise Caching Solutions using IBM WebSphere DataPower SOA appliances and IBM WebSphere EXtreme scale.pdf

sg248048 WebSphere application Server V8.5 Migration Guide.pdf

sg248056 WebSphere application Server V8.5 Administration and Configuration Guide for the full profile.pdf

sg248076 IBM WebSphere Application Server Liberty profile Guide for Developers.pdf

sg248076 WebSphere Application Server Liberty profile Guide for Developers.pdf

sg248079 POWER7 and power7+ optimization and Tuning guide.pdf

sg248194 Configuring and Deploying Open Source with IBM WebSphere application Server Liberty profile.pdf

sg248194 Configuring and Deploying Open Source with WebSphere application Server Liberty profile.pdf

tips1024 from development to Production with the IBM WebSphere application Server Liberty profile.pdf

Linux-based WebSphere performance tuning and fault diagnosis

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.