Hession deserialization results in high CPU usage

Background

Today, the release of an online service, called O, after the release, relying on the O service of 2 services C and w a large number of time alarm, and the CPU consumption of the two services have soared to about 40%, usually only 10% appearance.

At this point to see O service monitoring, time did not rise, and the QPS was reduced by half. A large number of warning appear in the C and W server logs, as follows:

java.lang.ClassNotFoundException: com.我是不可描述的信息.PropertyAoDec 02, 2016 6:24:33 PM com.alibaba.com.caucho.hessian.io.SerializerFactory getDeserializerWARNING: ‘com.我是不可描述的信息.PropertyAo‘ is an unknown class in WebappClassLoader  context:  delegate: false  repositories:    /WEB-INF/classes/----------> Parent Classloader:[email protected]

Short time can not find the reason, and C service is the core service, find the QA child shoes to the O service Rollback, C and w alarm recovery, CPU occupied back to normal.

Positioning

may have seen this warning friend already know what I have done this release, in fact, in the API returned by the model added two custom types of properties, as follows:

private List<PropertyAo> properties1;private List<PropertyAo> properties2;

These two properties W will be used, the reason is that the above mentioned warning, because I first released the O service, O Service set the two properties, and the W service has not been published, so that hession in deserialization, the detection of Propertyao does not exist, So the warning is given. But does this have anything to do with the high CPU?
Having discussed it with colleagues, he mentioned that Hession would use reflection when he was deserializing, and that he had experienced high CPU usage (because the reflection code was heavily called), which reminded me Follow Com.alibaba.com.caucho.hessian.io.SerializerFactory Getdeserializer This method to see this implementation:

try {    ClassClassfalse, _loader);    deserializer = getDeserializer(cl);} catch (Exception e) {    log.warning("Hessian/Burlap: ‘""‘ is an unknown class in "":\n" + e);    log.log(Level.FINER, e.toString(), e);}

You can see that Hession is the name to get class, where the reflection is used, and when the reflection fails, the above warning will be typed. At this point you may think of the smart, even if there is no failure to use reflection Ah, continue to look down the code:

ifnull) {    ifnull)        new HashMap(8);    synchronized (_cachedTypeDeserializerMap) {        _cachedTypeDeserializerMap.put(type, deserializer);    }}

The reflection succeeds and caches it, that is, if the reflection succeeds, only one reflection is invoked and the reflection fails, and every time the reflection is performed.

Verify

First upgrade C to the latest API, then publish, then publish the O service, C behaved normally, W's CPU began to soar, execution jstack look at the scene of the accident, you can see some threads are performing reflection, stack information is as follows:

"New I/O worker #17"Daemon prio=TenTid=0x00007fb1ed33b000Nid=0x63ferunnable [0x00007fb22fcfa000] Java. Lang. Thread. State: RUNNABLE at Java. Lang. Class. forname0 (Native Method) at Java. Lang. Class. forname(Class. Java: the) atcom. Alibaba. com. Caucho. Hessian. IO. Serializerfactory. Getdeserializer(serializerfactory. Java: -)

Solve

• When the server-side model is upgraded, especially when new custom types are added, all consumers are upgraded as much as possible, but when the consuming side is too high, the scenario is too expensive and unfriendly
• Improved serializerfactory, caching of reflection failures and avoidance of repeated reflections
• Issue for Dubbo, but it's not going to work out.

Conclusion

Hession The default deserialization implementation satisfies the following 2-point condition, which results in high CPU consumption:

• New custom types are added to the server
• The call to the service interface is high, and my application is 100+

The essence of the reason or because of reflection, so the development process with the use of reflection, reflection of the information as far as possible cache, avoid frequent reflection.

This article from: Gao | Coder, the original address: http://blog.csdn.net/ghsau/article/details/53414694, reprint please specify.

Hession deserialization results in high CPU usage