Java Web Services: Comparing Metro and AXIS2 performance

The Metro Web Services stack is a reference implementation based on the JAXB 2.x data binding and JAX-WS 2.x Web service standards, but it uses additional components to provide features beyond the basic support defined by JAX-WS. Ws-security and other SOAP extension technologies are implemented by WEB Services Interoperability Technologies (WSIT) projects, and actual ws-security processing is implemented by another add-on component: XML and Web Services Security Project (XWSS).

About this series

WEB services constitute a key part of Java technology in enterprise computing applications. In this series of articles, XML and Web Services Advisor Dennis Sosnoski describes the key frameworks and technologies that are more important for Java developers who use Web services. By following this series of studies, you will learn the latest developments in this area and know how to use them to help with your programming projects.

Axis2 is based on completely different technologies, including the default Axis2 data Binding (ADB) databinding implementation, the AXIS2 engine itself, and the ws-security support, Federated Web Services Security for Java (WS s4j) Rampart module. An earlier article in this series, "The Big cost of Ws-security," describes the performance impact of ws-security processing in the Axis2 Web service stack.

"Metro profile" and "Metro service Ws-security" show you the differences between the two stacks in terms of installation, configuration, and actual use. This article looks at the difference in performance, including the difference between using ws-security.

Check performance

Similar to the "ws-security", this article uses the following methods: Measuring the time required to execute a special request sequence when the client and server are running on a single system. This approach is useful for comparing Web service processing overhead, because the impact and overhead of network latency can be excluded from time results. Assuming that the client code is not much slower than the server, this data is the best indication of the actual performance of the server under load.

This article uses the same test application as the earlier article: a seismic data retrieval service. The service uses a real database that contains records of more than 93,000 earthquakes that have occurred throughout the world over a period of time. The request for the service specifies a time range and a geographical coordinate range, and the service returns all seismic information within the specified range. See "The big cost of ws-security" for more details and a request-response message pair example.

As mentioned in the previous article, there are two sets of request sequences for performance testing. The first group uses 1,000 requests and adjusts the query parameters to find a small portion of the match in the entire seismic database (only 816 matching earthquakes are returned for these 1,000 requests). The second group was adjusted with 100 requests to find a large part of the match in the database (176,745 matching earthquakes were returned for these 100 requests). Each request sequence is run multiple times under different security configuration conditions, and only the best one at a time can be saved in the result.

The test runs on a mandriva 2009.1 64-bit Linux system equipped with Athlon X2 5400+ processors and 4 GB of RAM, using a Sun Java 1.6.0_13 32-bit JVM (for a given heap size, it is better than 64-bit JVM has better performance). Server code runs on Tomcat 6.0.20, configured to use a 1024 MB heap, while client code uses the MB heap. The version of the Web service stack is Metro 1.5 (which contains WSIT and XWSS) and Axis2 1.5.1 with the current version Rampart code (because there are no AXIS2 releases that match Rampart 1.5.x code).

Earlier articles only checked the performance of AXIS2, including plain text, SSL, and various ws-security/ws-secureconversation configurations. This article uses a more limited set of configurations, but directly compares the performance of Axis2 and Metro under each configuration.

Performance when ws-security is not in use

Figure 1 shows the test time measured by both AXIS2 and Metro when no ws-security is in use. The chart shows only slight differences between the two stacks. In tests with 1,000 instructions and fewer responses, Metro is 0.5 seconds faster than AXIS2. In tests with 100 requests and more responses, two stacks are as fast (within 0.1 seconds).

Figure 1. Test time without security settings

These time results show (the data used for the test application) Metro may be slightly faster at processing each request, but they are comparable in actual data sessions (when using the default ADB data that is bound to the AXIS2, other data bindings may have different results, especially XMLBeans binding, it's much slower).