Introduction to Java code optimization process

Criteria for measuring procedures

To measure whether a program is good or not, it can be analyzed from multiple angles. The most common metrics are the time complexity of the program, the complexity of the space, and the readability and scalability of the code. For the program time complexity and space complexity, want to optimize the program code, need to have a deep understanding of data structure and algorithm, and familiar with the basic concepts and principles of computer systems, and for the readability and scalability of the code, want to optimize the program code, need to understand the software architecture design, Familiar with and will apply the appropriate design pattern.

First, the computer system's storage space is already large enough to reach TB levels, so time complexity is the primary consideration for programmers compared to space complexity. In order to pursue high performance, in some frequent operation execution, even may consider uses the space to exchange the time. Secondly, due to the physical limitations of the processor manufacturing process, cost constraints, CPU frequency growth encountered a bottleneck, Moore's law has been gradually ineffective, every 18 months CPU frequency that doubled the era has passed, the programmer's programmatic approach has undergone a radical change. In the current era of multi-core multiprocessor, native-supported multi-threaded languages (such as Java) and distributed parallel computing frameworks (such as Hadoop) emerged. In order for the program to make full use of multi-core CPUs, it is not enough to simply implement a single threaded program, and programmers need to be able to write concurrent or parallel multithreaded programs. Finally, the number of code lines for large software systems has reached millions, and if there is no well-designed software architecture to develop on the basis of existing code, the cost of development and maintenance is unimaginable. A well-designed software should be readable and scalable, adhering to the "open and close Principle", "dependency inversion principle", "interface-oriented programming", and so on.

Project Introduction

This article will introduce the author's experience of a part of a project, through this example to analyze the process of code optimization. The following is a brief introduction to the relevant parts of the system.

The system's development language is Java, deployed in a total of 4 of the CPU's Linux server, the relevant part of the main operations: through an external system D provided by the REST API to obtain information, extract effective information, and through JDBC storage to a database system S, for the system In other parts, the execution frequency of the above operations is once a day, usually at midnight when the system is idle. To achieve high availability (availability), External system D is deployed on two servers, so you need to get information from both servers and insert information into the database, the number of valid messages is thousands, and the number of database inserts is twice times the number of valid information bars.

Figure 1. System Architecture Diagram

In order to realize the expected effect quickly, the implementation of the function was given priority in the initial implementation, but the system performance and code readability were not considered. The system generally has the following implementations: (1) REST API access information, the exception information that the database operation may throw is logged to the log file for debugging purposes, (2) a total of 5 database connection operations, including the first empty database table, two external system D each have two times the database insert operation, which 5 A connection is independent, release after use; (3) All database INSERT statements are generated using the Java.sql.Statement class, and (4) All database INSERT statements are executed by a single line, that is, to generate one execution, and (5) the entire process is executed in an individual thread, Includes database table emptying operations, database inserts, freeing database connections, (6) The JDBC code for database insert operations is scattered in the code. While this version of the system works as expected, it is inefficient, with a total of about 100 seconds to get information from the REST API, to parse and extract valid information, and to insert operations into the database. And the expected time should be within a minute, which is clearly not in line with the requirements.

Code optimization Process

The author begins to analyze the time-consuming operation of the whole process and how to improve the efficiency and shorten the time of program execution. Get information through the REST API, because it is the API provided by the external system, so it is not possible to improve efficiency here, and after the information is obtained, the effective part is parsed, because it is the resolution of the information in a particular format, so there is no room for efficiency improvement. Therefore, the efficiency can greatly enhance the space in the database operation part as well as the program control part. The following section describes the improved methods for time-consuming operations.

Optimization for logging

Turn off logging, or change the log output level. Because the information obtained from the external system D of the two servers is the same, the database insert operation throws an exception, and the exception information is similar to the "attempt to insert duplicate record", where the exception information is equal to the number of valid messages, thousands of them. This situation is predictable, so consider turning off logging, or changing the log output level without turning off logging, logging only the error messages for severity levels (severe level), and adjusting the log level of such operations to warning levels (warning level). This will not record the above exception information. This project uses the Java self-contained logging class, and the following configuration file sets the log output level to the severity level.

Listing 1. Log.properties set the log output level fragment

# Default file output is in user ' s home directory. 
# levels can Be:severe, WARNING, INFO, FINE, Finer, FINEST java.util.logging.consolehandler.level=severe 
Java.util.logging.filehandler.formatter=java.util.logging.simpleformatter 
Java.util.logging.filehandler.append=true

Through the above optimization, the performance has been greatly improved, from the original 100 seconds or so dropped to about 50 seconds. Why do you have such a large performance boost just by not logging? Consult the data, found that someone has done the relevant research and experiments. Often hear Java programs slower than C + + program speech, but the real reason for the slow speed, it is estimated that many people are not clear. For CPU-intensive programs (that is, programs that contain a large number of calculations), Java programs can achieve the same level of speed as C + + programs, but for I/O-intensive programs (that is, the program contains a large number of I/O operations), the Java program is much slower than the C/s C + + programs can directly access the underlying storage device. As a result, a significant performance boost without logging is due to the slow and time-consuming operation of the Java program's I/O operation.