Rowhandler of ibatis

When ibatis is used for SQL queries with a large amount of data (such as millions and tens of millions of data records), if you follow the general programming method (such as queryforlist) when you query the data and perform the operation again, performance problems may occur, such:
1. A large amount of JVM memory consumption;
2. Intensive creation of a large number of objects puts a certain amount of pressure on GC;
3. Data Query and subsequent data operations cannot be performed in parallel.

 

To cope with such scenarios, ibatis provides a rowhandler interface that allows you to customize the query result set in callback mode, you can customize the handlerow method to process a piece of data at a time to improve the flexibility of the framework.

/** * Event handler for row by row processing. * <p/> * The RowHandler interface is used by the SqlMapSession.queryWithRowHandler() method. * Generally a RowHandler implementation will perform some row-by-row processing logic * in cases where there are too many rows to efficiently load into memory. * <p/> * Example: * <pre> * sqlMap.queryWithRowHandler ("findAllEmployees", null, new MyRowHandler())); * </pre> */public interface RowHandler {  /**   * Handles a single row of a result set.   * <p/>   * This method will be called for each row in a result set.  For each row the result map   * will be applied to build the value object, which is then passed in as the valueObject   * parameter.   *   * @param valueObject The object representing a single row from the query.   * @see com.ibatis.sqlmap.client.SqlMapSession   */  void handleRow(Object valueObject);}

 

The role of rowhandler is as follows:
In the case of large data volumes, for example, if a bank makes a daily "accrual" operation on its account and has 10 million accounts, it needs to query 10 million accounts, then perform 10 million updates (if you can use addbatch, you certainly do not need to perform 10 million updates ). To improve performance as much as possible:
1. it is necessary to prevent burst object creation, because the burst creation will cause a sharp increase in memory. If there is no good design, it will also lead to the destruction of the object burst (that is: after processing the last data, all objects will be invalidated, and the memory will be occupied during the 10 million update ). You can customize rowhandler to process data one by one, and execute the update operation in the handlerow method. After the update, the object is released immediately, and the operation is "discretization ", reduces memory usage.
2. The above "discretization" is used to solve the memory usage problem, but the total time of query and data processing is not reduced. In order to achieve higher efficiency, "Asynchronization" processing is also required. In the handlerow method, send valueobject to the asynchronous queue for parallel query and execution, thus reducing the total time. In fact, the decrease in the total time reflects only one advantage. Another advantage is that the query time after "Asynchronization" does not differ much from the query time when queryforlist is directly called, after the query, the connection is quickly released, reducing the pressure on the database.


Note:
When the application rowhandler is applied, it may still find a sharp increase in memory, because the resultmap returned by the underlying JDBC still returns all the data at a time. This problem can be solved by setting fetchsize. fetchsize is a configuration variable provided by JDBC. The JDBC driver cyclically retrieves data from the database based on the configuration value of this variable, instead of loading data to the memory at a time, in this way, you can set the optimal parameters according to your actual situation. The select tag of ibatis exposes this attribute and can independently set a specific statement, to achieve optimal performance. However, fetchsize cannot be too small. Otherwise, frequent interaction between Java and the database will occur. The memory problem is solved, but the performance problem caused by frequent interaction may occur again, therefore, the fetchsize value must be set flexibly based on specific scenarios and experimental data.
Link to fetchsize:
Http://www.2cto.com/database/201305/209625.html
Http://www.java3z.com/cwbwebhome/article/article8/828.html? Id = 2244
Http://www.smithfox.com /? E = 153
Http://stackoverflow.com/questions/3870500/ibatis-querywithrowhandler-still-seems-to-fetch-all-rows
Http://docs.aws.amazon.com/redshift/latest/dg/jdbc-fetch-size-parameter.html

Rowhandler can also be used to create object associations to solve the "n + 1 select" problem. This is not detailed here. For more information, see the relevant chapter in ibatis in action.
Rowhandler can do a lot of things. It is a public mechanism that is often used to solve efficiency problems. When the ibatis framework cannot meet our needs, you can consider to use this interface.
As for performance, this article focuses on both theoretical and general scenarios. Specific solutions must vary with scenarios, and experience can be combined with requirements to achieve the best design.

 

 

Rowhandler of ibatis