Hibernate batch Operations (ii)

Hibernate provides a series of query interfaces that differ in implementation. Here is a summary of the query interface in Hibernate.

Let's take a look at the session loading the Entity object: Session before invoking a database query, first the query is made in the cache. If the data state is valid when the entity type and ID are found and hit in the internal cache, it is returned directly. If no valid data is found in the internal cache, the second-level cache is queried and returned if the second-level cache hits. If there is no hit in the second-level cache, the database query operation (Select SQL) is initiated, and the corresponding data object is created based on the mapping configuration and the resultset obtained by Select SQL. Include its data object in the current session Entity management container (level cache). Execute the Interceptor.onload method (if there is a corresponding interceptor) to include the data object in level two cache. If the data object implements the lifecycle interface, the OnLoad method of the data object is called. Returns the data object.

Caching can greatly improve query performance, but not all interfaces in Hibernate implementations take advantage of the caching mechanism. The Find/iterator method of Session2 can query and return the set of entity objects that meet the query criteria according to the specified criteria. The List/iterate method in query interface queries in Hibernate3 also implements the same functionality (the Find () method is eliminated in the hibernate3.x version, and the interface is queried using query). From the implementation mechanism, these two interfaces have no essential difference. From the cache aspect, the Find/list method implements the query operation through a select SQL, while iterate executes the 1+ n query, first executes select SQL to get the ID that satisfies the condition, and then gets the corresponding record according to each ID.

However, the list (hibernate3) of the Find Method (Hibernate2)/query does not actually take advantage of the cache, it writes only to the cache. The Find method executes a select to get all the qualifying records from the database and constructs the corresponding entity objects, which are included in the cache after the entity objects have been built.

The Iterate method (Hibernate2)/query Iterate (hibernate3) takes full advantage of the data in the cache, and when the iterate method executes, it first executes a select SQL to get all the data IDs that satisfy the query criteria. Then, from the internal cache based on the ID of the corresponding entity object (similar to the Session.load method), if there is a corresponding data in the cache, the data object as the query results, if not found, then execute the corresponding SELECT statement from the database to obtain the corresponding record, and build the corresponding data object as the query results , and the result is also included in the cache. Thus, if the query data is read-only or read relatively frequently, this mechanism can greatly reduce the performance loss.

Another aspect, we know that the internal cache capacity is not limited, during the query process, a large number of data objects are included in the internal cache, resulting in the corresponding memory consumption. To control the size of the internal cache, you can manually clear Hibernate's cache. The data object can also be processed by the iterator method and the evict method, and the memory is consumed within an acceptable range. As follows:

String hql = "from XXX";

Query query = session.createquery (HQL);

Iterator iter = Query.iterate ();

while (Iter.hasnext ()) {

Object obj = Iter.next ();

Session.evict (obj);

}

Therefore, when the data volume is too large, you should avoid using Find (list), we recommend using iterate (iterate). In real-world application development, SQL or stored procedures are recommended for high-volume data processing to achieve higher performance.

The use of cursor-based data traversal operations is also a good query method, through the cursor, you can fetch data one by one, so that the memory is in a more stable state of use. As follows:

String hql = "from XXX";

Query query = session.createquery (HQL);

Scrollableresults scres = Query.scroll ();

while (Scres.next ()) {

Object obj = scres.get (0);

。。。

}

In addition, Hibernate provides an implementation that adds query criteria from the criteria object. Criteria query encapsulates the data query criteria into an object through an object-oriented design. In short, Criteria query can be seen as an object representation of traditional SQL, such as:

Criteria = Session.createcriteria (Tuser.class);

Criteria.add (Expression.eq ("name", "Erica"));

Criteria.add (Expression.eq ("Sex", New Integer (1));

Criteria.addorder (ORDER.ASC ("name")); Add sort

The criteria instance here is actually the encapsulation of SQL "SELECT * from T_user where name= ' Erica ' and sex=1". Hibernate generates the appropriate SQL statement at run time based on the query criteria specified in the criteria, which is the query expression added by the Criteria.add method in the above code.

For queries with large amounts of data, paging can be used to control the number of records returned per query. In Hibernate, the Setmaxresults method is used to limit the range of records returned by a query through the setfirstresult of the criteria (or query) interface. The following is an example of paging using the query interface in a spring template.

Public <T> list<t> findbypage (final String hql, final object[] values, final int offset, final int pageSize) {

list<t> list = Gethibernatetemplate (). Executefind (//Execute query through a Hibernatecallback object

New Hibernatecallback () {

Methods to implement the Hibernatecallback interface

Public Object Doinhibernate (session session) throws Hibernateexception, SQLException {

Performing Hibernate paging queries

Query query = session.createquery (HQL);

if (values! = null) {

Passing parameters to a HQL statement

for (int i = 0; i < values.length; i++) {

Query.setparameter (i, values[i]);

}

}

list<t> result = Query.setfirstresult (offset). Setmaxresults (pageSize). List ();

return result;

}

});

return list;

}

Paging is usually related to sorting, and there are two main types of sorting in Hibernate: 1) sort, 2) order-by.

A sortable set is provided in Sort:hibernate, which implements the Java.util.SortedSet. The Sort property can be configured in the Set element (sort= ' natural ', which specifies the Java default sorting mechanism by invoking the CompareTo method of the data type. The implementation of the Java.util.Comparator interface can be customized to implement a custom sorting algorithm that acts as a property value for sort. The map type is basically the same as set. But bag and list do not support sort sorting.

order-by: Adding order-by attributes to the element (such as order-by= "Address desc") enables the database to be sorted. This feature takes advantage of Linkedhashset in jdk1.4+ and Linkedhashmap, which must be JDK1.4 above the environment to succeed. Set, Map, bag support, list does not support this feature.

Configuration example: Sets the Sort property in the image file, for example, if set, as follows:

<set name= "Addrs" table= "Addrs" sort= "natural" >

<key column= "user_id"/>

<element type= "string" column= "ADDRESS" not-null= "true"/>

</set>

If it is a map, the following settings are set:

<map name= "Files" table= "Files" sort= "natural" >

<key column= "user_id"/>

<index column= "DESCRIPTION" type= "string"/>

<element type= "string" column= "FILENAME" not-null= "true"/>

</map>

This is set in set:

<set name= "Addrs" table= "Addrs" order-by= "ADDRESS desc" >

<key column= "user_id"/>

<element type= "string" column= "ADDRESS" not-null= "true"/>

</set>

It is also the same setting in map, and you can take advantage of function functions in the database, such as:

<map name= "Files" table= "files" order-by= "Lower (FILENAME)" >

<key column= "user_id"/>

<index column= "DESCRIPTION" type= "string"/>

<element type= "string" column= "FILENAME" not-null= "true"/>

</map>

In the query, you also need to consider a problem: After setting the 1-to-many relationship, the query will have an N + 1 problem.
1) 1-to-many, in the 1-party find the N objects, then need to have n objects associated with the collection, so that an original SQL query into n + 1 bar
2) Many to 1, in the multi-party query to get M objects, then will also be M objects corresponding to the 1-side object out, also become m+1

The idea of solving the n+1 problem is to utilize two kinds of retrieval strategies provided by Hibernate: Deferred retrieval strategy and urgent left outer connection retrieval strategy. The deferred retrieval strategy avoids loading the associated objects that the application does not need to access, and the urgent left outer connection retrieval strategy takes full advantage of SQL's outer join query function and reduces the number of SELECT statements.

? With the deferred retrieval policy, when lazy is set to true,lazy=true in the configuration, the associated object is not immediately queried, and the query action occurs only if you need to associate the object (access its properties, not the ID field). Using annotations, the declaration of a Table object with an associated appearance in this type of DTO adds a @fetch=fetchtype.lazy to its get method. (Hibernate3 default is Lazy=true).

Although the use of deferred retrieval avoids the need to execute redundant SELECT statements to load objects that the application does not require access to, it improves retrieval performance and saves memory space; However, if an application wants to access a free-state proxy class instance, it must ensure that the data object is initialized in the persisted state.

? Using the left OUTER join to retrieve the policy, set the outer-join=true in the configuration (or annotated: @ManyToOne () @Fetch (Fetchmode.join)

The left outer connection retrieval policy is fully transparent to the application, and the application can easily navigate from one object to the object associated with it, regardless of whether the object is persisted or is in a free state. At the same time, the number of SELECT statements is reduced due to the use of outer joins, but left join can also load objects that the application does not need to access to waste many memory space, and complex database table connections can also affect the retrieval performance and do not take advantage of SQL optimization;

For the urgent left outer connection retrieval, query's collection retrieval is not applicable, it takes the immediate retrieval strategy, and we also need to control the depth of the outer join through the Hibernate.max_fetch_depth property, because the outer join makes the SELECT statement more complex, The association between multiple tables will be time-consuming, and the performance of queries will fall sharply as the correlation becomes deeper.

Other ideas are: using level two caching, if the data is hit in the two cache, no more SQL queries will be generated. You can also set @batchsize (size=2) on an association class, where only two statements occur.

(@org. Hibernate.annotations.BatchSize allows you to define the number of instances of a bulk fetch of the entity (for example, @BatchSize (size=4)). When a specific entity is loaded, Hibernate will load non-initialized entities of the same type in the persistence context until the volume quantity (upper limit))
The criteria interface can also be set Setfetchmode to set the retrieval policy. See an article on the Internet

Session = Sessionfactory.opensession ();

Criteria = Session.createcriteria (One.class);

Criteria.add (Expression.eq ("Coneid", New Integer (1)));

one = (one) criteria.setfetchmode ("twos", Fetchmode.join). Setfetchmode ("Twos.threes", Fetchmode.join). SetFetchMode ( "Twos.threes.fours", Fetchmode.join). Uniqueresult ();

Session.close ();

When you set Fetchmode before using criteria, you should generate SQL statements dynamically for the criteria, so the resulting SQL is a layer of join down.

Setfetchmode (String,mode) The first parameter is association path, with "." to represent the path. This is a few specific examples and the documentation is not clearly written. I tried it a long time before I tried it out.

For this example, it takes three Setfetchmode to take the fourth floor.

The first path is twos, with two set in one. This specific to be more hbm.xml configuration to set.

The second path is twos.threes.

The third one is twos.threes.fours.

An analogy, an increase in layers.

In addition, SQL can be used to query directly, writing SQL statements as a form of union queries.

Attached: The following reference from http://www.blogjava.net/dreamstone/archive/2007/07/29/133071.html

Review the status of Hibernate entity objects. In Hibernate, an object defines three states: transient (transient or Free State), persistent (persistent state), detached (off-state or Free State).

? Transient state (transient): just established with new statement, not persisted, not in session cache, no corresponding record in database. A Java object that is in a temporary state is called a temporary object.
? Persistent object (persistent): Has been persisted, added to the session cache, and has a corresponding record in the database. Java objects that are persisted are called persistent objects and are automatically synchronized to update the database based on the properties of the persisted object.
? Managed (free) state (detached): Persistent object is managed after the session is closed, not in the session cache, if no other program deletes its corresponding record, then the database should have its record. You can continue to modify and then associate to the new session, which becomes the persistent object again, and the changes in the managed period are persisted to db. This makes long-time operations possible.

These three states can be transformed by some API calls to the session:

? Instances that are in a free state can be persisted by calling the save (), persist (), or Saveorupdate () method.
? A persisted instance can become a de-state by calling Delete (). The instances obtained through the get () or load () methods are persisted.
? Instances of the off-state can be persisted by calling update (), Saveorupdate (), lock (), or replicate ().

? Save () and persist () will trigger the SQL insert operation, and Delete () will trigger SQL Delete, and update () or merge () will trigger SQL update. Modifications to a persisted (persistent) instance are detected when the commit is refreshed, and it also causes SQL UPDATE. Saveorupdate () or replicate () will cause SQL INSERT or update

The differences between these APIs are as follows:

Save and update :
Save is a new object to be saved, and the update is to save the object in a de-state.

Update and Saveorupdate :
The object that the update () method operates on must be a persisted object that is persisted when the persisted object changes, and throws an exception if the object does not exist in the database. The Saveorupdate () method operates on objects that can either be persisted or made non-persisted. If the object called Saveorupdate () is persisted, the object in the database is updated, and if it is an object that is not persisted, save to the database is equivalent to a new object.

persist and save :
Refer to a description in http://opensource.atlassian.com/projects/hibernate/browse/HHH-1682:

I found that a lot of people has the same doubt. To the solve this issue I ' m quoting Christian Bauer: Anybody finds this thread ... Persist () is the well defined. It makes a transient instance persistent. However, It doesn ' t guarantee that the identifier value would be assigned to the persistent Instance immediately, the assignment might happen at flush time. The spec doesn ' t say That, which is the problem I has with persist (). Persist () also guarantees that it won't execute an INSERT statement if it's Called outside of transaction boundaries. This was useful in long-running conversations With an extended session/persistence context. A method like persist () is required. Save () does not guarantee the same, it returns a identifier, and if an INSERT Have to is executed to get the identifier (e.g. "identity" generator, not "sequence"), This inserts happens immediately, no matter if you are inside or outside of a transaction. This isn't good in a long-running conversation with an extended session/persistence context. "

Briefly translate the main contents of the above sentence:
1,persist persists a transient instance, but does not guarantee that the identifier is immediately populated into the persisted instance, and that the fill of the identifier may be deferred until the flush time.

2,persist "Guaranteed" does not trigger a SQL Insert when it is called outside of a transaction, which is useful when we encapsulate a long session process by inheriting Session/persistence context , a function such as persist is needed.

3,save "does not guarantee" 2nd, it returns an identifier, so it executes the SQL insert immediately, whether inside or outside transaction

saveorupdatecopy , Merge and update :
Merge is used instead of saveorupdatecopy, refer to
Http://www.blogjava.net/dreamstone/archive/2007/07/28/133053.html
。 Merge: Saves the state of the current object to the database and does not convert the object to a persisted state.

The difference between merge and update is that when we use update, when we do, we provide the State of object A to the persistence state, but when we use the merge, the execution is done, the object A is still out of the tube State, Hibernate or New has a B, or retrieves a persistent object B, and copies all of the values we provided for object A to this B, after execution completes B is the persistent state, and the A we provide is the managed state.

Flush and update :
The update operation is an object in a de-state, while flush is an object that operates on a persisted state.
By default, a persistent state object is not required for update, as long as you change the value of the object, waiting for Hibernate flush to be automatically saved to the database. Hibernate flush occurs in several cases:
1, when some queries are called
When 2,transaction commits,
3, when flush is manually called

Lock and update :
Update is to turn an object that has changed out of the tube state into a persistent state, and lock is to make a persistent state of an object that has not changed the state of the de-tube.
Corresponding to change the contents of a record, two different operations:
The procedure for update is:
(1) Change the object of the riser, call Update
The operation steps for lock are:
(2) Call lock to turn the object from the off-tube state to a persistent state--change the contents of the persisted state--wait for flush or flush manually

Load and get :

The 1:session.load/get method can read records from the database according to the specified entity class and ID, and returns the corresponding entity object. The difference is:
If a qualified record is not found, the Get method returns NULL, and the load method throws a Objectnotfoundexception;load method to return the entity's proxy class instance, and the Get method returns the entity class forever The Load method can take full advantage of the existing data in the internal cache and the level two cache, while the Get method only makes data lookups in the internal cache, and if no corresponding data is found, it will pass through the level two cache and directly call SQL to complete the data read.

The difference 2:load supports lazy loading, and get does not support lazy loading. Load loads when loaded, and the load policy defaults to lazy loading, that is, only the ID is loaded. If additional data is needed, a property must be loaded before the session is closed. Lazy= "true" or "false" if the load policy is loaded immediately, then it will load the data information all loaded, this time even, the session closed, because the data has been loaded, but also to obtain data. Get will load the data directly with the immediate load policy, whether you are configuring lazy loading or loading immediately. About immediate loading and lazy loading will work not only on the table itself, but also on the future tables that have a relationship with the table.

Either get or load, the cache is first looked up, and if not, the database lookup is done, the clear () method is called, the session cache can be forced to clear, and the flush () method can be forced to synchronize from memory to the database.

The list and iterator methods of query are different:

The list will not use the cache, and iterate will fetch the database select ID First, and then an ID of the load, if there is in the cache, from the cache, no words go to the database load.

Whether it is a list method or the Iterate method, the first time the query, they are the same way they usually do, the list executes a sql,iterate execution 1+n bar, the more the behavior is that they populate the cache query cache needs to open the class cache of the relevant classes. When the list and iterate methods are executed for the first time, both the query cache and the class cache are populated.

There is also an important problem that can easily be overlooked, that is, even the list method may encounter 1+n problems after opening the query cache!

A>list Remove All

B>iterate take out the ID, wait for the time to use the object by ID

The list in C>session is sent for the second time, and will still go to the database query

D>iterate the second time, first find session level cache

Getcurrentsession () must be used in the transaction!!!

Hibernate batch Operations (ii)