Different data sources, different data manipulation processes--record recent project experience __hibernate

Recently in doing a very simple small project, although simple, but also encountered a lot of problems, now roughly recorded:

The project is basically divided into two main modules:

I do this piece, with the current universal SSH framework to maintain two tables, a user table, an Account table, two tables have no connection, everything is smooth.

The other is a colleague to do another module, is a timer task, set a fixed time interval, every time this period will carry out this task, carried out also very smoothly.

But when we had two integrations, the problem came, the data that I added to the foreground page he hold, only at the beginning of the time to be able to query the data table, I am here to add a new data query is not, looking for a long time to find: The original is the problem of data sources, I used a data source, He used a data source, and we used two data sources for two of them, so this was a different step, so it was solved temporarily by modifying the spring configuration and injecting my data source to him. Now I have the new data added to his side can be queried, is solved, last night to get more than eight to finish. But this morning I tried again and found a new problem, his side is a timer task, the dynamic maintenance of the data table, when the table data no longer meet the requirements will be deleted, but my front desk at this time the page also shows the deleted data, because there is no refresh, now show also normal, if the normal walk down, It should be me. A refresh will display only the data in the database, but the fact is not so, a flush directly error: No row with the given identifier exists

Find relevant information on the Internet, as follows:

The cause and solution of the problem of No row with the given identifier exists in hibernate

The cause of this problem:

There are two tables, Table1 and table2. The reason for this problem is that Table1 has an association <one-to-one> or <many-to-one unique= "true" > (a special Many-to-many mapping, which is actually one-to-one) To associate Table2. When the hibernate lookup, the data in table2 does not match the Table1, which will report no row with the given identifier this error. (A word, is the problem of data!)

If you say, Table1 has its own primary key ID1, and Table2 's primary key id2, these two fields.

If the hibenrate set is a single association, even if the id2 in Table1 is a null value, there is a value in Id2 in Table2, and the query does not make an error. But if the Table1 field in Id2 has a value, but the value is not in the primary key value in Table2, it will report the above error!

If the hibernate is a two-way association, then the ID2 in Table1 is a null value, but if there is a value in the Table2, the error is reported. The current solution to this situation is to change to a single link, or to change the corresponding data right!

This is the reason to report this mistake, know the reason for the corresponding change on the line. Maybe some people are confused Hibernate association are all good, how can there be such a mistake? In fact, this is the problem of programming, if you say I add information, The page passes through the formbean of struts to the DAO method that needs to be encapsulated into a hibernate po (that is, hibenrate bean), and if Po.get (Form.set ()) is too much trouble, it will generally write a special way to encapsulate , encountered Po.get (Form.set ()) This situation directly to the struts Formbean object to the encapsulation on the line, if I have a field is the creator ID, then this field is never changed, I added the time also called this method, This specially encapsulated method is somewhat judgmental, if I judge, if you encounter the creator ID passed to null value, I judge if it is null value, I set the Creator ID to 0, but the user table UserID is the primary key from 1 start, then this data is not appropriate, It would be a mistake to check it out. This error often occurs at the beginning of development, because each person's module is by the corresponding individual development completes after the integration together, each person writes alone that piece of time often will ignore these, therefore the integration time these questions often all come out suddenly all at once .... It's hard to integrate, tnnd!.

Comparison of Hibernate queries
There are a lot of hibernate queries, query,find,criteria,get,load

Query using the HSQL statement, you can set parameters is a common way

The criteria approach avoids writing HQL statements as much as possible and looks more object-oriented.

Find way, this way has been discarded by the new hibernate

Get and load methods take a record based on the ID
Here's a detailed look at the difference between get and load, because sometimes you can add find to the comparison.

1, from the results of the return comparison:
The Load method will throw a Org.hibernate.ObjectNotFoundException exception if it is not retrieved
The Get method will return NULL if it is not retrieved

2, from the search execution mechanism comparison:
Both the Get method and the Find method are retrieved directly from the database
But the execution of the load method is more complicated
1, first find out if there is a cache in the persistent context of the session, and return directly if there is one
2, if not to determine whether it is lazy, if not directly access to the database search, records returned, can not find thrown exception
3, if it is lazy, you need to establish a proxy object, the initialized property of the object is False,target property is null
4, retrieves the database when accessing the properties of the obtained proxy object, and copies the object of the record to target of the proxy object if a record is found
, and will Initialized=true, and throw an exception if it is not found.

The situation is similar to mine, although my side only a table, but the operation is very similar, I have a set of operating procedures, the timer there is also a set of operating procedures, the timer over there directly with the JDBC operation database, I use the hibernate operation database, because hibernate have a cache, data query out after , the hibernate inside has the corresponding cache record, the timer has deleted the data, the foreground one refreshes will report afore-mentioned such mistake, knows what reason, wants to solve also is very simple, the way tries to hibernate the cache to stop, The corresponding cache settings are available in the Hibernate profile Hibernate.cfg.xml:

<?xml version= ' 1.0 ' encoding= ' UTF-8 '?> <! DOCTYPE hibernate-configuration Public "-//hibernate/hibernate configuration DTD 3.0//en" "Http://hibe  Rnate.sourceforge.net/hibernate-configuration-3.0.dtd "> <!--generated by MyEclipse hibernate Tools.
		-->  
There are two lines of code:

<property name= "Hibernate.cache.use_query_cache" >false</property>
<property name= "Cache.use_" Second_level_cache ">false</property>
 
The original value is true and is now false to indicate that the cache is stopped


Hibernate Bulk Inserts, updates, and deletes 

BULK INSERT

During the development of the project, we often need to insert large quantities of data into the database because of the requirements of the project. Order of magnitude have million, level 100,000, millions, even tens other. This number of levels of data with Hibernate do insert operations, there may be an exception, the common exception is outofmemoryerror (memory overflow exception).

First, let's briefly review the mechanism of the hibernate insert operation. Hibernate to maintain its internal cache, when we perform the insert operation, the object to be manipulated is put into its own internal cache for management.

When it comes to hibernate caching, Hibernate has an internal cache and a level two cache. Since hibernate has different management mechanisms for the two caches, for level two caching, we can configure the size of the cache, and for the internal cache, Hibernate takes a "laissez-faire" attitude, with no restrictions on its capacity. Now that the crux is found, we do massive data insertion, generating so many objects will be included in the internal cache (the internal cache is cached in memory), so that your system memory will be 1.1 points of encroachment, if the last system is squeezed "fried", it is understandable.

Let's think about how to deal with the problem better. Some development conditions must use hibernate to deal with, of course, some projects more flexible, you can seek other methods.

The author recommends two kinds of methods here:
(1): Optimize the Hibernate, the procedure uses the subsection inserts in time clears the cache the method.
(2): Bypassing the Hibernate API, directly through the JDBC API to do bulk inserts, this method performance is the best, but also the fastest.
For the above Method 1, its basic idea is: Optimize hibernate, set the Hibernate.jdbc.batch_size parameter in the configuration file, specify the number of each commit SQL, and the method of using segmented inserts to clear the cache in time ( Session implementation of the asynchronous Write-behind, which allows hibernate explicit write operations of the batch), that is, each inserted a certain amount of data in a timely manner from the internal cache removed, freeing up memory occupied.
To set the Hibernate.jdbc.batch_size parameter, refer to the following configuration.

<session-factory>
.........
<property name= "Hibernate.jdbc.batch_size" >50</property>
.........
<session-factory>

The reason for configuring the Hibernate.jdbc.batch_size parameter is to read the database as little as possible, and the greater the number of hibernate.jdbc.batch_size parameters, the less the database will be read, the faster. As you can see from the above configuration, the hibernate is to wait until the program accumulates to 50 SQL before submitting the batch.

I also think that the Hibernate.jdbc.batch_size parameter value may not be set larger the better, from the performance point of view is still open to discussion. This should consider the actual situation, set as appropriate, the general situation set 30, 50 to meet the demand.
program implementation, the author to insert 10,000 data as an example, such as

Session session=hibernateutil.currentsession ();
Transatcion tx=session.begintransaction ();
for (int i=0;i<10000;i++)
{
Student st=new Student ();
St.setname ("Feifei");
Session.save (ST);

With every 50 data as a processing unit
if (i%50==0)
{
Simply submits the data in the hibernate cache to the database, keeping it synchronized with the database data

Session.flush ();

Clears all internal cached data and frees up memory in time
Session.clear ();
}
}
Tx.commit ();
.........

In a certain data scale, this approach can maintain the system memory resources in a relatively stable range.

Note: The previous mentioned level two cache, the author here is necessary to mention. If level two caching is enabled, from the mechanism hibernate in order to maintain level two cache, when we do insert, UPDATE, delete operation, hibernate will fill the corresponding data to the level two cache. There is a great loss in performance, so the author recommends disabling level two caching in batch processing.

For Method 2, a traditional JDBC batch is used, which is handled using the JDBC API.

Some methods refer to Java Batch for self execution SQL

Look at the code above, is not always feel that there is something wrong. Yes, you didn't find it. This is still the traditional JDBC programming, not a little hibernate flavor.
You can modify the above code to do the following:

Transaction tx=session.begintransaction (); Working with Hibernate transactions
Border Connection conn=session.connection ();
Preparestatement stmt=conn.preparestatement ("INSERT into t_student (name) VALUES (?)");
for (int j=0;j++;j<200) ... {
for (int i=0;i++;j<50)
... {
Stmt.setstring (1, "Feifei");
}
}
Stmt.executeupdate ();
Tx.commit (); Using hibernate transactions to process boundaries
.........

This change is very hibernate taste. The author after testing, using the JDBC API to do batch processing, performance than the use of the Hibernate API is nearly 10 times times higher performance of JDBC on the advantage of this is no doubt.

Batch update and delete

In Hibernate2, for batch update operations, hibernate is to identify the data that meets the requirements, and then do the update operation. Bulk deletion is also the case, first the eligible data detected, and then do the delete operation.
This has two big disadvantages:
(1): Occupy a large amount of memory.
(2): processing massive data, the execution of update/delete statements is massive, and a update/delete statement can only operate an object, so frequent operation of the database, low performance should be imagined.
After the release of Hibernate3, bulk Update/delete was introduced to the batch update/delete operation, which was based on a HQL statement to complete the bulk update/delete operation, much like a JDBC batch update/delete operation. In performance, there is a significant increase in bulk update/deletion than Hibernate2.

Transaction tx=session.beginsession ();
String hql= "Delete STUDENT";
Query query=session.createquery (HQL);
int size=query.executeupdate ();
Tx.commit ();
.......

Console output also on a DELETE statement hibernate:delete from T_student, the statement execution is less, performance is similar to the use of JDBC, is a good way to improve performance. Of course, in order to have better performance, the author recommends batch update and delete operation or use JDBC, method and the basic knowledge point and the above bulk Insert Method 2 is basically the same, here is not redundant.

The author here again provides a method, is to consider the performance from the database side, in the Hibernate program to invoke the stored procedure. Stored procedures run at the database end faster. Take the batch update as an example, give the reference code.
First, establish the name Batchupdatestudent stored procedure on the database side:

Create or replace Produre batchupdatestudent (a in number) as
Begin
Update STUDENT set age=age+1 where age>a;
End
The calling code is as follows:
Transaction tx=session.beginsession ();
Connection conn=session.connection ();
String pd= "... {call Batchupdatestudent (?)} ";
CallableStatement Cstmt=conn. Preparecall (PD);
Cstmt.setint (1,20); Set the age parameter to 20.
Tx.commit ();

Observe the above code, bypassing the Hibernate API, using the JDBC API to invoke the stored procedure, or the Hibernate transaction boundary. Stored procedures are undoubtedly a good way to improve batch processing performance, directly run with the database end, to some extent, the pressure of batch processing to the database.

Three: PostScript language
In this paper, we discuss the batch operation of Hibernate, which is based on the improvement of performance and only provides a small aspect of performance improvement.
Regardless of the approach to improve performance should be based on the actual situation to consider, to provide users with a satisfied demand and efficient and stable system is the most important.