Solution of data batch processing under Hibernate

Solve | Data Many people have the idea that Java is the right place to deal with bulk data, and that this extends, and that ORM may not be particularly suitable for batch processing of data. In fact, I think that if we apply it properly, we can completely eliminate the problem of ORM batch processing performance. Take hibernate as an example to illustrate, if we really have to use hibernate in Java to deal with the data in batches. Insert 100 000 data into the database, hibernate might look like this:

Session session = Sessionfactory.opensession ();
Transaction tx = Session.begintransaction ();
for (int i=0; i<100000; i++) {
Customer customer = new Customer (...);
Session.save (customer); }
Tx.commit ();
Session.close ();

Probably when running to 50th 000, there will be memory overflow and failure. This is hibernate the most recently inserted customer is cached in Session-level cache, we do not forget that Hiberante does not limit the cache size of First-level caches:

# Persistent object instance is managed at the end of a transaction, at which point hibernate synchronizes any changed managed objects with the database.

# Session implements the asynchronous Write-behind, which allows hibernate to explicitly write the batch of operations. Here, I give a hibernate how to implement a BULK insert:
First, we set a reasonable JDBC batch size, hibernate.jdbc.batch_size 20. The session is then flush () and clear () at a certain interval.

Session session = Sessionfactory.opensession ();
Transaction tx = Session.begintransaction ();
for (int i=0; i<100000; i++) {
Customer customer = new Customer (...);
Session.save (customer);
if (i% 20 = 0) {
Flush insert data and free memory:
Session.flush (); Session.clear (); }
}
Tx.commit ();
Session.close ();

So, how do you delete and update data? Well, in Hibernate2.1.6 or later versions, scroll () will be the best way to:

Session session = Sessionfactory.opensession ();
Transaction tx = Session.begintransaction ();
Scrollableresults customers = Session.getnamedquery ("GetCustomers")
. Scroll (scrollmode.forward_only);
int count=0;
while (Customers.next ()) {
Customer customer = (customer) customers.get (0);
Customer.updatestuff (...);
if (++count% 20 = 0) {
Flush update data and free memory:
Session.flush (); Session.clear (); } }
Tx.commit (); Session.close ();

This approach is not difficult, nor is it not elegant. Note that if the customer has Second-level caching enabled, we still have some memory management issues. The reason is that for every time the user inserts and updates, Hibernate has to notify the Second-level cache after the transaction is finished. Therefore, we will disable the user's use of caching in the case of batch processing.