[Record] Storing tens of millions of data into mysql

[Record] Storing tens of millions of data into mysql saves tens of millions of data records. The definition of storing data with hibernate is simple:

SQL code create table testEntity2 (id varchar (255) not null, description varchar (255), name varchar (255), primary key (id ))

 

Primary Key Generation Policy UUID to prevent data from being too large <property name = "hibernate. jdbc. batch_size "> 50 </property> at the same time, the Code flush the command 50 times and clear the command. In eclipse, the memory usage is about MB, and the command is printed every times. If batch_size is not used, the command is not flush. and clear memory will be around 2 GB and there will also be a memory overflow error :( but the previous several times the speed is very impressive to 280th thousand times when the heap overflow) but these are only the cache of the session that does not actually uncommit to the database. So how fast is there... hard to say... inserting

1:1  2:2  3:2  4:3  5:3  6:5  7:5  8:7  9:7  10:7  11:8  12:8  13:8  14:11  15:11  16:11  17:12  18:15  19:15  20:15  21:16  22:16  23:17  24:17  25:24  26:28  27:35  28:49  Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded      at org.hibernate.engine.internal.StatefulPersistenceContext.addEntry(StatefulPersistenceContext.java:539)      at org.hibernate.event.internal.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:249)      at org.hibernate.event.internal.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:192)      at org.hibernate.event.internal.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:135)      at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEventListener.java:206)      at org.hibernate.event.internal.DefaultSaveEventListener.saveWithGeneratedOrRequestedId(DefaultSaveEventListener.java:55)      at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.java:191)      at org.hibernate.event.internal.DefaultSaveEventListener.performSaveOrUpdate(DefaultSaveEventListener.java:49)      at org.hibernate.event.internal.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java:90)      at org.hibernate.internal.SessionImpl.fireSave(SessionImpl.java:764)      at org.hibernate.internal.SessionImpl.save(SessionImpl.java:756)      at org.hibernate.internal.SessionImpl.save(SessionImpl.java:752)      at org.cc.data.test.DataGenerator.main(DataGenerator.java:25)  

 

Then perform the test. Set batch_size to 10. flush and clear: insert 1: 372: 723: 1084: 144 ....... the tests of 50 and 100 are similar to the results, so I won't test the other ones. I will write it again later ..