Spring Batch_ using Jdbcpagingitemreader_ multi-threaded step

Spring Batch_ using Jdbcpagingitemreader_ multi-threaded step

The most common use of jdbccursoritemreader is to use cursors to read each data in one way or the other. But from the official spring documentation we know that he is not thread safe. Here, we use the Jdbcpagingitemreader

Read the data from the database, and it is paged read, and this class is thread-safe, then we can use multi-threaded step, so as to improve JOB execution efficiency.

The following are the main configuration files:

<beans xmlns= "Http://www.springframework.org/schema/beans" xmlns:xsi= "http://www.w3.org/2001/ Xmlschema-instance " xmlns:batch=" Http://www.springframework.org/schema/batch "xmlns:context="/HTTP/ Www.springframework.org/schema/context "xsi:schemalocation=" http://www.springframework.org/schema/beans  Http://www.springframework.org/schema/beans/spring-beans-4.0.xsdhttp://www.springframework.org/schema/batch  http://www.springframework.org/schema/batch/spring-batch.xsdhttp://www.springframework.org/schema/ Context http://www.springframework.org/schema/context/spring-context.xsd "><!--  Pack Scan  -- ><context:component-scan base-package= "Com.lyx.batch"  /><bean id= "ExceptionHandler"  class= "Com.lyx.batch.ExceptionListener"  /><batch:step id= "Abstractstep"  abstract= " True "><batch:listeners><batch:listener ref=" Exceptionhandler " /></batch:listeners ></batch:step><bean id= "ABstractjdbcpagingitemreader " abstract=" true "class=" Org.springframework.batch.item.database.JdbcPagingItemReader "><property name=" DataSource " ref= "DataSource"  /></bean><!-- add people desc job begin -->< Batch:job id= "Addpeopledescjob" ><batch:step id= "Adddescstep"  parent= "AbstractStep" > <batch:tasklet><batch:chunk reader= "Peopleadddescreader"  processor= "AddDescProcessor" writer = "Adddescpeoplewriter"  commit-interval= "2"  /></batch:tasklet></batch:step></batch: job><!-- add people desc job end --><!--  Reader begin with pagination  --><bean id= "Peopleadddescreader"  parent= "Abstractjdbcpagingitemreader" scope= "Step" > <property name= "Queryprovider" ><beanclass= " Org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean "><property name=" DataSource " ref= "DataSource"  /><property name= "Selectclause"  value= "select person_id,  first_name, last_name " /><property name=" FromClause " value=" From people "  /><property name= "Whereclause" value= "where  ( first_name like :first_name  or last_name like :last_name )   " /><property name=" SortKey "  value= "person_id"  /></bean></property><property name= "ParameterValues" > <map><entry key= "first_name"  value= "#{jobparameters[' first_name ']}"  /><entry  key= "last_name"  value= "#{jobparameters[' last_name ']}"  /></map></property><!- -  Configure the size of limit  --><property name= "PageSize"  value= "2"  /><property name = "RowMapper"  ref= "Peoplerowmapper"  /></bean><!--  using pagination reader end --> <bean id= "PeoplerowmaPper " class=" Com.lyx.batch.PeopleRowMapper " /><bean id=" Adddescprocessor " class=" Com.lyx.batch.AddPeopleDescProcessor " /><bean id=" Adddescpeoplewriter " class=" Com.lyx.batch.AddDescPeopleWriter " /><!--tomcat jdbc pool Data Source Configuration  --><bean  Id= "DataSource"  class= "Org.apache.tomcat.jdbc.pool.DataSource" destroy-method= "Close" ><property  name= "Poolproperties" ><bean class= "Org.apache.tomcat.jdbc.pool.PoolProperties" >< Property name= "Driverclassname"  value= "com.mysql.jdbc.Driver"  /><property name= "url"  value= "Jdbc:mysql://localhost:3306/test"  /><property name= "username"  value= "root"  /><property name= "Password"  value= "034039"  /></bean></property></ bean><!-- spring batch  Configuration Jobrepository --><batch:job-repository id= " Jobrepository "data-source=" DataSource " Transaction-manager= "TransactionManager" isolation-level-for-create= "Repeatable_read"  table-prefix= "BATCH_ "Max-varchar-length="  /><!-- spring transaction manager  --><bean id= "TransactionManager "Class=" Org.springframework.jdbc.datasource.DataSourceTransactionManager "><property name=" DataSource " ref=" DataSource " /></bean><!-- batch luncher --><bean  id= "Joblauncher" class= "Org.springframework.batch.core.launch.support.SimpleJobLauncher" ><property  name= "Jobrepository"  ref= "Jobrepository"  /></bean></beans>

The main configuration is:

<!--  using reader begin --><bean id= "Peopleadddescreader"  parent= "for pagination" Abstractjdbcpagingitemreader "scope=" Step "><property name=" Queryprovider "><beanclass=" Org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean "><property name=" DataSource " ref=" DataSource " /><property name=" Selectclause " value=" select  Person_id, first_name, last_name " /><property name=" FromClause " value=" from  people " /><property name=" Whereclause "value=" where  ( first_name like  :first_name or last_name like :last_name )    /><property  Name= "SortKey"  value= "person_id"  /></bean></property><property name= " ParameterValues "><map><entry key=" first_name " value=" #{jobparameters[' first_name ']} "  /><entry key= "Last_Name"  value= "#{jobparameters[' last_name '} '  /></map></property><!--  Configure the size of limit  -->< Property name= "PageSize"  value= "2"  /><property name= "RowMapper"  ref= " Peoplerowmapper " /></bean><!--  reader end --> with pagination

Other classes from the previous article find out where I'm debugging in order to debug a different class from the previous

Peoplerowmapper.java

Package Com.lyx.batch;import Java.sql.resultset;import Java.sql.sqlexception;import Org.springframework.jdbc.core.rowmapper;public class Peoplerowmapper implements rowmapper<people> {public People Maprow (ResultSet rs, int rowNum) throws SQLException {people p = new people (); System.out.println ("-----------------------person_id-" + rs.getint ("person_id"));p. SetId (Rs.getint ("person_id")) ;p. Setfirstname (rs.getstring ("first_name"));p. Setlastname (rs.getstring ("last_name")); return p;}}

Run the program

Appmain14.java

package com.lyx.batch;import org.springframework.batch.core.exitstatus;import  org.springframework.batch.core.job;import org.springframework.batch.core.jobexecution;import  org.springframework.batch.core.jobparametersbuilder;import  org.springframework.batch.core.jobparametersinvalidexception;import  org.springframework.batch.core.launch.joblauncher;import  org.springframework.batch.core.repository.jobexecutionalreadyrunningexception;import  org.springframework.batch.core.repository.jobinstancealreadycompleteexception;import  org.springframework.batch.core.repository.jobrestartexception;import  org.springframework.context.applicationcontext;import  org.springframework.context.support.classpathxmlapplicationcontext;/** *  testing   using  reader for pagination  *  *  @author  lenovo * */public class appmain14 {public  static void main (String[] args) throws jobexecutionalreadyrunningexception, jobrestartexception,jobinstancealreadycompleteexception,  Jobparametersinvalidexception {long starttime = system.currenttimemillis (); //  Get Start time @suppresswarnings ("resource") applicationcontext context = new  Classpathxmlapplicationcontext (new string[] {  "Classpath:spring-batch-paging.xml"  }); Jobparametersbuilder jobparametersbuilder = new jobparametersbuilder (); Jobparametersbuilder.addstring ("first_name",  "%john%"); Jobparametersbuilder.addstring ("Last_Name",  " %doe% "); job job =  (Job)  context.getbean ("Addpeopledescjob"); joblauncher launcher =  (Joblauncher)  context.getbean ("Joblauncher"); Jobexecution result = launcher.run (Job,jobparametersbuilder.tojobparameters ()); Exitstatus es = result.getexitstatus ();if  (Es.getexitcode (). Equals ( ExitStatus.COMPLETED.getExitCode ()))  {system.out.println ("task completed properly");}  else {system.out.println ("Task failed, exitcode="  + es.getexitcode ());} Long endtime = system.currenttimemillis (); //  Gets the end time System.out.println ("program run time:  " +  (endtime - starttime)  + " MS ");}}

Operation Result:

-----------------------person_id-157

-----------------------person_id-158

Process People desc

Process People desc

Write People desc

Write People desc

Task completed properly

Program Run time: 11929ms

November 22, 2014 12:29:50 am Org.springframework.batch.core.launch.support.SimpleJobLauncher Run

Info: Job: [Flowjob: [Name=addpeopledescjob]] completed with the following parameters: [{first_name=%john%, Last_name=%doe %}] and the following status: [Completed]

Finally, it was a success. In fact, more important is the implementation of Jdbcpagingitemreader and source code. Why he is thread-safe, why he can read pages, that's what we finally care about. Here first not analysis, to the next article.

Here we are using a single-threaded way to run the job, let's configure the multithreaded step. Configuration is simple. is to configure an asynchronous spring task executor that uses the asynchronous task executor to run our

Job:

Look at the following configuration:

<bean id= "Taskexecutor" class= "Org.springframework.core.task.SimpleAsyncTaskExecutor"/><!--Add people Desc Job begin--><batch:job id= "Addpeopledescjob" ><batch:step id= "Adddescstep" parent= "AbstractStep" ><batch:tasklet task-executor= "Taskexecutor" ><batch:chunk reader= "Peopleadddescreader" processor= " Adddescprocessor "writer=" Adddescpeoplewriter "commit-interval=" 2 "/></batch:tasklet></batch:step> </batch:job><!--Add People desc job End--

Here Taskexecutor is an asynchronous task executor, specifically how to implement, please see the source code.

Run the multithreaded step below.

Operation Result:

-----------------------person_id-157

-----------------------person_id-158

Process People desc

Process People desc

Write People desc

Write People desc

November 22, 2014 1:01:35 am Org.springframework.batch.core.launch.support.SimpleJobLauncher Run

Info: Job: [Flowjob: [Name=addpeopledescjob]] completed with the following parameters: [{first_name=%john%, Last_name=%doe %}] and the following status: [Completed]

Task completed properly

Program Run time: 8577ms

Success, OK, have you noticed that there is less time to run the program than the single-threaded step above.

On the implementation of Jdbcpagingitemreader and its thread security, how to page, Jdbcpagingitemreader paging policy we are in the following article.

=================================end=================================

Spring Batch_ using Jdbcpagingitemreader_ multi-threaded step