Spring Batch_parallel Steps_ uses a parallel step

Spring Batch_parallel Steps_ uses a parallel step

Spring Official Document: Http://docs.spring.io/spring-batch/trunk/reference/html/scalability.html#scalabilityParallelSteps

As long as the application logic that needs to be parallelized can is split into distinct responsibilities, and assigned T o Individual steps then it can is parallelized in a single process. Parallel Step execution is easy-to-configure and use, for-example, to-execute steps (STEP1,STEP2) in Parallel with Step3, Could configure a flow like this:

<beans:bean id= "Taskexecutor" class= "Org.spr ... Simpleasynctaskexecutor "/>

The configurable "Task-executor" attribute is used to specify which Taskexecutor implementation should being used to execute The individual flows. The default is Synctaskexecutor, but a asynchronous taskexecutor is required to run the steps in parallel. Note that the job would ensure that every flow in the split completes before aggregating the exit statuses and Transitionin G.

the parallelism between multiple step can improve the efficiency of batch processing. When you can apply the parallelization between step, it depends on the specific business requirements.

Then we assume that there is a kind of scenario: There is a kind of data, which exist in the file and the database, the content of the data, but the form is different, then we can define the parallel step to process the data from the file and the data from the database, and then, Perform the same processor separately, and then write to the database.

Let's follow the scenario we've just talked about to implement our parallel step, as follows:

Spring-batch-split.xml

<beans xmlns= "Http://www.springframework.org/schema/beans" xmlns:xsi= "http://www.w3.org/2001/ Xmlschema-instance " xmlns:batch=" Http://www.springframework.org/schema/batch "xmlns:context="/HTTP/ Www.springframework.org/schema/context "xsi:schemalocation=" http://www.springframework.org/schema/beans  Http://www.springframework.org/schema/beans/spring-beans-4.0.xsdhttp://www.springframework.org/schema/batch  http://www.springframework.org/schema/batch/spring-batch.xsdhttp://www.springframework.org/schema/ Context http://www.springframework.org/schema/context/spring-context.xsd "><!--  Pack Scan  -- ><context:component-scan base-package= "Com.lyx.batch"  /><bean id= "ExceptionHandler"  class= "Com.lyx.batch.ExceptionListener"  /><batch:step id= "Abstractstep"  abstract= " True "><batch:listeners><batch:listener ref=" Exceptionhandler " /></batch:listeners ></batch:step><bean id= "ABstractcursorreader " abstract=" true "class=" Org.springframework.batch.item.database.JdbcCursorItemReader " ><property name= "DataSource"  ref= "DataSource"  /></bean><bean id= " Taskexecutor " class=" Org.springframework.core.task.SimpleAsyncTaskExecutor " /><!--  Parallel step --><batch:job id= "Addpeopledescjob" ><batch:split id= "split1"   Task-executor= "Taskexecutor" ><batch:flow><batch:step id= "Parallel_step_1" ><batch: Tasklet><batch:chunk reader= "peopleadddescreader_db" processor= "Adddescprocessor"  writer= " Adddescpeoplewriter "Commit-interval="  /></batch:tasklet></batch:step></batch:flow ><batch:flow><batch:step id= "Parallel_step_2" ><batch:tasklet><batch:chunk  Reader= "Peopleadddescreader_file" processor= "Adddescprocessor"  writer= "Adddescpeoplewriter" Commit-interval = "Ten"  /></batch:tasklet></batch:step></batch:flow></batch:split></batch:job><!--  reader -->< reading data from a database Bean id= "peopleadddescreader_db"  parent= "Abstractcursorreader" scope= "Step" ><property name = "SQL" ><value><! [cdata[select first_name ,last_name from people where first_name like  ?  or last_name like ]] ></value></property><property name= "RowMapper"  ref= "PeopleRowMapper"  /> <property name= "PreparedStatementSetter"  ref= "PreparedStatementSetter"  /><property  name= "Fetchsize"  value= " /></bean><bean id=" Peoplerowmapper " class=" Com.lyx.batch.PeopleRowMapper " /><bean id=" PreparedStatementSetter " class=" Com.lyx.batch.PeoplePreparedStatementSetter " /><bean id=" Adddescprocessor " class=" Com.lyx.batch.AddPeopleDescProcessor " /><bean id=" adddescpeopleWriter " class=" Com.lyx.batch.AddDescPeopleWriter " /><!--  read data from a file reader -->< Bean id= "Linetokenizer" class= "Org.springframework.batch.item.file.transform.DelimitedLineTokenizer" > <property name= "delimiter"  value= ","  /><property name= "names" ><list>< Value>firstname</value><value>lastname</value></list></property></bean> <bean id= "Fieldsetmapper" class= "Org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper" ><property name= "Prototypebeanname"  value= "People"  /></bean><bean id= " People " class=" Com.lyx.batch.People " scope=" prototype " /><bean id=" Linemapper "class= "Org.springframework.batch.item.file.mapping.DefaultLineMapper" ><property name= "Linetokenizer"  ref= "Linetokenizer"  /><property name= "Fieldsetmapper"  ref= "FieldSetMapper"  / ></bean><bEan id= "Resource"  class= "Org.springframework.core.io.ClassPathResource" ><constructor-arg  index= "0"  type= "java.lang.String" value= "Sample-data.csv"  /></bean><bean id= " Peopleadddescreader_file " class=" Org.springframework.batch.item.file.FlatFileItemReader "><property  name= "Resource"  ref= "resource"  /><property name= "encoding"  value= "Utf-8"   /><property name= "Linemapper"  ref= "Linemapper"  /></bean><!--reader reading data from a file  end --><!--tomcat jdbc pool Data Source configuration  --><bean id= "DataSource"   class= "Org.apache.tomcat.jdbc.pool.DataSource" destroy-method= "Close" ><property name= "poolproperties "><bean class=" Org.apache.tomcat.jdbc.pool.PoolProperties "><property name=" Driverclassname " value=" com.mysql.jdbc.Driver " /><property name=" url " value=" JDBC: Mysql://localhost:3306/test " />&Lt;property name= "username"  value= "root"  /><property name= "password"  value= " 034039 " /></bean></property></bean><!-- spring batch  Configure Jobrepository --><batch:job-repository id= "Jobrepository" data-source= "DataSource"   Transaction-manager= "TransactionManager" isolation-level-for-create= "Repeatable_read"  table-prefix= "BATCH_ "Max-varchar-length="  /><!-- spring transaction manager  --><bean id= "TransactionManager "Class=" Org.springframework.jdbc.datasource.DataSourceTransactionManager "><property name=" DataSource " ref=" DataSource " /></bean><!-- batch luncher --><bean  id= "Joblauncher" class= "Org.springframework.batch.core.launch.support.SimpleJobLauncher" ><property  name= "Jobrepository"  ref= "Jobrepository"  /></bean></beans>

If the above configuration is slightly abrupt, please find the source in the previous series of spring batch articles

The above is the main configuration of step parallelization.

Appmain7.java

package com.lyx.batch;import org.springframework.batch.core.exitstatus;import  org.springframework.batch.core.job;import org.springframework.batch.core.jobexecution;import  org.springframework.batch.core.jobparametersbuilder;import  org.springframework.batch.core.jobparametersinvalidexception;import  org.springframework.batch.core.launch.joblauncher;import  org.springframework.batch.core.repository.jobexecutionalreadyrunningexception;import  org.springframework.batch.core.repository.jobinstancealreadycompleteexception;import  org.springframework.batch.core.repository.jobrestartexception;import  org.springframework.context.applicationcontext;import  org.springframework.context.support.classpathxmlapplicationcontext;/** *  Testing for Parallel job *   *  @author  lenovo * */public class appmain7 {public static void  main (String[] args) Throws jobexecutionalreadyrunningexception, jobrestartexception,jobinstancealreadycompleteexception, jobparametersinvalidexception { Long starttime = system.currenttimemillis (); //  Get start time @suppresswarnings ("resource") Applicationcontext context = new classpathxmlapplicationcontext (new String[] {   "Classpath:spring-batch-split.xml"  }); Jobparametersbuilder jobparametersbuilder = new jobparametersbuilder (); job job =  (Job)  context.getbean ("Addpeopledescjob"); joblauncher launcher =  (Joblauncher)  context.getbean ("Joblauncher"); Jobexecution result = launcher.run (Job,jobparametersbuilder.tojobparameters ()); Exitstatus es = result.getexitstatus ();if  (Es.getexitcode (). Equals ( ExitStatus.COMPLETED.getExitCode ()))  {system.out.println ("task completed");}  else {system.out.println ("Task failed, exitcode="  + es.getexitcode ());} Long endtime = system.cuRrenttimemillis (); //  Get end Time System.out.println ("program run Time: "  +  (endtime -  StartTime)  +  "MS");}}

=====================end=====================

Spring Batch_parallel Steps_ uses a parallel step