Spring Batch Build Batch application 2

go from: http://www.ibm.com/developerworks/cn/java/j-lo-springbatch2/using the Spring batch batch, the feeling can get started. Before working on your own is the thread pool production-consumption mode to do batch processing, the next opportunity to try batch to try.
Preface

In part 1th of this series of articles, we have set up a batch task for user payment notifications. Although this simple application demonstrates the basic functionality of Spring Batch, it is far from true. In practical applications, our job may have to contain multiple Step, and in order to improve performance, we may need to consider the concurrency problem of the job. What good features does Spring Batch offer in these areas? Let's move on.

Step Flow

As we know from the previous article, step is a separate, sequential process that includes complete input, processing, and output. But in enterprise applications, we are faced with more situations where multiple steps are processed in a certain order. So how to maintain the order of execution between steps is what we need to consider. Spring Batch provides Step Flow to solve this problem. Example Improvements

Let's go back to the Job of user billing notice. Customers put forward further requirements: billing, deduction, payment notification to ensure that the sequential implementation. First, generate a bill for each user, then deduct from the user's balance, and send a payment notice for users with insufficient balance. Here's a look at how to use Step Flow to achieve that requirement.

Before we talk about Step Flow, let's make an improvement to the 1th part of the example by migrating it from a file operation to a database so that we can explain it later. The database initialization script is shown in Init_db_mysql.sql (located at the root of the sample code package batch_sample) and is configured in Listing 1: Listing 1. Billing_job.xml

<beans:bean id= "Jobrepository" class= " Org.springframework.batch.core.repository.support.JobRepositoryFactoryBean "> <beans:property name=" DataSource "ref=" DataSource "/> <beans:property name=" TransactionManager "ref=" TransactionManager "/> </ beans:bean> <beans:bean id= "Userdbreader" class= " Org.springframework.batch.item.database.JdbcPagingItemReader "> <beans:property name=" DataSource "ref=" DataSource "/> <beans:property name=" RowMapper "ref=" Userdbmapper "/> <beans:property name=" QueryProvider "ref=" Userqueryprovider "/> </beans:bean> <beans:bean id=" Userdbmapper "class=" Org.springframework.batch.sample.UserDbMapper "/> <beans:bean id=" Userqueryprovider "class=" Org.springframework.batch.item.database.support.MySqlPagingQueryProvider "> <beans:property name="
	Selectclause "value=" u.id,u.name,u.age,u.balance "/> <beans:property name=" FromClause "value=" Users U "/> <beans:property name= "SortKey"Value=" U.id "/> </beans:bean> <beans:bean id=" Messagedbwriter "class=" Org.springframework.batch.item.database.JdbcBatchItemWriter "> <beans:property name=" DataSource "ref=" DataSource "/> <beans:property name=" sql "value=" INSERT into messages (Id,user_id,content) VALUES (: Id,:user.id,:
Content) "/> <beans:property name=" Itemsqlparametersourceprovider "ref=" Itemsqlparametersourceprovider "/> </beans:bean> <beans:bean id= "Itemsqlparametersourceprovider" class= " Org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider "/>

We use the Jdbcpagingitemreader and Jdbcbatchitemwriter provided by Spring Batch respectively to read and write. At the same time, I modified jobrepository to Jobrepositoryfactorybean, so before running the sample, you also need to perform the schema-mysql.sql provided by Spring Batch (located in the core package org\ Springframework\batch\core directory). The relevant content will be explained in detail in part 3rd, which is not covered here. First Process

When you configure step, we can specify its next property, which points to another step. By configuring the next property of Step, we can easily implement the above process. As shown in Listing 2, Listing 2. Billing_job.xml

<job id= "Billingjob" restartable= "true" > <step id= "billingstep" next= "Paystep" > <tasklet> <chun K reader= "Userdbreader" processor= "Billingprocessor" writer= "Billdbwriter" commit-interval= "5" Chunk-completi On-policy= "" > </chunk> </tasklet> </step> <step id= "Paystep" next= "Messagestep" > <t  asklet> <chunk reader= "Billdbreader" processor= "Payprocessor" writer= "Paydbwriter" commit-interval= "5" Chunk-completion-policy= "" skip-limit= "> <skippable-exception-classes> <include class=" org.spring Framework.batch.sample.MoneyNotEnoughException "/> </skippable-exception-classes> </chunk> </ta sklet> </step> <step id= "Messagestep" > <tasklet> <chunk reader= "Billarrearsdbreader" proces Sor= "Messageprocessor" writer= "Messagedbwriter" commit-interval= "5" chunk-completion-policy= "" > </chunk
> </tasklet>	</step> </job> 

We set billstep next to Paystep, Paystep next to Messagestep, and specify the read, process, and write interfaces separately. When Spring Batch runs Billingjob, it first executes billingstep, finds user information to generate billing charges, then executes Paystep, finds billing information to generate a charge record, and skips if the user's balance is insufficient. Finally, find overdue bills and generate payment notifications. The next step is performed only if the previous execution succeeds.

The itemprocessor implementations of Billstep and Paystep are shown in Listing 3 and listing 4, respectively: listing 3.BillingItemProcessor class

public class Billingitemprocessor implements Itemprocessor#<user, bill> {public

	Bill process (User Item) Throws Exception {
		Bill B = new bill ();
		B.setuser (item);
		B.setfees (70.00);
		B.setpaidfees (0.0);
		B.setunpaidfees (70.00);
		B.setpaystatus (0);/*unpaid*/
		return b;
	}

}

Listing 4.PaymentItemProcessor Classes

 public class Paymentitemprocessor implements Itemprocessor<bill, payrecord> {public Payrecord process (Bill item
		) throws Exception {if (Item.getuser (). GetBalance () <= 0) {return null; } if (Item.getuser (). GetBalance () >= item.getunpaidfees ()) {//create Payrecord Payrecord PR = new Payrecord ()
			;
			Pr.setbill (item);
			Pr.setpaidfees (Item.getunpaidfees ());
			Update Balance Item.getuser (). Setbalance (Item.getuser (). GetBalance ()-item.getunpaidfees ());
			Update Bill Item.setpaidfees (Item.getunpaidfees ());
			Item.setunpaidfees (0.0);
		Item.setpaystatus (1);/* paid */return PR;
		} else {throw new moneynotenoughexception (); }
	}
}

In Listing 3, we generate a $70 bill for each user, with a paid fee of 0 and a 70 unpaid fee. In Listing 4, the bill amount is deducted from the user's balance and the billing and unpaid fees are updated, and if the balance is insufficient, the exception is indicated (we have skipped over this type of exception by listing 2).

In addition, our current payment notice needs to be generated based on the unpaid bill, so we need to provide a new itemprocessor for the payment notice, as shown in Listing 5: listing 5.ArrearsMessagesItemProcessor class

public class Arrearsmessagesitemprocessor implements
		Itemprocessor<bill, message> {public

	Message Process (Bill Item) throws Exception {
		if (item.getpaystatus () = = 0) {/*unpaid*/
			message m = new Message ();
			M.setuser (Item.getuser ());
			M.setcontent ("Hello" + item.getuser (). GetName ()
					+ ", please pay promptly at end of this month.");
			return m;
		}
		return null;
	}

}

Each Step read-write interface can refer to Billing_job.xml, using the implementation class provided by Spring Batch, which is not mentioned here (special attention is paid here to paydbwriter, we need to generate a deduction record at the same time, and update the user and Bill, So we used the Compositeitemwriter). Now that we have completed the first step, we have implemented the basic multi-step sequential processing, where you can run Main2 and view the results of the operation through the database (bills, payrecords, messages). conditional processes and process decisions

Through Step Flow above, we have satisfied the initial needs of the customer, but the customer further requests: whether the user fees are sufficient, no longer perform payment notice processing. Because querying the expense bill in some way reduces the processing performance. Spring Batch provides conditional process and process decisions to support similar scenarios.

First, let's look at how to use conditional processes to implement this requirement. Step supports the conditional flow by setting the On property on the next element, with the on attribute being the end state of step, such as completed, FAILED, and so on, while also supporting * and? Wildcard characters, which can be read in a reference manual.

Because we want to be in a situation where there is insufficient balance, that is, Paystep's skip number is greater than 0 o'clock, then the bill of payment step is executed, so we need to specify an end state specifically. Here, we can add a listener to Step to return the specified end state.

The modified Paystep as shown in Listing 6, the listener implementation is shown in Listing 7: listing 6. Billing_job.xml

<step id= "Paystep" >
	<tasklet>
		<chunk reader= "Billdbreader" processor= "Payprocessor" writer= "Paydbwriter"
			commit-interval= "5" chunk-completion-policy= "skip-limit=" >
			< skippable-exception-classes>
				<include
        class= " Org.springframework.batch.sample.MoneyNotEnoughException "/>
			</skippable-exception-classes>
		</chunk>
	</tasklet>
<next on= "completed with skips" to= "Messagestep"/>
	< listeners>
		<listener ref= "Paystepcheckinglistener" ></listener>
	</listeners>
</step>

Listing 7. Paystepcheckinglistener class

public class Paystepcheckinglistener extends Stepexecutionlistenersupport {

	@Override public
	exitstatus Afterstep (stepexecution stepexecution) {
		String ExitCode = Stepexecution.getexitstatus (). Getexitcode ();
		if (!exitcode.equals (ExitStatus.FAILED.getExitCode ())
				&& stepexecution.getskipcount () > 0) {
			return new Exitstatus ("Completed with skips");
		} else {
			return null;}}

}

Next, let's look at how to use process decisions to implement this functionality. In most cases, the end state of the Step does not satisfy the more complex condition process, and the process decision maker is used. Through it, we can control the process according to the execution status of Job and Step.

First, we need to define a process decider, as shown in Listing 8: listing 8. Messagesdecider class

public class Messagesdecider implements Jobexecutiondecider {public

	flowexecutionstatus decide (jobexecution Jobexecution,
			stepexecution stepexecution) {
		String ExitCode = Stepexecution.getexitstatus (). Getexitcode ();
		if (!exitcode.equals (ExitStatus.FAILED.getExitCode ())
				&& stepexecution.getskipcount () > 0) {
			return new Flowexecutionstatus ("Completed with skips");
		} else {
			return flowexecutionstatus.completed;}}

}

Unlike Stepexecutionlistener, the decide method of the class returns a Flowexecutionstatus object. In response, the JOB configuration is modified as shown in Listing 9: listing 9. Billing_job2.xml

<job id= "Billingjob" restartable= "true" >
	<step id= "Billingstep" next= "Paystep" >
	</step>
	<step id= "Paystep" next= "decider" >
	</step>
	<decision id= "decider" decider= " Messagesdecider ">
		<next on=" completed with skips "to=" Messagestep "/> <end
		on=" completed "/>
	</decision>
	<step id= "Messagestep" >
	</step>
</job>

You can see that Paystep's next becomes decider, and the execution path is determined in decider based on the return result: if there is a skip, execute the messagestep, otherwise directly end the Job (note: Here we use the end element).

Through the above, we generally understand the support of Spring Batch for the conditional process (in addition, we can implement the step's next property for the previously executed step, which enables the job to support the loop, but I do not think this is a good solution to the implementation of the cyclic task, So let's take a look at another important feature in batch processing-concurrency.

Back to top concurrent processing

If our batch task is simple enough, the hardware configuration and the network environment are good enough, we can fully design the batch task as a single thread, but the reality is that enterprise applications have more hardware requirements than the hardware itself, What's more, there are so many businesses that want to run their own enterprise applications in a poor hardware environment and want to have an acceptable performance. Therefore, in the enterprise application, especially involves the large-scale data processing, the concurrency is unavoidable. So, what support does Spring Batch provide in terms of concurrency?

First, Spring Batch provides concurrency in Step, which is also the simplest kind of concurrency processing support. By setting the Task-executor property for step, we can make the current step execute in a concurrent manner. Also, you can set the number of concurrent threads via Throttle-limit (default is 4). This means that you do not have to modify any of the business processing logic, and you can switch synchronously to asynchronous simply by modifying the configuration.

If we want the billingstep in the example to execute concurrently and the number of concurrent tasks is 5, then only the following configuration is required, as shown in the list: billing_job3.xml

<step id= "Billingstep" next= "Paystep" >
	<tasklet task-executor= "Taskexecutor" throttle-limit= "5" >
		<chunk reader= "Userdbreader" processor= "billingprocessor"
		writer= "Billdbwriter" commit-interval= "5" Chunk-completion-policy= "" >
		</chunk>
	</tasklet>
</step>
<beans:bean id= "Taskexecutor"
	class= "Org.springframework.core.task.SimpleAsyncTaskExecutor" >
</beans:bean>

As you can see from the list, we have specified an asynchronous task executor Simpleasynctaskexecutor for Billingstep, which will create a specified number of threads for data processing according to the configuration. In this way, we avoid the task of manually creating and managing threads, so that we only need to focus on the business process itself.

What needs to be added is that Spring Core provides us with a variety of actuator implementations (including multiple asynchronous actuators) that we can choose to use flexibly, depending on the actual situation. Of course, the asynchronous executor must be used when we need concurrent processing here. Several major implementations are shown in table 1: table 1. Task Executor list

synctaskexecutor The actuator is an ornament class for any other actuator and completes the function of providing a limit on the number of executions

class name	description	whether async
simple Synchronous executor	no
throttledtaskexecutor	depending on the decorated actuator
simpleasynctaskexecutor	Simple asynchronous executor, providing a most basic asynchronous execution implementation	Yes
workmanagertaskexecutor	This class is implemented as a task execution through the JCA specification, which contains Jbossworkmanagertas Kexecutor and Glassfishworkmanagertaskexecutor two subclasses	are
threadpooltaskexecutor