How Spring Batch is robust for applications that can restart traceable Skip/retry/restart policies

Premise: You already have a certain spring foundation

You can already run a simple instance of spring batch

Reference: http://www.cnblogs.com/gulvzhe/archive/2011/10/25/2224249.html

Http://www.cnblogs.com/cdutedu/p/3789396.html

Steal a few pictures first

Joblauncher Specifies a jobrepository

The jobrepository contains some parameters for incoming jobs, and there are six main tables to store

Each job can correspond to multiple step

... <batch:step id= "Astep" next= "bstep" ><batch:tasklet><batch:chunk reader= "AReader" writer= "AWriter "Processor=" Aprocessor "commit-interval=" "/></BATCH:TASKLET></BATCH:STEP>

Tesklet inside the work block for chunk, each chunk execute commit-interval, that is, submit once, you can open more than one chunk

Each step is executed like this

(Excerpt from the following paragraph)

When data is fetched from a db or file, the read () operation reads only one record at a time, then passes the read data to the processor (item) processing, and the framework repeats the two steps until the number of records is read to the batch configuration information " Commin-interval "When the value is set, a write operation is called. The processing is then repeated until all the data is processed. When the step is finished, either jump to another step, or end the process.

So that's the problem? What happens when I read the data to the processing? Will it cause the entire batch to be interrupted?

There are exceptions that are not captured, and the entire process is suspended to the topmost level, resulting in an entire batch outage.

Obviously, in order to not affect the subsequent processing, either catch the exception, play the log, skip. Either, retry (very efficient in the case of IO timeouts or table locking-transient conditions).

Even if there is a disruption, we need to restart the job

Some of the above correspond to the Skip\retyr\restart in Spring-batch

First, SKIP

<job id= "Importproductsjob" ><step id= "Importproductsstep" ><tasklet><chunk reader= "Reader" Writer= "Writer" commit-interval= "skip-limit=", "><skippable-exception-classes><include class=" Org.springframework.batch? item.file.FlatFileParseException "/></skippable-exception-classes></ Chunk></tasklet></step></job>

Skippable-exception-classes with exception types that require skip

Skip-limit can be up to a maximum number of fault tolerance times, and this step interrupts

You can also add listener to log

<bean id= "SkipListener" class= "com.manning?. Sbia.ch08.skip.DatabaseSkipListener" ><constructor-arg ref= " DataSource "/></bean><job id=" Importproductsjob "xmlns=" Http://www.springframework.org/schema/batch " ><step id= "Importproductsstep" ><tasklet><chunk reader= "reader" writer= "writer" commit-interval= " "Skip-limit=", "><skippable-exception-classes><include class=" Org.springframework.batch.item.file?. Flatfileparseexception "/></skippable-exception-classes></chunk><listeners><listener ref = "SkipListener"/></listeners></tasklet></step></job>

Second, retrying on error

Mainly for IO operation, concurrency, etc., transient errors

Configuration similar to skip

<job id= "Importproducsjob" ><step id= "Importproductsstep" ><tasklet><chunk reader= "Reader" Writer= "Writer" commit-interval= "retry-limit=" 3 "><retryable-exception-classes><include class=" Org.springframework.dao?. Optimisticlockingfailureexception "/></retryable-exception-classes></chunk></tasklet></ Step></job>

If you do not want to retry the number of retries, you can mix retry and skip both because these errors result in a step interrupt exit.

<job id= "Job" ><step id= "step" ><tasklet><chunk reader= "reader" writer= "writer" commit-interval= "Retry-limit=" 3 "skip-limit=" ><retryable-exception-classes><include class= " Org.springframework.dao?. Deadlockloserdataaccessexception "/></retryable-exception-classes><skippable-exception-classes> <include class= "Org.springframework.dao? Deadlockloserdataaccessexception "/></skippable-exception-classes></chunk></tasklet></ Step></job>

You can also control the retry by customizing the policy

<job id= " Retrypolicyjob "xmlns=" Http://www.springframework.org/schema/batch "><step id=" Retrypolicystep ">< Tasklet><chunk reader= "Reader" writer= "writer" commit-interval= "" retry-policy= "Retrypolicy"/></ Tasklet></step></job> 

<bean id= "Retrypolicy" class= "Org.springframework?. Batch.retry.policy.ExceptionClassifierRetryPolicy "><property name=" PolicyMap "><map><entry key= "Org.springframework.dao.ConcurrencyFailureException" ><bean class= "Org.springframework.batch.retry?" Policy. Simpleretrypolicy "><property name=" maxattempts "value=" 3 "/></bean></entry><entry key=" Org.springframework.dao?. Deadlockloserdataaccessexception "><bean class=" org.springframework.batch.retry? policy. Simpleretrypolicy "><property name=" maxattempts "value=" 5 "/></bean></entry></map></ Property></bean>

You can also add listener

The configuration method is similar to the same skip, inherited Retrylistenersupport, configured in Retry-listeners

You can also retry by Retrytemplate. Retrytemplate Configuration Properties Retrypolicy

Third, restart

The focus of the restart is to be able to save the previous job repository, with the reader, can be written by their own reader needs to inherit the interface

Spring batch will be restarted by default.

Allow-start-if-complete configuration on Tasklet, determines whether the Tasklet will retry the step when the job is retried (perhaps the next step has an exception).

The start-limit is used to control the retry count at the step level, and the job interrupts exit after the retry count is over.

deal with the status in step, have processed a lot of item, failed? Middle of a chunk-oriented Step,item level restart. The first is the restart of reader. If you need to write your own reader of the item, you can also implement a restart, you need to implement the Itemstram interface, the value of the item count, save to ExecutionContext, restart the time to read.

How Spring Batch is robust for applications that can restart traceable Skip/retry/restart policies