Spring Batch (i): Basic section

Spring Batch (i): Basic section Blog Category:

• Spring
• Java

Spring Batch

Website:

Http://www.springsource.org/spring-batch

Download page:

Http://static.springsource.org/spring-batch/downloads.html

Document:

Http://static.springsource.org/spring-batch/reference/index.html

database table creation Connection: DDL

Http://static.springsource.org/spring-batch/reference/html/metaDataSchema.html#exampleDDLScripts

Chapter 1, Introducing Spring Batch

Spring batch requires a relational database to run, and in its official documentation, in the appendix, there is a database table structure that creates its dependencies.

The goal of the spring batch design:

1. Big Data processing

2, automation of

3. Robust

4. Trustworthy

5, high-performance

The size of the commit-interval= "100" in the chunk in the configuration file is the size of the chunk block. They are one thing.

Chunk inside there is Itemreader itemprocessor Itemwriter, Itemreader read one record at a time, itemprocessor processing a record once, Itemwriter one input commit-interval= "100" specifies the number of records (that is, the size of the chunk block) of the object size.

The recommended size of the Commit-interval is 10 to 200. If this value is too large, if it is output data to the database, it will cause the database of things too big data, affecting the performance of the database, the database will record a lot of rollback data content, resulting in poor performance. This value is too small and causes many transactions to occur, resulting in slow spring batch operation.

Parameters that can be run using the spel dynamic configuration program, for example:

<property name= "Resource" value= "file:#{jobparameters[' Inputresource ']}"/>

One thing to note here is that the bean's scope must be "step". Spel expressions are supported in spring 3.x.

<bean id= "Reader" class= "Org.springframework.batch.item.file.FlatFileItemReader" scope= "Step" >

The following is the metadata table for spring batch, the DDL for the Oracle database

SQL code

2. CREATE TABLE batch_job_instance (
4. job_instance_id number (19,0) not NULL PRIMARY KEY,
6. VERSION number (19,0),
8. Job_name VARCHAR2 (+) not NULL,
10. Job_key VARCHAR2 (+) not NULL,
12. constraint Job_inst_un unique (job_name, job_key)
14. ) ;
18. CREATE TABLE batch_job_execution (
20. job_execution_id number (19,0) not NULL PRIMARY KEY,
22. VERSION number (19,0),
24. job_instance_id number (19,0) is not NULL,
26. Create_time TIMESTAMP not NULL,
28. Start_time TIMESTAMP DEFAULT NULL,
30. End_time TIMESTAMP DEFAULT NULL,
32. STATUS VARCHAR2 (10),
34. Exit_code VARCHAR2 (100),
36. Exit_message VARCHAR2 (2500),
38. last_updated TIMESTAMP,
40. constraint JOB_INST_EXEC_FK foreign Key (job_instance_id)
42. references Batch_job_instance (job_instance_id)
44. ) ;
48. CREATE TABLE Batch_job_params (
50. job_instance_id number (19,0) is not NULL,
52. TYPE_CD VARCHAR2 (6) not NULL,
54. Key_name VARCHAR2 (+) not NULL,
56. String_val VARCHAR2 (250),
58. Date_val TIMESTAMP DEFAULT NULL,
60. Long_val number (19,0),
62. Double_val number,
64. constraint JOB_INST_PARAMS_FK foreign Key (job_instance_id)
66. references Batch_job_instance (job_instance_id)
68. ) ;
72. CREATE TABLE batch_step_execution (
74. step_execution_id number (19,0) not NULL PRIMARY KEY,
76. VERSION number (19,0) is not NULL,
78. Step_name VARCHAR2 (+) not NULL,
80. job_execution_id number (19,0) is not NULL,
82. Start_time TIMESTAMP not NULL,
84. End_time TIMESTAMP DEFAULT NULL,
86. STATUS VARCHAR2 (10),
88. Commit_count number (19,0),
90. Read_count number (19,0),
92. Filter_count number (19,0),
94. Write_count number (19,0),
96. Read_skip_count number (19,0),
98. Write_skip_count number (19,0),
100. Process_skip_count number (19,0),
102. Rollback_count number (19,0),
104. Exit_code VARCHAR2 (100),
106. Exit_message VARCHAR2 (2500),
108. last_updated TIMESTAMP,
110. constraint JOB_EXEC_STEP_FK foreign Key (job_execution_id)
112. references Batch_job_execution (job_execution_id)
114. ) ;
118. CREATE TABLE Batch_step_execution_context (
120. step_execution_id number (19,0) not NULL PRIMARY KEY,
122. Short_context VARCHAR2 (2500) not NULL,
124. Serialized_context CLOB,
126. constraint STEP_EXEC_CTX_FK foreign Key (step_execution_id)
128. references Batch_step_execution (step_execution_id)
130. ) ;
134. CREATE TABLE Batch_job_execution_context (
136. job_execution_id number (19,0) not NULL PRIMARY KEY,
138. Short_context VARCHAR2 (2500) not NULL,
140. Serialized_context CLOB,
142. constraint JOB_EXEC_CTX_FK foreign Key (job_execution_id)
144. references Batch_job_execution (job_execution_id)
146. ) ;
150. CREATE SEQUENCE batch_step_execution_seq START with 0 MINVALUE 0 MAXVALUE 9223372036854775807 nocycle;
152. CREATE SEQUENCE batch_job_execution_seq START with 0 MINVALUE 0 MAXVALUE 9223372036854775807 nocycle;
154. CREATE SEQUENCE batch_job_seq START with 0 MINVALUE 0 MAXVALUE 9223372036854775807 nocycle;

Notes SQL scripts are available in the core Jar package of Spring batch. For example: Spring-batch-core-2.1.9.release.jar's core package Org.springframework.batch.core, Schema-drop-[database].sql In the form of SQL scripts that provide a variety of mainstream relational databases.

The databases supported by spring batch are: Derby, H2, Hsqldb,mysql, Oracle, PostgreSQL, SQL Server, and Sybase.

Chapter 2, Spring Batch Concepts

1. main components of Spring batch:

Job repository The underlying component of the persistence job execution metadata

Job launcher The underlying component of the boot job execution

Application Components for Jobbatch processing

A word within the stepjob, a job consisting of a series of step
Tasklet, repeatable processing steps for applying transactions in step

Item a record entered or output from the data source

Chunk a list of item sizes of the specified size.

Item Reader is responsible for reading the component of the item record from the data source

Item processor is responsible for handling (conversion, validation, filtering, etc.) Item record components

Item writer is responsible for outputting a chunk item to the data source: the component that outputs the list of specified size items to the data source

2. How the spring batch interacts with the external

External Cron,quartz, such as scheduling, or the Web, can call launch, triggering the operation of spring batch.

3, the basic components of Spring batch include: Job launch and Job repository. They do not require developer development, they need to be configured in configuration files only.

The job of Spring batch is a series of step components in the spring batch XML.

4, the use of Tasklet

Create a step, including writing a tasklet execution or using a tasklet provided by a spring batch.

When you need to unzip a file, call a stored procedure, or delete a file, call the SH script, you need to create your own Tasklet implementation.

When implementing Tasklet, these requirements can be divided into batch read-process-write mode, then the chunk element should be used to configure Tasklet as chunk processing processing steps. The chunk element allows your program to read, process, and write data more efficiently.

Summary: Job is a series of steps (step) that you can easily define in spring batch XML. The steps (step) are composed of Tasklet. Tasklet can be made up of "module-oriented" Chunk, or Tasklet is completely customizable by developers.

5. Job Related Concepts

A, job work

b, job instance work example

c, job execution execution

A job can have many work instances (job instance), and each work instance (job instance) can have a lot of work execution (job execution).

In spring batch, the work instance (job instance) includes work (job) and work parameters (job parameter).

Java code

1. Joblauncher.run (Job, new Jobparametersbuilder ()
2. . addstring ("date", "2010-06-27")
3. . Tojobparameters ()
4. );

Work Example equation:

Jobinstance = Job + job-parameters.

Some rules for the life cycle of job instance and job execution:

A. When you run the job for the first time, spring batch creates the job instance and the first job execution.

b, you cannot run the job instance again after the same job instance was successfully executed.

C, you cannot execute multiple identical instances at the same time.

================ Basic Part End ============================

Spring Batch (i): Basic section