ETL Tool kettle Practical Application Analysis Series 3 [ETL background process execution configuration method]

The main indexes of this series of articles are as follows:

I. ETL Tool kettle Application Analysis Series I [Kettle Introduction]

Ii. ETL Tool kettle Practical Application Analysis Series 2 [application scenarios and demo downloads]

Iii. ETL Tool kettle Practical Application Analysis Series III [ETL background process execution configuration method]

The main reading directory of this article is as follows:

1. Introduction to kettle's kitchen and Span

2. Command Line scheduling task Configuration

3. Configure the running mode of background processes

4. Windows Task settings

5. Download demo

 

1. Introduction to kettle's kitchen and Span

In the previous two articles, we mainly talked about the spoon conversion of kettle and the GUI Design and running of the job, as well as the demo practice, in fact, our application mode may require that the ETL task be run in the form of processes on the server, just as we traditionally process data using Windows Services, so how can we implement kettle? This requires us to understand kettle's kitchen and span.

Kitchen -- job Executor (command line)
Span -- trasform Executor (command line mode)

Here we will focus on kitchen. batOperatorThis is frequently used.

It is a job execution engine used to execute jobs. This is a command line execution tool. Its parameters are described as follows:

-Rep: name of the storage where the repository name task package is located
-User: Repository username executor
-Pass: password of the Repository password executor
-Job: the name of the job to launch task package name
-Dir: the directory (don't forget the leading/or \)
-File: the filename (job XML) to launch
-Level: the logging level (basic, detailed, debug, rowlevel, error, nothing) specifies the log level.
-Log: the logging file to write to specify the log file
-Listdir: list the directories in the repository list the directory structure in the specified storage.
-Listjobs: list the jobs in the specified directory to list all tasks in the specified directory
-Listrep: list the defined repositories list all storage
-Norep: Don't log into the repository. Do not write logs.

2. Command Line scheduling task Configuration

Let's take the first example I mentioned in the previous article to talk about the specific configuration method. The main idea is to first locate kitchen in kettle. bat root directory, and then execute the kitchen command;

Step 1: first create a BAT file, and then click Edit to enter, as shown in the following figure. This is the directory of kitchen. bat;

E:cd E:\ETL\pdi-open-3.1.0-826

 

Step 2: run the kitchen command as follows:

Kitchen/file: e: \ ETL \ script file \ mytest. kjb/level: Basic> D: \ mytest. Log

 

Note that the correct path is configured, followed by a basic log mode, and the log output path needs to be configured later;

3. Configure the running mode of background processes

We have already created the BAT file. At this time, we can run our ETL task in bat mode, but the problem is that there will always be a black cmd screen on the server desktop, how can I make bat disappear on the screen and the background can still run? There are many methods:

Method 1

Bat has the command to hide the window, which is very simple. You only need to add a piece of code in the Code header.

@ Echo off

If "% 1" = "H" Goto begin

MSHTA VBScript: Createobject ("wscript. Shell"). Run ("% ~ Nx0 H ", 0) (window. Close) & Exit

: Begin

: The following is your own code.

Method 2

Create a notepad under the same directory of your batch processing, and enter:

Dim objshell set objshell = wscript. Createobject ("wscript. Shell ")

Ireturn = objshell. Run ("cmd.exe/c: \ 1.bat", 0, true)

(Here, win. bat is your batch name and you can change it yourself)

Save this notepad as a file with the suffix. VBE. Then you just need to run this VBE file to achieve your goal!

Simple points to write:

Set Ws = Createobject ("wscript. Shell ")

WS. Run "CMD/c: \ 1.bat", 0

Or

Createobject ("wscript. Shell"). Run "CMD/c: \ 1.bat", 0

(This method is used to kill soft virus reporting. The parameter 0 must be replaced)

The last 0 refers to the window parameter. Its usage is as follows:

0. Hide the window and activate another window.

1. Activate and display a window. If the window is minimized or maximized, It is restored to its original size and position.

2. Activate the window and minimize the display of the window.

3. Activate the window and maximize the display.

4. display by the nearest size and position of the window. The activity window maintains the activity.

5. Activate and display the window with the current size and position.

6. Minimize the specified window and activate the next top-level window in Z-order.

7. Minimize the display window. The activity window maintains the activity.

8. The window is displayed in the current status. The activity window maintains the activity.

9. Activate and display the window. If the window is minimized or maximized, It is restored to the original size and position. Specify this flag when restoring the minimal window of the application.

4. Windows Task settings

After the previous two steps, we can make the ETL job run silently on the server. But how can we schedule it freely? We can combine the Windows task plan. I will not go into details here!

5. Download demo

ETL script. bat

If you have read this series of articles and feel useful, click [recommended] haha!