DataStage get the CSV file and insert the database

DataStageOperation Flow

Document Creator: Moonlight Maple

Date:20180112

version: V1.0

1, processing files

The source file of this project originates from 10.126.124.39 server, andthe server does not turn on FTP function, but the directory sharing function is turned on .

1.1, usingBatThe batch script copies the source data to the local server first10.126.124.14in1.2, merging multiple files at the same time to facilitate subsequentDataStagecall, improve efficiency, or you need to run multiple timesDataStageinJob

2, creatingJob, prepare the globalJobParameters

Create Job, Configure the job parameters, mainly include: Linked server parameters, link database parameters, in the data conversion process required by the system parameters

3, creatingFtpSource Component3.1, inJobinterface,PaletteBar, get aProcessRelated ComponentsFTP,SetPropertiesParameters

Note:The Connect parameter is the FTP login password for the server where the source file resides , and it needs to be set by the network administrator.

3.2,Formatparameter Settings

3.3,columnparameter settings, select the one you createdtable Difination,Automatic Introduction, in4.1there are specific steps to create

3.4, configure parameters, and check configuration issues.

in the 3.1 diagram, there is a view datathat, if configured successfully, can be read from here.

Note: If Chinese characters are garbled, change the character set below

Click the button below to change

Select NLS, changing the character set

4, creatingNettzaDatabase Target Component4.1, you need to create aNettzadatabase corresponds to thetable Difinitioncomponents, which are stored under the corresponding directory and used for subsequent use.

To create a step: on the Main page

import-à table difinition-à Start Connector Import Wizard-à Nettza Connector

-à input account password address and other related information -à Select the desired table, or the table has been created

4.2, creating a database Target component

in the job interface,palette bar, get a database select Nettza connector, configure related parameters

Configuring The column parameter is the same as the 3.3 method, which is not repeated.

5, creatingTransformerConversion Rules

Create transformer, in the job interface,palette bar, get two link arrows to connect three components according to the data twist direction.

setting parameters for transformer

1, need to click button 1 for Component column 1 to 1 Association

2, if you need to process the data of the target table to select 2 places to choose, you can change the data type, the target data conversion and other operations

6, Test

save files, compile, run, input parameters to run, test according to the log, troubleshoot bugs

7. Making Windows timed Tasks

1. Enter the scheduled Task Manager

Start, Control Panel, System and security, management tools, Task Scheduler, open after

The interface is as follows

Right-click Task Scheduler, create task, enter task name and description

Finally need to enter the Administrator account password

2. Change the Scheduled task

Double-click the properties of the point-open task, and note that the property is changed to run regardless of whether it is logged in or not, and runs with the highest privileges.

DataStage get the CSV file and insert the database