Datastage Tips 2: Use crontab + shell to schedule datastage jobs regularly

In the process of using datastage to extract data, it is often required to extract data in Quasi Real Time (such as synchronizing data every 3 minutes. These requirements can be implemented using crontab + shell. As shown in, this is the entire shell scheduling DS job process: first, use the jobinfo parameter of dsjob to obtain the process ID (process_id) and state value (State) of the job ). when the State is abnormal, reset the job. When the State is normal, determine whether the job is running (that is, whether process_id is 0 ). If process_id = 0, it indicates that the job is not currently running and the job is scheduled normally. If process_id> 0, it indicates that the job is running and the end of the transpose operation is completed.

The Code is as follows:

#!/bin/sh. /home/dsadm/.bash_profileAPT_CONFIG_FILE=/opt/IBM/InformationServer/Server/Configurations/default.aptPROJECT_NAME=$1JOB_NAME=$2#job stat flagstatus=1job_process_id=1cd $DSHOME/bin#check job status  job_process_id=`./dsjob -jobinfo "$PROJECT_NAME" "$JOB_NAME"|sed -n '10,1p'|sed 's/\(.*\)\([:]\)\([ ]\{1,\}\)\([0-9]\{1,\}\)/\4/g'`  ./dsjob -jobinfo $PROJECT_NAME $JOB_NAME  echo "job_process_id=dsjob -jobinfo $PROJECT_NAME $JOB_NAME"  status=`./dsjob -jobinfo $PROJECT_NAME $JOB_NAME|sed -n '1,1p'|sed 's/\(.*\)\([:]\)\([ ]\{1,\}\)\(.*\)/\4/g'|grep FAILED|wc -l`  echo "status=dsjob -jobinfo $PROJECT_NAME $JOB_NAME"if [ $status -ne 0 ]; then   ./dsjob -run -mode RESET $PROJECT_NAME $JOB_NAME   echo "Invoke: dsjob -run -mode RESET $PROJECT_NAME $JOB_NAME" else  if [ "$job_process_id" -eq 0 ]; then        ./dsjob -run -param \$APT_CONFIG_FILE="$APT_CONFIG_FILE" $PROJECT_NAME $JOB_NAME  fifiecho "Finished"

Then, use crontab to schedule the above script.