datastage常用操作

查看job状态
dsjob -report ODS_ETL RENWANG_T_BUSINESS_DEPARTMENT DETAIL

重置job状态，从aborted（不可调用）转为Reset（可调用）
dsjob -run -mode RESET ODS_ETL RENWANG_T_BUSINESS_DEPARTMENT

调用job，直到job结束命令才返回
dsjob -run -mode NORMAL -warn 0 -jobstatus ODS_ETL RENWANG_T_BUSINESS_DEPARTMENT

查看job最近100条日志
dsjob -logsum -max 100 ODS_ETL RENWANG_T_BUSINESS_DEPARTMENT


dsjob解释一
ETL调度中不乏要用shell脚本来运行DATASTAGE中的JOB,现测试一下DSJOB的用法然后总结一下（边测试边总结）
dsjob -run [-mode ] PROJECT_NAME JOB_NAME
-mode为选择运行模式，
NORMAL表示正常运行，此时和不写-mode是一样的；
RESET表示重置，当job在中止状态时要先RESET，之后才能再正常运行；
VALIDATE表示验证，即验证job的可运行性。
例子：dsjob -run -mode NORMAL sn_dev test_job
注：新开发的job一定要先编译，否则，三种mode都不可行。

dsjob -run [-mode ] -param = -jobstatus PROJECT_NAME JOB_NAME
-param运行带参数的job
name：参数名字
value：参数值
例子：dsjob -run -param YR_MON_DAY='20111020' sn_dev test_job
-jobstatus运行结束展示job的状态
-warn表示job中有多少条警告时中止job，设置为0表示无限制
例子：dsjob -run -mode NORMAL -warn 0 -jobstatus sn_dev test_ldf_2
正在等待作业...
已完成对作业的等待
作业状态 ：（2）
状态码为 2

dsjob -stop PROJECT_NAME JOB_NAME
-stop 停止正在运行的job
例子：dsjob -stop sn_dev test_job


列出所有的项目
dsjob -lprojects
状态码为 0

列出项目ODS_ETL中所有的job名字
dsjob -ljobs ODS_ETL
状态码为 0

列出指定job的详细信息
dsjob -jobinfo ODS_ETL ZZSR_YB_ZX_PLY
Job Status : RUN OK (1)
Job Controller : SEQ_CXYJ_ZZSR
Job Start Time : Mon Feb 29 02:00:34 2016
Job Wave Number : 109
User Status : not available
Job Control : 0
Interim Status : NOT RUNNING (99)
Invocation ID : not available
Last Run Time : Mon Feb 29 02:00:38 2016
Job Process ID : 0
Invocation List : ZZSR_YB_ZX_PLY
Job Restartable : 0
Status code = 0

列出指定job所用的stage
dsjob -lstages sn_dev test_ldf_2
Sequential_File_10
Sequential_File_1
状态码为 0

dsjob解释二
Datastage 的job可以通过dsjob命令来调用job或者获得job的信息，以及运行的报告和日志。
安装DataStage 的目录下 例如C:/Ascential/DataStage/Engine/bin>
dsjob的的语法如下：
Command Syntax:
dsjob [-file | [-server ][-user ][-passwod ]]
[]
Valid primary command options are:
-run 运行job
-stop 停止运行中的job
-lprojects 列出server中所有的工程
-ljob

s 列出指定project中的所有的job
-linvocations列出指定job中的所有的invocation
-lstages 列出指定job中的所有的stage
-llinks 列出指定job中的所有的link
-projectinfo 列出指定project的信息
-jobinfo 列出指定job的信息
-stageinfo 列出指定stage的信息
-linkinfo 列出指定link的信息
-lparams 列出指定job的所有的parameter
-paraminfo 列出指定parameter的信息
-log 获得job的所有的日志
-logsum 获得job的所有的日志
-logdetail 获得job的所的详细日志
-lognewest
-report 获得job的所的详细日志
-jobid

-run
Invalid arguments: dsjob -run
[-mode ]
NORMAL 正常运行
RESET 重置
VALIDATE 验证job是否可行
[-param =]
设置变量值
[-warn ]
warn 警告多少条记录以后job停止运行
[-rows ]
rows 运行过多少记录以后job停止运行
[-wait]
wait 等待多长时间停止运行
[-opmetadata ]
[-disableprjhandler]
[-disablejobhandler]
[-jobstatus]
Jobstatus 运行的状态
[-userstatus]
[-useid]

1 dsjob -server 10.240.12.67 -user peace.zhao -password Yanzhang0717 -run -mode NORMAL sysup1_MPHASIS_1 Sequential
最简单的运行

2 C:/Ascential/DataStage/Engine/bin>dsjob -server 10.240.12.67 -user peace.zhao -password Yanzhang0717 -run -mode NORMAL -jobstatus sysup1_MPHASIS_1 Sequential
Waiting for job...
Finished waiting for job
Job Status : (1)
Status code = 1

dsjob -server 10.240.12.67 -user peace.zhao -password Yanzhang0717 -run -mode NORMAL -param input=7.txt -param output=8.txt -jobstatus sysup1_MPHASIS_1 Sequential
Waiting for job...
Finished waiting for job
Job Status : (1)
Status code = 1


停止正在运行的job
-stop
Invalid arguments: dsjob -stop [-useid]


列出所有的server中的projuec
dsjob -lprojects
sysup1_MPHASIS
sysup1_MPHASIS_1
sysup1_MPHASIS_2
sysup1_MPHASIS_3
Status code = 0


获得project的信息
C:/Ascential/DataStage/Engine/bin>dsjob -projectinfo sysup1_MPHASIS_1
Host Name : 10.240.12.67
Project Name : sysup1_MPHASIS_1
Status code = 0

获取所有jobs
dsjob -ljobs ODS_ETL

获取job信息
dsjob -jobinfo sysup1_MPHASIS_1 Sequential
Job Status : RUN OK (1)
Job Controller : not available
Job Start Time : Mon Jul 06 15:04:33 2009
Job Wave Number : 3
User Status : not available
Job Control : 0
Interim Status : NOT RUNNING (99)
Invocation ID : not av

ailable
Last Run Time : Mon Jul 06 15:04:42 2009
Job Process ID : 0
Invocation List : Sequential
Status code = 0


获得job运行的报告
dsjob -report [-useid] [report type>]
report type = BASIC | DETAIL | XML
dsjob -report ODS_ETL ZZSR_YB_ZX_PLY DETAIL
**************************************************
STATUS REPORT FOR JOB: ZZSR_YB_ZX_PLY
Generated: 2016-03-09 14:48:42
Job start time=2016-03-09 02:00:50
Job end time=2016-03-09 02:00:54
Job elapsed time=00:00:04
Job status=1 (Finished OK)
Stage: YB_CUSTOMERADDDETAIL, 0 rows input
Stage start time=2016-03-09 02:00:51, end time=2016-03-09 02:00:54, elapsed=00:00:03
Link: DSLink3, 0 rows
Stage: Copy_1, 0 rows input
Stage start time=2016-03-09 02:00:51, end time=2016-03-09 02:00:54, elapsed=00:00:03
Link: DSLink3, 0 rows
Link: DSLink4, 0 rows
Stage: YB_ZX_PLY, 0 rows input
Stage start time=2016-03-09 02:00:53, end time=2016-03-09 02:00:54, elapsed=00:00:01

Status code = 0
Link: DSLink4, 0 rows-sh-4.1$



查询日志
dsjob -logsum
[-type ]
[-max ] ：获得最近的 条记录
[-useid]


获得运行时job的信息
dsjob -logsum -type INFO -max 10 ODS_ETL ZZSR_YB_ZX_PLY
2079 INFO Mon Feb 29 02:00:34 2016
main_program: orchgeneral: loaded (...)
2080 INFO Mon Feb 29 02:00:36 2016
main_program: IS9.1 Oracle EE Stage (last fix: 114774) built on 2012-11-07 for linux-x86. (...)
2081 INFO Mon Feb 29 02:00:36 2016
main_program: APT configuration file: /opt/IBM/InformationServer/Server/Configurations/default.apt (...)
2082 INFO Mon Feb 29 02:00:36 2016
YB_ZX_PLY,2: Records inserted: 0 (...)
2083 INFO Mon Feb 29 02:00:36 2016
YB_ZX_PLY,0: Records inserted: 0 (...)
2084 INFO Mon Feb 29 02:00:38 2016
YB_ZX_PLY,1: Records inserted: 0 (...)
2085 INFO Mon Feb 29 02:00:38 2016
YB_ZX_PLY,3: Records inserted: 0 (...)
2086 INFO Mon Feb 29 02:00:38 2016
main_program: Step execution finished with status = OK.
2087 INFO Mon Feb 29 02:00:38 2016
main_program: Startup time, 0:02; production run time, 0:01.
2088 INFO Mon Feb 29 02:00:38 2016
Parallel job reports successful completion
Status code = 0

dsjob -logdetail ODS_ETL ZZSR_YB_ZX_PLY

显示job的最新id
dsjob -lognewest ODS_ETL ZZSR_YB_ZX_PLY
Newest id = 2261
Status code = 0


shell调用dsjob的注意事项
在现实项目中，有时我们不会在Designer Client或者Director Client直接运行datastage job, 而是通过shell调用，下面简单介绍一下可能遇到的问题及解决办法。


本机调用命令（命令中的参数：SourceFile是区分大小写的，需要多加注意）
./dsjob -run -mode NORMAL -param SourceFile=DAD_D

DDC_Multi-Date_Oct_2013_v1.csv ANALYZERPROJECT Untitled1


远程调用命令（下面的ip地址也可以使用[server name]代替）
dsjob -server sn130127 -user stmttest -password 123456 -run -mode NORMAL -param SourceFile=DAD_DDDC_Multi-Date_Oct_2013_v1.csv ANALYZERPROJECT Untitled1

如果要列出其他服务器上的工程：
dsjob -server sn130127 -user username -password password -lprojects
此时-server后面用IP的话未成功，报错：Status code = 80011，求高手解答


dsjob -server DSPRD1 -user wangjun -password wangjun -run -mode RESET -wait ODS_ETL DianShang_DZSW_WEHICLE_DICTIONARY

dsjob -server DSPRD1 -user wangjun -password wangjun -lprojects



Datastage作业导入导出命令
1、导出
例子如下：
$DSHOME/../../Clients/istools/cli/istool export -dom dpapp01 -u edpusr -p edpusr -ar /home/dsadm/LOD_COBS_SVRHA1.isx -ds '-incexec "dpapp01/BRCB_EDP2/Jobs/FS2/COBS/LOD_COBS_SVRHA.pjb"'
详细说明：
【$DSHOME/../../Clients/istools/cli/istool】：istool命令行程序
【export】:指定istool执行导出动作
【-dom】：连接服务器，可使用hostname或者详细IP，例如dpapp01或者193.168.1.1:9080
【-u】:用户
【-p】：密码
【-ar】：导出文件存放全路径（注意需要有写权限）
【-ds】：导出的内容，详细如下：
【-incexec】：指定导出的作业包括可执行文件
【"dpapp01/BRCB_EDP2/Jobs/FS2/COBS/LOD_COBS_SVRHA.pjb"】里各项说明：
dpapp01为hostname
BRCB_EDP2：实际工程名
Jobs/FS2/COBS/：作业所在路径
LOD_COBS_SVRHA.pjb：作业名加后缀，其中并行作业为pjb，服务器作业未sjb

2、导入：
例子如下：
$DSHOME/../../Clients/istools/cli/istool import -dom dpapp01 -u edpusr -p edpusr -ar /home/dsadm/LOD_COBS_SVRHA.isx -ds '"dpapp01/BRCB_EDP"'
详细说明：
【$DSHOME/../../Clients/istools/cli/istool】：istool命令行程序
【import】:指定istool执行导入动作
【-dom】：连接服务器，可使用hostname或者详细IP，例如dpapp01或者192.168.1.1:9080
【-u】:用户
【-p】：密码
【-ar】：导出文件存放全路径（注意需要有写权限）
【-ds】：导入的内容，填写为 hostname/工程名
【-replace】：若要覆盖已有作业，需带上此参数。执行前需备份已有作业。


另外关于uv的命令，在Linux环境下启动或关闭Datastage时经常用到，在这里一并介绍：

uv命令的位置$DSHOME/bin/uv

启动datastage命令：$DSHOME/bin/uv -admin -start

关闭datastage命令：$DSHOME/bin/uv -admin -stop

其他命令选项如下：

$ $DSHOME/bin/uv -admin

uv [-admin option]

-autostart [on|off] enable/disable autostart feature

-clearshm clear BASIC catalog bit.

-freeLOGSEM release LOGSEM semaphore.

-impersonation [on|off] enable/disable impersonation mode

-info implementation information

-lic_report display licensing information.

-regen {opts} generate configuration file.

导入dsx工程文件
dsimport.exe /H=hostname /U=username /P=password /O=omitflag /NUA project|/ALL|/ASK dsx_pathname1 dsx_pathname2 ...
dscmdimport /H=hostname /U=username /P=password /O=omitflag /NUA project|/ALL|/ASK pathname1 pathname2... /V
此处多了一个/V：
V. Use this flag to switch the verbose option on.
For example, the following command imports the components in the
file jobs.dsx into the project dstage1 on the R101 server:
dscmdimport /H=R101 /O=1 dstage1 C:/scratch/jobs.dsx
Messages from the import are sent to the console by default, but can
be redirected to a file using '>', for example:
dscmdimport /H=R101 /O=1 /NUA dstage99 c:/scratch/project99.dsx /V > c:/
scratch/importlog
You can simply type dscmdimport at the command prompt to get help
on the command options.

导出dsx工程文件
dsexport.exe /H hostname /U username /P password /O omitflag project pathname1
dscmdexport /H hostname /U username /P password /O omitflag project pathname /V
此处多了一个/V：
V. Use this flag to switch the verbose option on.
For example, the following command exports the project dstage2
from the R101 to the file dstage2.dsx:
dscmdexport /H R101 /O 1 dstage2 C:/scratch/dstage2.dsx
Messages from the export are sent to the console by default, but can
be redirected to a file using '>', for example:
dscmdexport /H R101 /O 1 dstage99 c:/scratch/project99.dsx /V > c:/
scratch/exportlog
You can simply type dscmdexport at the command prompt to get
help on the command options.

编译指定名称的JOB
dscc /h ip /u dsadm /p dsadm prjname /j jobname

Usage: dscc [@argfile] [/?] /h [/o] [/u ] [/p ]
[/j ] [/r ] [/bo ] [/f] [/ouc] [/rd ] [/rt ]
[/jt ] [/mful ] [/mfcgb ]

@argfile Read arguments from a file.
/? Show usage.
/h Name of Host to attach to
/o Omit logon details
/u User Name
/p Password
project Project To Attach to
/j Job to compile, * = All Jobs, cat\* = category
/r Routine to compile, * = All Routines, cat\* = category
/bo Build Op to compile, * = All Build Ops, cat\* = category
/f Use the Force Compile
/ouc Only Compile Uncompiled Objects
/rd Report Directory and Name
/rt Type of report to produce : X = xml, T = txt - default
/jt Job Type To Compile
-1 (Default) = All Types
0 = Server
1 = Mainframe
2 = Sequence
3 = Parallel
/mful Mainframe Job Upload Profile
/mfcgb Mainframe Code Gen base directory location


显示project列表
dsjob -server ip -user dsadm -password dsadm -lprojects

显示project的job列表
dsjob -server ip -user dsadm -password dsadm -ljobs prjname

编译全部JOB
dscc /h ip /u dsadm /p dsadm prjname /j *

运行job

，默认设置
dsjob -run prjname jobname

运行job，传递参数
dsjob -run -param /$date=20091128 prjname jobname

运行job，传递多个参数
dsjob -run -param /$srcuser=test -param /$srcpwd=test prjname jobname

运行job sequence，多个参数
dsjob -run -param srcuser=test -param srcpwd=test prjname jobname

运行job，不等待job运行完成
dsjob -server ip -user dsadm -password dsadm -run prjname jobname

运行job，等待job运行完成
dsjob -server ip -user dsadm -password dsadm -run -wait prjname jobname



$ dsadmin

Command Syntax:

dsadmin [-file | [-server ][-user ][-password ]]

[]

Valid primary command options are:

-createproject

-deleteproject

-oshvisible

-enablercp

-enablejobadmin

-envadd

-envdelete

-envset

-advancedruntime

-basedirectory

-deploymentdirectory

-customdeployment

-listprojects

-listproperties

-listenv

-enablegeneratexml