Top of Form

3072

My Home
Topics
People
Companies
Jobs
White Paper Library
Home
Blogs
Groups
Wiki
Communities
White Papers
Q&A and Docs
Directory
Events
Subscriptions
Toolbox for IT
Topics
Business Intelligence
Blogs

Tooling Around in the IBM InfoSphere

by Vincent McBurney IBM Information Champion

The blog dedicated to a tool based approach to data integration with news and tips on IBM
InfoSphere, Informatica, Oracle, ... more
The blog dedicated to a tool based approach to data integration with news and tips on IBM
InfoSphere, Informatica, Oracle, Microsoft and any breaking data integration news. less
Blog Main | Archive
Previous
|
Next
0

Please Sign In or Register to like this blog.

What is new for DataStage 8 on the

Information Server
Vincent McBurney | Feb 2, 2007 | Comments (68)

DataStage 8 on the Information Server looks the same as previous releases but has some major
changes under the hood and a few extra bells and whistles.
This post looks at what is new or changed in DataStage jobs. There are a lot of new functions for
managing, running and reporting on jobs but I will talk about that in another post or you can look
back at my (much) earlier DataStage Hawk preview post.
Goodbye DataStage 7
It's time to bid goodbye to tired old DataStage 7.

You did a good job, you struggled on for as long as you could, but like all DataStage versions
through the annuls of history you didn't have the right metadata repository and you didn't play
well with your brothers and sisters in your suite.
DataStage 8 on the other hand is much shinier and comes with a better metadata story as you get
the new Metadata Server and the common connectors:

Release Date
The Windows version of the Information Server and DataStage 8 are out now. No sign yet of the
version for other platforms.
DataStage Versions
DataStage 8 can only upgrade a DataStage 7 server, it cannot upgrade previous versions of
servers though it can co-exist with previous versions. DataStage 8 can however import and
upgrade export files from earlier versions of DataStage. I don't know how far back this support
goes.
All the DataStage 7.x versions are available in version 8:
• DataStage Enterprise Edition: Parallel, Server and Sequence Jobs
• DataStage Server Edition: Server and Sequence Jobs
• DataStage MVS: Mainframe Jobs
• DataStage Enterprise for z/OS: runs on Unix System Services
DataStage for PeopleSoft: 2 CPU limit with Server and Sequence jobs.
I don't know whether you will ever see this version of DataStage in the PeopleSoft EPM
bundle, however you may be able to upgrade existing PeopleSoft implementations to this
version. Drop me a message if you try.
DataStage Addons
The DataStage Enterprise Packs and Change Data Capture components are available in version 8
as shown in the version 8 architecture overview:
Enterprise PACKs
• SAP BW Pack
○ BAPI: (Staging Business API) loads from any source to BW.
○ OpenHub: extract data from BW.
• SAP R/3 Pack
○ ABAP: (Advanced Business Application Processing) auto generate ABAP,
Extraction Object Builder, SQL Builder, Load and execute ABAP from
DataStage, CPI-C Data Transfer, FTP Data Transfer, ABAP syntax check,
background execution of ABAP.
○ IDoc: create source system, IDoc listener for extract, receive IDocs, send IDocs.
○ BAPI: BAPI explorer, import export Tables Parameters Activation, call and
commit BAPI.
• Siebel Pack
○ EIM: (data integration manager) interface tables
○ Business Component: access business views via Siebel Java Data Bean
○ Direct Access: use a metadata browser to select data to extract
○ Hierarchy: for extracts from Siebel to SAP BW.
• Oracle Applications Pack
○ Oracle flex fields: extract using enhanced processing techniques.
○ Oracle reference data structures: simplified access using the Hierarchy Access
component.
○ Metadata browser and importer
• DataStage Pack for PeopleSoft Enterprise
○ Import business metadata via a metadata browser.
○ Extract data from PeopleSoft tables and trees.
• JD Edwards Pack
○ Standard ODBC calls
○ Pre-joined database tables via business views
Change Data Capture
These are add on products (at an additional fee) that attach themselves to source databases and
perform change data capture. Most source system database owners I've come across don't like
you playing with their production transactional database and will not let you near it with a ten
foot poll, but I guess there are exceptions:
• Oracle
• Microsoft SQL Server
• DB2 for z/OS
 • IMS
There are three ways to get incremental feeds on the Information Server: the CDC
products for DataStage, the Replication Server (renamed Information Integrator:
Replication Edition, does DB2 replication very well) and the change data capture functions
within DataStage jobs such as the parallel CDC stage.
Removed Functions
These are the functions that are not in DataStage 8, please imaging the last waltz playing in your
head as you peruse this list:
• dssearch command line function
• dsjob "-import"
• Version Control tool
• Released jobs
• Oracle 8i native database stages
• ClickPack
The loss of the Version Control tool is not a big deal as the import/export functions have
been improved. Building a release file as an export in version 8 is easier than building it in
the Version Control tool in version 7.
Database Connectivity
The common connection objects functionality means the very wide range of DataStage database
connections are now available across Information Server products.
Latest supported databases for version 8:
• DB2 8.1, 8.2 and 9.1
• Oracle 9i, 10i, 10gR2 not Oracle 8
• SQL Server 2005 plus stored procedures.
• Teradata v2r5.1, v2r6.0, v2r6.1 (DB server) / 8.1 (TTU) plus Teradata Parallel
Transport (TPT) and stored procedures and macro support, reject links for bulk
loads, restart capability for parallel bulk loads.
• Sybase ASE 15, Sybase IQ 11.5, 12.5, 12.7
• Informix 10 (IDS)
• SAS 612, 8.1, 9.1 and 9.1.3
• IBM WS MQ 6.1, WS MB 5.1
• Netezza v3.1
• ODBC 3.5 standard and level 3 compliant
• UniData 6 and UniVerse ?
• Red Brick ?
This is not the complete list. Some database versions are missing, more databases can be
accessed through the ODBC stage and there may be some databases missing.
New Database Connector Functions
This is a big area of improvement.
• LOB/BLOC/CLOB Data: pictures, documents etc of any size can now be moved
between databases. After choosing the LOB data type you can choose to pass the data
inline or as a link reference.
• Reject Links: optionally append error codes and messages, conditionally filter types of
rejection, fail a job based on a percentage threshold of failures.
• Schema Reconciliation: where the hell has this function been all my life? Automatically
compare your DataStage schema to the database schema and perform minor data type
conversions.
• Improved SQL Builder that supports more database types, although if you didn't like the
version 7 one you wont like the 8 one either. (Kim Duke, I'm looking at you).
• Test button on connectors. Test! You don't have to view data or run a job to find out if
the stupid thing works.
• Drag and drop your configured database connections onto jobs.
• Before and after SQL defined per job or per node with a failure handling option. Neater
than previous versions.
DataStage 8 gives you access to the latest versions of databases that DataStage 7 may never
get. Extra functions on all connectors includes improved reject handling, LOB support and
easier stage configuration.
Code Packs
These packs can be used by server and/or parallel jobs to interact with other coding languages.
This lets you access programming modules or functions within a job:
• Java Pack: produce or consume rows for DataStage Parallel or Server jobs. Use a java
transformer.
• Web Service Pack: access web services operations in a Server job transformer or Server
routine.
• XML Pack: read, write or transform XML files in parallel or server jobs.
The DataStage stages, custom stages, transformer functions and routines will usually be
faster at transforming data than these packs however they are useful for re-using existing
code.
New Stages
A new stage from the IBM software family, new stages from new partners and the convergence
of QualityStage functions into Datastage. Apart from the SCD stage these all come at an
additional cost.
• WebSphere Federation and Classic Federation
• Netezza Enterprise Stage
• SFTP Enterprise Stage
• iWay Enterprise Stage
• Slowly Changing Dimension: for type 1 and type 2 SCDs.
• Six QualityStage stages
There are four questions that have been asked since the dawn of time. What is the meaning
of life? What's this rash that comes and goes? If you leave me can I come too? How do a
populate a slowly changing dimension using DataStage? The answers being 42, visit a
clinic, piss off and use the new SCD stage.
New Functions Existing Stages
• Complex Flat File Stage: Multi Format File (MFF) in addition to existing cobol file
support.
• Surrogate Key Generator: now maintains the key source via integrated state file or
DBMS sequence.
• Lookup Stage: range lookups by defining checking high and low range fields on the
input or reference data table. Updatable in memory lookups.
• Transformer Stage: new surrogate key functions Initialize() and GetNextKey().
• Enterprise FTP Stage: now choose between ftp and sftp transfer.
You can achieve most of these functions in the current version with extra coding except for
in-memory lookups. This is a killer function in DataStage 8.
Platforms
These are the platforms for the released Windows version and the yet to be released Linux/Unix
version along with the C++ compiler that you only need for parallel jobs that will use
transformers. You do not need this compiler for Server Edition.
-Windows 2003 SP1
•Visual Studio .NET 2003 C++, Visual Studio .NET 2005 C++ or Visual Studio .NET 2005
Express Edition C++
-AIX 5.2 & 5.3
•XL C/C++ Enterprise Edition 7.0, 8.0 compiler
-HP-UX 11i v1 & v2
•aC++ A.03.63 compiler
-Red Hat ASE 4.0
•gcc3.23 compiler
-SuSEES, 9.0
•gcc3.3.3 compiler
-Solaris 2.9 & 2.10
•Sun Studio 9, 10 , 11 compiler
Database Repository
Note the database compatibility for the Metadata Server repository is the latest versions of the
three DBMS engines. DB2 is an optional extra in the bundle if you don't want to use an existing
database.
• IBM UDB DB2 ESE 9
-IBM Information Server does not support the Database Partitioning Feature (DPF) for
use in the repository layer
-DB2 Restricted Enterprise Edition 9 is included with IBM Information Server and is an
optional part of the installation however its use is restricted to hosting the IBM
Information Server repository layer and cannot be used for other applications
• Oracle 10g
 • SQL Server 2005
If you are a cheapskate and you really don't like DB2, in fact you would cross the street if
you saw it coming in the other direction, you might be able to load the repository into a free
(express) version of SQL Server or Oracle, however you might hit a problem with the
DBMS license CPU restriction. If you get this working drop me a comment.
Languages
Foreign language support for the graphical tools and product messages:
Chinese (Simplified and Traditional), Czech, Danish, Finnish, French, German, Italian, Japanese,
Korean, Norwegian, Polish, Portuguese, Russian, Spanish and Swedish.
Commiserations to the Welsh. For the Trekkies out there keep the writin campaign going,
it is only a matter of time before they add Klingon. It is on the product path right after
High Elf.
Disclaimer: The opinions expressed herein are my own personal opinions and do not represent
my employer's view in any way.
Related White Papers
• Oracle Exadata and Netezza TwinFin Compared

• Appliance Power: Crunching Data Warehousing Workloads Faster And Cheaper Than
Ever

• DMI Informix - Network IP Customer Reference

Jobs by
Senior Microsoft Business Intelligence Developer
Charlotte NC
Senior Developer/Analyst
Princeton NJ
SQL / ASP Developer - Junior to Mid Level
Malvern PA
Search More Jobs
68 Comments

Feb 2, 2007
Good info. Thanks a
lot.

Feb 2, 2007
I cant wait for the V8 to be out. I love the enhacements in the "New Database Connector
Functions". Thanks for the great info Vincent ;)

Feb 3, 2007
It is good Info .I like to know whether customers can have choice in datastage 8 to do customize
purchase( i mean instead of customer going for all the components in Datastage 8,is it possible
to purchase components according to his requirement)?

Zinna | Feb 5, 2007

sounds exciting. What's the timeframe for the solaris platform of datastage 8? I am reviewing
our server jobs and could really use some the new functions in existing stages!

Feb 6, 2007
Thanks Vincent for sharing good
information!

Kavita Dalvi | Feb 6, 2007

Would only like to know whether the "killer" transformer stage in parallel edition is
improvised with respect to performance?

Feb 6, 2007
Thanks for sharing the goos
info.

Vincent McBurney | Feb 6, 2007

Aneel, you can pick and choose packs (such as Siebel, CDC and PeopleSoft) but you cannot
pick and choose your parallel stages. Zinna the Linux/Unix version was supposed to be out in
November and I still haven't seen it. Perhaps some holdups in beta testing as the Windows
version was being used for many months in beta before the release date. There are almost no
new functions in server jobs but you do benefit from the new metadata functions. Kavita, a good
question. I had already blogged about how the transformer is a good option and performance
problems are overstated. In DataStage 8 for performance the job startup time is faster, buffering
is beter and more combinable stages. The transformer may be able to combine with more stages
in the new release (it current can combine with the lookup stage).

Feb 9, 2007
Realy a good information on DataStage 8. Helps me a lot Thank you very much
Burney

sasidhar_kari | Mar 30, 2007

Gr8 Work Vincent,

Thanks for the Info

us1aslam1us | Apr 2, 2007

Excellent Blog Vincent.

Thanks
Sam
 Rajender Reddy Marikanti | Apr 9, 2007
Hi Vincent,

Good work Keep it up.

Raj Marikanti

May 23, 2007

Great info on DS8. Thanks for all the
effort.

Srinivas

Jun 1, 2007
Hey Vincent, Info is great from development prospective, but how about scurity features from
administrators prospective I mean creating DS users & groups, do we have to do at OS level or
we can we create from DS8.

Thanks
Srimitta

Minhaj uddin | Jun 3, 2007

Another beautiful blog by our blog
man!
Great information Vincent.

Jun 15, 2007

Vincent, Such an Excellent Blog .Keep
going

Hari

gillisj1 | Jul 24, 2007

Will the job still lock if you loose your connection to the repository

Vincent McBurney | Jul 24, 2007

A job is locked when it is opened for editing, they have made it easier to manage these locks:
you can open a locked job in read only mode, you can see who has a lock on the job and
someone with the right privileges can release that lock through the client tools.

Rama Krishna | Aug 8, 2007

nice info .. I want to know whether we can import the idoc structure using datastage or do
we need to type all the columns .

Bob Averack | Sep 26, 2007

Hi Vincent,
 You spoke of a common connection object for SAS v9 and v9.1.3. Is this a rebranding of
the earlier DS Pack for SAS (that, I believe, reads and writes SAS Data Sets), or does this
perform a different function?

Thanks,
Bob

Vincent McBurney | Sep 26, 2007

It's the same DataStage SAS plugin component as before with support for SAS 9 added.

Oct 4, 2007
Hi everybody,

Can anybody tell me what the different between Datastage 8.0.1 for windows and Infomation
Server 8.0.1 for windows ? Do I need to install both softwares to do the ETL works ? Or I just
need to install one of them ?.

Thanks

Golden Orbit | Oct 4, 2007

Hi Dung,

Either one should be enough. Do you have license for both?

DataStage has more out-of-box transformations than PowerCenter.

Regards...

Vincent McBurney | Oct 5, 2007

Information Server is a suite - at a bare minimum it has the Metadata Server and one product
using it. So DataStage 8.0.1 or QualityStage 8.0.1 are both products that need the Metadata
Server to run so they are both considered part of the Information Server suite.

Nov 2, 2007
kindly send me the quotation for the data stage tools and informatica
tolls

padmakar

Vincent McBurney | Nov 2, 2007

There are list prices for DataStage that put a 4 CPU parallel configuration at about US$240K
and an Information Server Blade at US$350K (comes with hardware, operating system,
metadata server and DataStage on 4 dual core CPUs). But those prices would be likely to come
down during evaluation and negotiation.
Don't know the prices for Informatica. If both vendors were evaluated I'm sure the prices would
end up lower and similar. They have different pricing for addons so you really need to get a
quote from each vendor with a specific configuration request.

Wes Dumey | Nov 5, 2007

I priced out Informatica PowerCenter 7 for a client in 2005, and it was something like $100k for
the base toolset, plus a yearly maintenance fee, plus additional fees for more processors,
capabilities, etc. In the end, it was quite comparable to DataStage in cost.

Nov 16, 2007

Of course, any self respecting welsh person would hand craft their ETL code, or else use
OWB.

Jan 14, 2008

Hi Vincent,
This information is very useful and helpful for
me.
Thanks for your effort and Regard

Feb 4, 2008
Hi Vicent,
Thanks for this information, but I have a problem with DataStage 8 and Pak for SAP R/3
6....When I try to use an ABAP stage I get the error "Invalid User or Password", but I'm trying to
login with the same user and password that I use in my GUI interface for SAP

Windows 2003 Server SP2

DataStage 8
Pak for SAP R/3 6

I really don't know how to fix it...

000150 | CAA2413D 69BF0514 04020043 4E6F6D62 |..A=i......CNomb|
000160 | 7265206F 20636C61 76652064 65206163 |re o clave de ac|
000170 | 6365736F 20696E63 6F727265 63746F73 |ceso incorrectos|
000180 | 20287265 70697461 206C6120 656E7472 | (repita la entr|
000190 | 61646120 616C2073 69737465 6D612904 |ada al sistema).|
0001a0 | 02FFFF00 00FFFF00 00000000 00000000 |................|
>> RfcReceive [1] ...
Received RFCHEADER [1]: 01/LIT/IEEE/SPACE/1100
Received UNICODE-RFCHEADER [1]: cp:1100/ce:IGNORE/te:IGNORE/cs:1/rc:0x00000023
UUID: ab_rfccheck_uuid compare uuid's {4C6B9137-1CFE-4765-8514-CAA2413D69BF}
======> Nombre o clave de acceso incorrectos (repita la entrada al sistema)
>>>> [1] : EXT L 172.27.12.140 >>> CLOSE abrfcrcv.c 405 (48111827)
-{4C6B9137-1CFE-4765-8514-CAA2413D69BF}
*> RfcReceive
handle = 1
parameter =
tables =
Data conversion On
>>>RfcLastErrorEx
>>>> [1] : EXT L XXX.XX.XX.XXX >>> FREE abrfcio.c 3382 (48111827)
-{4C6B9137-1CFE-4765-8514-CAA2413D69BF}

regards.

Paddu | Feb 6, 2008

Wonderful Information :-)

mar | Mar 14, 2008

hi!

I'm using Datastage 8 on windows server 2003, but I not found the LOB data type.

SQL Type show LongVarChar and NLongVarChar only.

Rajesh Bonepalli | Mar 18, 2008

I am using Datastage. I want connectivity through java.can any body tell how to do?

Rajesh Bonepalli | Mar 18, 2008

Hi,
We are using DataStage.But we are not getting how to connect to datastage from java can u
tell me?

Vincent McBurney | Mar 18, 2008

You can submit your questions to the Ascential group at groups.ittoolbox.com or at
www.dsxchange.com. You can run DataStage jobs from Java by calling the dsjob command to
run jobs - details in the DataStage Server Job Developers Guide.

soumita | Apr 28, 2008

It is really wonderful information we received from this
blogg!

Apr 28, 2008

Hello,

How to use Datastage to convert Japanese character Katakana from (Single byte to Double
Byte). Input as the Sigle Byte character while output will be Double byte character.

regards,
SP

rajuyarra | Jun 4, 2008

exllent...
Can anybody provide me what configuration required in SAP R3 system to connect datastage
and The IDOC extract stage.

Thanks in advance

rajuyarra | Jun 4, 2008

what configuration required in SAP R3 system to connect Datastage to the IDOC Load Stage

Jul 7, 2008
Hello
I would like to know where can i get traing on Data Stage 8.Please provide me
info

Thanks

Vincent McBurney | Jul 8, 2008

You can get training from a scheduled IBM DataStage Essentials training course. Go to the
www.ibm.com website, switch to your home country page using the country selector in the top
right, find "Training" under the support tab in the box near the bottom. That should lead you to
the training section. Do a Training search for "DataStage".

Sep 2, 2008
Hi Vincent, like always ... "Job well done". Am looking at teaching datastage to a handful of
people. Any idea if there is an education license available?

Vincent McBurney | Sep 2, 2008

That is a tricky one. Unlike open source vendors or smaller ETL vendors IBM do not make a
trial or education version available for download. You will need to contact your local IBM
office to try and get a copy. Partners of IBM who purchase a PartnerWorld Value Pack are
entitled to use IBM software for education purposes but even after joining the PartnerWorld
download section is incredibly convoluted and the Information Server software is difficult to
find. You are better off trying to get it straight from IBM.

Sep 4, 2008
Hi Vincent, great blog. Has anyone managed to get the SCD stage working properly with type 1
changes? If the SCD stage includes both type 1 and type 2 columns, it misses key values or just
ignores type 1 changes. We have DataStage 8.0.1.

Regards

Saugat Dey | Sep 4, 2008

Vincent
Thanks for the info.
I can not see DataStage Admin in DS8.0 ?

what is the replacement for it?

Vincent McBurney | Sep 5, 2008
For some reason the DataStage Administrator gets installed but not added to the start menu. You
should find the Administrator icon on your desktop and you will find it in the Program Files -
Information Server directory but you will have to add it to your Start menu manually.

There are some security functions in the Information Server browser Console and some
DataStage Administration functions only available through the Administrator client tool.

Sudipta Chatterjee | Sep 5, 2008

Can anybody please tell me how to use the Java Client stage available in DataStage v8. I tried to
use it but the job is aborting. I can compile and run the same java program from unix prompt
successfully. But cannot call it from DS. The java program is a simple program which has no
input but will print "Java Rules".The program is given below:
---------
class Motto {
public static void main(String[] args) {
System.out.println("Java rules!");
}
}
---------
I am using the Java Client stage as a source in the DS job.
Not very sure of what I should specify in the Transformer Class Name, User's Class Path, and
other tabs available. Someone who has used the Java Client Stage do let me know what should
be specified in the required tabs.

Sep 11, 2008

Hi Vincet,

DataStage 8.0.1 looks for DB2 version 9.1, but I need to install DataStage with DB2 Enterprise
Edition version 9.5.
Is this feasible ? If so could you please give some guidance
for the same.
Any help will be appreciated.

Trisha.

glennstage | Sep 11, 2008

Hi,

We are looking at installing 8.1 when it comes out, we are told end of the summer by IBM, and
we are struggling with the best way to lay the the seperate layers out on Windows 2003 servers.
If anyone has any horror or success stories regarding this when they installed Information Server
8.0.1 that would be helpful. What I'm looking at so far is one server with the meta data layer and
websphere layer installed on it running two dual core processors. A second server I'd install the
engine layer with two quad core processors. Any thoughts? Is this a way to go? Am I
maximizing efficiency here?

Thanks,
Glenn

Vincent McBurney | Sep 11, 2008

@Tricia, try a 9.1 client library on the DataStage server connecting to a 9.5 database server.
Otherwise try to get the 9.5 client connecting but make sure in your dsenv LIBPATH variable
you point to the 32bit library and not the default 64bit library or it wont work.

@Glenn, what you propose sounds exactly like an Information Server Blade - see my post The
IBM Information Server is Software is Hardware is People!. It puts all the metadata on the first
blade, the DataStage head node on the second blade and a DataStage compute server on the third
blade. The metadata blade as a single dual core CPU and the DataStage blades have twin dual
core CPUs.

So on Windows it's a good architecture to separate DataStage from the Metadata Server. On
Unix/Linux where you have a massively scalable MPP it might be better to put everything on
the one server - the machine will handle the Metadata services without raising a sweat.

For more details on your layer architecture options have a look at the IBM Information Server
Information Center. They have a good section on it.

glennstage | Sep 12, 2008

Thanks Vincent but I think that's where we are getting confused with regards to the layers and
nodes. You mentioned metadata on one blade which I'm good with then you say the DataStage
head node ona second and the compute server on a third. That's where I'm lost. Is the websphere
layer considered the "head node" and the Engine layer considered the "compute server"?
Basically I was thinking(2 blades) Websphere and Meta data on one blade and the Engine on the
second blade. Then one of our network guys suggested for fail over we split the engine into two
seperate blades in case one blade was down the other would still be up. That's where it got more
confusing as we asked support about that and they didn't really give us a definite good or bad
idea response but rather started expalining the Conductor node talks to the section leaders which
spawn off players that actually do the work. From what you said when you said, "head node" are
you refering to the Conductor Node?

Thanks
Glenn

hari reddy | Nov 14, 2008

Hi Vincet,

DataStage 8.0.1 looks for DB2 version 9.1, but I need to install DataStage with DB2 Enterprise
Edition version 9.5.
Is this feasible ? If so could you please give some guidance
for the same.
; but i have some problems .ith my client
Dec 10, 2008
Hi Vincent

Could you please tell me whether Datstage 8.0.1 is compatible with db2 version 9.5. Appreciate
your quick response.

regards

Dec 11, 2008

Hi Vincent,

I am in the process of installing datastage 8.0.1 with db2 v9.5 and encountering problems in
creating the metadata repository.I have tried without the option of the repository also,but all
have gone in vein.Could you please guide me or tell me what i am doing wrong?

Waiting for your quick response.

Thanx in advance.,

Sambit

Piya Verma | Dec 13, 2008

Hi Vincent,

How can we move datastage jobs from development environment to testing environment
with a Datastage job using unix commands?

nilu | Jun 3, 2009

Hi Vincent,

This was really very good and informative blog.

Could you please also let me know whether it is possible to call native mainframe routine from
the Datastage parallel environment?
Appreciate your early response on this.

Thanks and Regards,

Nilu

Ritu Sethi | Jun 23, 2009

Hi Vincent,

How can we configure a two server datastage environment (using DS 8.1 on AIX 6.1) for
failover/disaster recovery. The purpose is that the secondary server should take over when the
primary server fails or goes down .What steps are needed to setup this configuration? Do we
need to do this while installing DataStage or do configurations after installation ?

Would appreciate any help / suggestions you could provide .

Thanks

Ritu

Vincent McBurney | Jun 23, 2009

You just need to make sure both environments have the same settings - DataStage parameters,
the dsenv file, the same operating system scripts, DataStage components etc. You install them
both using the same steps, or install one and copy the image into the other. IBM have some
specific licensing requirements when it comes to production backups that you should check.

If you have separate dev, test and prod licenses you could consider making test your production
backup and keep a production copy project on it at all times. This could be better than having a
production backup system that doesn't get used 99% of the time.

Ritu Sethi | Jun 24, 2009

Thanks for the response Vincent .
So this would mean that both servers have to always be up and running and we need to install
the same code (application code or any other OS or middleware patches etc ) on both . So when
we run jobs in production what would happen if the primary server fails assuming we were
doing heavy duty loads or aggregating Terabytes of data . How does it work in terms of critical
jobs running in production ? Will the stanby server take over the job in process or we will need
to redo the jobs anyway ?

Thanks

Thanks

Vincent McBurney | Jun 25, 2009

I think you would need some type of production monitoring software, such as Tivoli, to detect
that a server is down and switch on the other server. The scheduling tool should be able to
restart DataStage jobs from where it left off as long as you are controlling your jobs from an
external tool or table rather than DataStage Sequence jobs. A DataStage Sequence can restart
automatically if it fails but not if the server it is on is not working.

Ritu Sethi | Jul 2, 2009

Thanks again for the information . In a nutshell if the active server fails we will need to
manually restart the jobs on the passive server once the passive server comes online .

Jul 15, 2009

Just to continue from the point raised by ritu....

how does this apply to datastage metadata repository database? in our environment we have
chosen oracle as database for housing DS repository, when we failover the repository to DR site
is there any property or xml that we need to update the new identity?
Please clarify.

Ritu Sethi | Jul 23, 2009

Good Question Madan,

Is the metadata repository on the same server as DataStage ?

If it is on a different server then the standby server would be pointing to it also . Question is if
the metadata repository is keeping the datastage server information then there needs to be a way
to update that on the database server.
Vincent may be able to help with clarifying this .
Hello Vincent ,

Had a question for you .

What are the different options in 8.1 (on AIX ) to automate importing DataStage jobs ?
I understand that we can use dsjob import options and can issue this from a shell script or perl
script . Is this a recommended option ?

Thanks

Ritu Sethi

Sep 23, 2009

Hi All,

I want to know about SAS Datastage Plugins.What is the purpose and how we are using SAS
Datastage plugins.Is sas reports suported by SAS Datastage plugins.

Vincent McBurney | Sep 23, 2009

The SAS plugin is quite simple - it can read or write SAS datasets natively. So you can populate
a SAS dataset from a DataStage job or read a SAS dataset without knowing much about SAS.
I'm not sure about SAS reports, DataStage is a data integration tool so it's just trying to move
data around. I think it can execute SAS routines from DataStage that could kick off a report.

Sep 24, 2009

Thanks, Vincent McBurney

Is there any documentation available for SAS Datastage Plugins.Could you please provide if you
have any documentation.

USER_2029320 | May 24
Hi, This is Bhargavi .Iam planning to join in a Data Stage Training pogram .Could u ppl please
help me out with new version(8.1) course content

Madhusudhan y | 9 days ago

hi all,am new to Datastage and am getting lot of struggle to install datastage 8 in my laptop.so
would you people can help me out, whats d sytem requirements needed and can any one can
help me to install. at present am staying in bangalore.

Leave a Comment

Submit
Connect to this blog to be notified of new entries.
Name PREVIEW

E-mail

You are not logged in.

Sign In to post unmoderated comments.
Join the community to create your free profile today.
Archive Category: IBM Information Server
Keyword Tags: datastage information
server ibm qualitystage netezza oracle teradata upgrade migrate metadata server
Disclaimer: Blog contents express the viewpoints of their independent authors and are not
reviewed for correctness or accuracy by Toolbox for IT. Any opinions, comments, solutions or
other commentary expressed by blog authors are not endorsed or recommended by Toolbox for
IT or any vendor. If you feel a blog entry is inappropriate, click here to notify Toolbox for IT.
Browse all IT Blogs
Suggested Links
HelpDesk or Customer Support
Web based IT HelpDesk with Asset Mgmt or Customer support Software with Account & Contact mgmt.
DOWNLOAD YOUR FREE TRIAL TODAY! www
Integrated Desktop Management Software
Affordable Web-based Windows Desktop Management Software for Enterprises. Free Trial Download !
DOWNLOAD YOUR FREE TRIAL TODAY ! w
Financial Compliance Tool
Mobile Data Poses A Risk To Financial Compliance. Learn More.
MaaS360.com/Forrester-Report
Desktop Management Made Easy - Slash the Rising Cost of PC Management with Virtual Desktop!
For every $1 spent on IT hardware, you’ll pay $3 to support it! Slash PC costs with Virtual Desktop!
www.ITtoolbox.com
Simplify and automate desktop management with VDI and HP Thin Clients.
Deploy comprehensive VDI solutions with PC Connection's VDI services and HP Thin Client technology.
www.ITtoolbox.com
Are Your PCs Overdue for an Upgrade? Refresh Your PCs for Lower TCO!
Refresh every 3 years for maximum payoff. Deploy new systems that deliver exceptional performance.
www.ITtoolbox.com
Buy a Link Now
Receive the latest blog posts in your inbox
Your E-mai
 Follow this Blog

Share Your Perspective

Share your professional knowledge and experience with peers. Start a blog on Toolbox for IT
today!
Begin Now

Work With Me
Want to work for the IBM Information Management Australian partner of the year 2010? I am
building a team of great Information Server and DataStage consultants in Melbourne, Sydney,
Canberra, Adelaide and Perth. Send an email to vmcburney at focuss.com.au.

Links

• Steal This IM Methodology

• Informatica Data Quality Blog

• DataFlux Community of Experts

• Group 1 Data Quality Blog

• IBM LeverageInformation Community Blog

• Data Governance Blog

• dq:view - Steve Tuck on Data Quality

Categories

Go

Toolbox for IT
My Home Topics People Companies Jobs White Paper Library
Collaboration Tools
Discussion Groups Blogs Wiki
Follow Toolbox.com
Toolbox for IT on Twitter Toolbox.com on Twitter Toolbox.com on Facebook
Topics on Toolbox for IT
Data Center
Data Center
Development
C Languages Java Visual Basic Web Design & Development
Enterprise Applications
CRM ERP Infor PeopleSoft SAP SCM Siebel
Enterprise Architecture & EAI
Enterprise Architecture & EAI
Information Management
Business Intelligence Database Data Warehouse Knowledge Management Oracle
IT Management & Strategy
Emerging Technology & Trends IT Management & Strategy Project & Portfolio Management
Networking & Infrastructure
Hardware Mobile & Wireless Networking Telephony
Operating Systems
Linux UNIX Windows
Security
Security
Storage
Storage
Toolbox.com
About News Privacy Terms of Use Work at Toolbox.com Advertise Contact us Provide
Feedback
Help Topics Technical Support
Other Communities
Toolbox for HR Toolbox for Finance
Copyright 1998-2010 Toolbox.com. All rights reserved. All product names are trademarks of
their respective companies. Toolbox.com is not
affiliated with or endorsed by any company listed at this site. Toolbox.com is a subsidiary of the
Corporate Executive Board.

New to Toolbox?
New to Toolbox?
New to Toolbox?
0

New to Toolbox.com?
The Toolbox.com community helps you solve workplace problems.
Get started today - solve your problems and stay current.
Join Now
Ask A Question
Join
E-mail or User ID

Password
 Go

Stay on Current Page

Keep me signed in
Help?
Recover Password
Sign in using your account with

Return to this page

Haven't registered yet?
Register now
/w EWMAK+25+G

/w EPDw ULLTE3M

Bottom of Form