TM Deployment Guide

Real Data Integration Solutions
TM Deployment
September 2010
ETL Solutions Limited 2010 ETL Solutions Ltd.
ETL Transformation Manager
TM Deployment
Part Number: TM418-TMD Doc. Issue: 02 (September 2010)
Version Information This edition of TM Deployment is subject to change without notice and is supplied without warranty of any kind. Copyright Notice All information contained in this document is the property of ETL Solutions Limited. The information contained in this document is subject to change without notice and does not constitute a commitment on the part of ETL Solutions Limited. No part of this document may be reproduced in any manner, including storage in a retrieval system, transmission via electronic means or other reproduction medium or method (electronic, mechanical, photocopying, recording or otherwise) without the prior written permission of ETL Solutions Limited. 2010 ETL Solutions Limited. All rights reserved. All trade marks acknowledged. All trade marks mentioned herein belong to their respective owners.
TM Deployment
Preface
About TM Deployment This is a user guide for users wishing to deploy TM transforms, (a) using the TM Deployment Tool, (b) from a command prompt or within a script or batch file, or (c) by using pure Java code. Documentation Set TMs documentation set, shown in the diagram below, is structured hierarchically, with document s aimed at specific types of user.
TM BASICS
TM Features
TM Deployment
TM Upgrade Tool
TM Release Notes
Installation Guide
Properties Editor Guide
Patterns Guide
Database Guide
Using and Creating Transform Packs
TM Runtime Error Handling
Building TM Projects Program matically
Key
User
Administrator
Data modeller
Database expert
Advanced User
The product distribution includes, For the User: TM Basics which contains step-by-step instructions for developing transformation projects that illustrate both general principles and specific techniques; TM Features which describes a number of individual features to be found in TM. It is a self-contained tutorial-style introduction to some of the more advanced features and is intended to supplement TM Basics. TM Deployment which explains how to deploy transforms generated by TM. These three documents constitute the primary user guides to TM, and it is strongly advised that the user begin with these documents in order to gain a sound knowledge of TM and its use. TM Upgrade Tool which explains how repositories created by previous versions of TM may be upgraded so that they may be used by the current version. TM Release Notes which outline any new features and known issues in the current release are also included. For the Administrator:
ii
TM Deployment
Installation Guide which provides Windows and UNIX installation instructions for TM. Properties Editor Guide which describes how users may edit and extend the properties stored in the lax files generated when installing TM. For the Data Modeller: Patterns Guide which is an advanced guide describing patterns of use for solving common transformation problems. For the Database Expert: Database Guide which provides an introduction on how to use TM to transform data to a database efficiently. For the Advanced User: Building TM Projects Programmatically which describes how to build TM projects programmatically. Using and Creating TransformPacks which explains what a TransformPack is, how to run one and how to create one. TM Runtime Error Handling which describes the behaviour of the TM runtime when errors are thrown.
Audience This guide is intended for those who wish to deploy TM transforms.
iii
TM Deployment
Contents
PREFACE .............................................................................................................................................. II CONTENTS ......................................................................................................................................... IV TABLE OF FIGURES ........................................................................................................................ VI INTRODUCTION .................................................................................................................................. 1 TM DEPLOYMENT TOOL.................................................................................................................. 2 INTRODUCTION ..................................................................................................................................... 2 USING TM DEPLOYMENT TOOL ........................................................................................................... 2 Available Tasks ............................................................................................................................... 3 Scheduled Tasks ............................................................................................................................ 12 Output ........................................................................................................................................... 16 COMMAND LINE DEPLOYMENT .................................................................................................. 17 INTRODUCTION ................................................................................................................................... 17 INSTRUCTIONS .................................................................................................................................... 17 Step 1 ............................................................................................................................................ 17 Step 2 ............................................................................................................................................ 17 Step 3 ............................................................................................................................................ 17 LICENSING INFORMATION................................................................................................................... 17 JNDI PASSWORDS .............................................................................................................................. 18 PURE JAVA DEPLOYMENT ............................................................................................................ 18 INTRODUCTION ................................................................................................................................... 18 LICENSING INFORMATION................................................................................................................... 18 GENERATION OF JAVA CODE .............................................................................................................. 19 TMTRANSFORMER / TMTRANSFORMEXECUTOR: OVERVIEW OF OPERATION .................................... 19 Initialising TMTransformer / TMTransformExecutor................................................................... 19 Setting Adaptors ............................................................................................................................ 19 Running the Transforms and Getting Results ............................................................................... 20 JNDI Passwords ........................................................................................................................... 21 TMTRANSFORMEXECUTOR - MAIN LOOP .......................................................................................... 22 Basic pattern ................................................................................................................................. 22 Variations ..................................................................................................................................... 22 SESSIONS DIVISIONS AND PARTITIONS ............................................................................................... 25 "Partitions" and "Divisions" ......................................................................................................... 25 Transformation Architecture with Partitions and Division .......................................................... 25 Stalled Handles ............................................................................................................................. 26 Built-in Adapters and Sessions ..................................................................................................... 26 CLEARING A PROJECT ......................................................................................................................... 26 TRANSFORMS FROM $DOCUMENT, $PREDOCUMENT AND $POSTDOCUMENT .................................... 26 RUNNING THE SAMPLES ..................................................................................................................... 27 SHORT-LIVED DEPLOYMENT EXAMPLES ............................................................................................ 28 Simple Example............................................................................................................................. 28 Java to Java Example ................................................................................................................... 37 Java to XML Example ( as serialised file) .................................................................................... 37 Java to XML Example (as DOM in memory) ................................................................................ 38
iv
TM Deployment
Java to XML Example (as serialised memory output) .................................................................. 39 XML (as file) to Java Example ...................................................................................................... 39 XML (as inputStream ) to Java Example ..................................................................................... 41 Java to Database URL Example ................................................................................................... 41 Java to Open RDBMS Connection Example ................................................................................. 42 Database URL to Java Example ................................................................................................... 42 Open RDBMS Connection to Java Example ................................................................................. 43 Java to Java (with Error Handler) Example ................................................................................. 43 Java to Java (with embedded objects) Example............................................................................ 44 Java to Generic Example .............................................................................................................. 46 Generic to Java Example ............................................................................................................. 46 Flat File to Flat File example (using the built in flat file adapters) ............................................. 47 LONG-LIVED DEPLOYMENT EXAMPLES.............................................................................................. 47 Simple Example............................................................................................................................. 47 Java to Java Example ................................................................................................................... 55 Java to XML (as serialised file) Example ..................................................................................... 56 Java to XML (as DOM in memory) Example ................................................................................ 57 Java to XML (as serialised memory output) Example .................................................................. 58 XML (as file) to Java Example ...................................................................................................... 58 XML (as W3C Document ) to Java Example ................................................................................ 59 XML (as inputStream ) to Java Example ...................................................................................... 60 JMS Queue to JMS Queue (With XML) ........................................................................................ 60 Java to Database URL Example ................................................................................................... 63 Java to Open RDBMS Connection Example ................................................................................. 64 Database URL to Java Example ................................................................................................... 64 Open RDBMS Connection to Java Example ................................................................................. 65 Java to Java (with Error Handler) Example ................................................................................. 66 Java to Java (with embedded objects) Example............................................................................ 67 Java to Generic Example .............................................................................................................. 69 Generic to Java Example .............................................................................................................. 69 CLASSPATH ........................................................................................................................................ 70
TM Deployment
Table of Figures
Figure 1 Deployment Tool Dialog ......................................................................................................... 2 Figure 2 New Task Project Name .......................................................................................................... 3 Figure 3 New Task Source Type ........................................................................................................... 4 Figure 4 New Task Source Configuration ............................................................................................. 4 Figure 5 New Task Target Type ............................................................................................................ 5 Figure 6 New Task Target Configuration .............................................................................................. 5 Figure 7 New Task Properties ............................................................................................................... 6 Figure 8 New Task Error Handlers ........................................................................................................ 6 Figure 9 Error Handlers Project Name .................................................................................................. 7 Figure 10 Error Handlers Target Type .................................................................................................. 7 Figure 11 Error Handlers Target Configuration .................................................................................... 8 Figure 12 Error Handlers Project Display ............................................................................................. 8 Figure 13 New Task Name .................................................................................................................... 9 Figure 14 Deployment Tool Main Dialog ............................................................................................. 9 Figure 15 New Main Method Task: Main Class.................................................................................. 10 Figure 16 New Main Method Task: Runtime Arguments ................................................................... 10 Figure 17 New Main Method Task: Task Name ................................................................................. 11 Figure 18 New Main Method Task Completed ................................................................................... 11 Figure 19 Scheduled Tasks .................................................................................................................. 12 Figure 20 Scheduled Tasks List ........................................................................................................... 13 Figure 21 Scheduling for New Task .................................................................................................... 13 Figure 22 Scheduled Task Name ......................................................................................................... 14 Figure 23 Completed New Scheduled Task ........................................................................................ 14 Figure 24 Task History ........................................................................................................................ 15 Figure 25 Output ................................................................................................................................. 16
vi
TM Deployment
Introduction
Transformation Manager (TM) is used to develop transformation programs that transform data from one format to another. The developed transformation programs may then be deployed in three main ways. This document describes how to deploy those programs using various methods: Using TM Deployment Tool, a stand-alone environment for scheduling and running transforms. Using Command Line Utilities Using Java code TM Deployment Tool provides the easiest deployment method. However, depending upon the deployment requirements, the user may choose to run TM transforms from the Command Line or to write a JAVA program that calls TM Transforms. The latter provides the greatest flexibility and supports many kinds of EAI, EII, Business Intelligence integration architectures. In reality, the Java-calling program required for these architectures are relatively simple and a number of examples are described in the Pure Java Deployment section. ETL will provide other examples to fit with EAI, EII, Business Intelligence integration architectures on request.
TM Deployment
TM Deployment Tool
Introduction
The TM Deployment Tool provides a stand-alone environment for scheduling and running transforms. A Deployment Task is a specific transform, with source and target defined, any parameters needed specified, and a schedule for running that transform.
Using TM Deployment Tool
Figure 1 Deployment Tool Dialog
The main dialog of the TM Deployment Tool lists the current tasks that have been created by the user. Tasks that have been created previously will be read in automatically when the Deployment Tool is launched. There are three main sections to the TM Deployment Tool dialog: Available Tasks Scheduled Tasks Output Transform Tasks to be used by the TM Deployment Tool must first be created. Available Tasks may then be Scheduled to run according to the users preference. Information on the running projects are displayed in the Output section. Previously created Available and Scheduled Tasks are loaded when the Deployment Tool is launched.
TM Deployment
Available Tasks Within the Available Tasks section of the TM Deployment Tool, the user may: Create a New Transform Task Create a New Main Method Task Edit a Task, or Delete a Task Create New Transform Task Wizard To create a new transform task, click on New Transform Task within the Available Tasks section of the main TM Deployment Tool dialog. The New Transform Task Wizard will take you through the tasks involved in creating a new task. Select Project Select the project that you want to deploy. You may either point at the generated Jar File, or manually specify the project name and major version. The project name is case sensitive and must be entered correctly. The generated Jar file must be in the current classpath:
Figure 2 New Task Project Name Press Next to go to the Source Type page.
ETL Transformation Manager Source Type Select your source model type whether it is an XML file, RDBMS, Java or generic:
TM Deployment
Figure 3 New Task Source Type Press Next to go to the Source Configuration page.
Source Configuration Specify the source configuration. The configuration fields depend on the type selected on the previous page.
Figure 4 New Task Source Configuration Press Next to go to the Target Type page.
ETL Transformation Manager Target Type Select the target type.
TM Deployment
Figure 5 New Task Target Type Press Next to go to the Target Configuration page.
Target Configuration Specify the target configuration. The configuration fields depend on the type selected on the previous page.
Figure 6 New Task Target Configuration Press Next to go to the Properties page.
TM Deployment
Properties You may specify any properties needed for the task at run-time here. These are usually required if a getString() appears in the transformation project. The properties are added as named, value pairs.
Figure 7 New Task Properties Press Next to go to the Error Handlers page.
Error Handlers It is possible to configure an error handling project into the Deployment Task.
Figure 8 New Task Error Handlers 6
TM Deployment
Press Add and specify the error handling project's name and major version:
Figure 9 Error Handlers Project Name
Specify the target model type:
Figure 10 Error Handlers Target Type
TM Deployment
Specify the target configuration:
Figure 11 Error Handlers Target Configuration
The details of the error handling project will be displayed on the dialog:
Figure 12 Error Handlers Project Display
Press Next to go to the Task Name page.
TM Deployment
Task Name Enter a name for the Task.
Figure 13 New Task Name Press Finish to save the task details.
New Main Method Task Wizard The TM Deployment Tool can launch Java code directly by means of the New Main Method Task Wizard.
Figure 14 Deployment Tool Main Dialog Click on New Main Method Task... to proceed to the Main Class page. 9
TM Deployment
Main Class Page Specify the fully qualified path of the class that contains the 'main(String[]args)' method to be executed.
Figure 15 New Main Method Task: Main Class Press Next to proceed to the Runtime Arguments page.
Runtime Argument Page Specify any run-time arguments here. Press Add... or Add Encrypted... if you wish the argument to be protected (e.g. for a password).
Figure 16 New Main Method Task: Runtime Arguments Press Next to go to the Task Name page. 10
TM Deployment
Task Name Page Supply a name for the Task Name.
Figure 17 New Main Method Task: Task Name Press Finish to create the task.
Completing the New Main Method Task Wizard
Figure 18 New Main Method Task Completed The newly created task may now be edited, deleted or scheduled. Edit Task The stages involved in editing a task are the same as when creating a task. Click Edit Task Delete Task
11
TM Deployment
Select the task to be deleted from the TM Deployment Tool main dialog. Choose Delete from the File menu, or use the Delete button on the toolbar. The task and its history will be permanently deleted. Scheduled Tasks Within the Scheduled Tasks section of the TM Deployment Tool, users may: Create a new Scheduled Task Edit a Scheduled Task Delete a Scheduled Task, or Display the History of a Task Create New Scheduled Task The New Scheduled Task Wizard takes the users through the tasks involved in creating a new scheduled task. The selected tasks will be run in sequence according to the users scheduling rules.
Figure 19 Scheduled Tasks Click on New Scheduled Task to bring up the New Scheduled Task Wizard. Select Tasks Specify at least one task to be scheduled. Press Add to select.
12
TM Deployment
Figure 20 Scheduled Tasks List Press Next to proceed to the Scheduling page. Scheduling Specify the scheduling rules.
Figure 21 Scheduling for New Task The Scheduling Page allows users to specify that a test runs as frequently as you wish. You may choose at which hour of which day of which month the test will first be run and how frequently thereafter. In the screenshot above, the test is scheduled to run at 1:00 pm on the 27 th of the month and at the same time every day thereafter. To run a test once a week, choose Run Periodically Every 7 Day(s). Press Next to proceed to the Task Name page. Task Name Specify a name for the task. 13
TM Deployment
Figure 22 Scheduled Task Name Press Finish to complete the task creation. Completed New Scheduled Task
Figure 23 Completed New Scheduled Task The newly created scheduled task may now be edited or deleted, and its history may also be viewed. The new scheduled task must be enabled (by ticking the Enabled box) before it will run.
Display the Task History A summary of the history of a particular scheduled task may be viewed by clicking on History
14
TM Deployment
Figure 24 Task History
15
TM Deployment
Output This section displays the output stream from all running projects, and includes information about transforms and performance.
Figure 25 Output
16
TM Deployment
Command Line Deployment

Introduction
This section describes how to deploy transforms from within a script or batch file.
Instructions
The following steps describe how to run transformation programs from a command prompt. Step 1 Ensure the following: There is a suitable Java JRE on your system. For more information on Java version compatibility see the section Java Versions in the Installation Guide. All required jars are on the classpath. For more information see the section Classpath in this document Step 2 Runtime configuration is specified in a properties file. If you have previously run the project in the TM Test Tool there is a file that can be copied: [TMHOME]/config/Exec.properties. Edit this file if any configuration details need to be changed, for example you may want to specify different database connection details. Step 3 Run the transforms with this command line (the generated Exec.properties file has been copied and renamed myExec.properties and is in the current directory): java Dfile=myExec.properties com.prismt.transform.runtime.exec.RunMaps The following command line executes the original Exec.properties file and can be used when that file has been edited directly: java com.prismt.transform.runtime.exec.RunMaps The above command is equivalent to using the following command: java -Dfile=[TMHOME]/config/Exec.properties com.prismt.transform.runtime.exec.RunMaps Note The original Exec.properties file is overwritten each time a transformation is run using the Test Tool and will therefore need to be edited repeatedly. Editing and running a copy of it avoids constant editing. You can write this file from the Test Tool without actually executing the transform by using the Setup menu option.
Licensing Information
Before deploying code from the command line you must be correctly licensed. When calling java com.prismt.transform.runtime.exec.RunMaps you need to pass in the licence location as a VM parameter (i.e. as a -D argument), using etl.license.file as the key i.e. java -Detl.license.file=etc/etltm.lic
17
TM Deployment
JNDI Passwords
If a project contains lookups which require passwords for JNDI database connections, they are added as application arguments to the RunMaps main method. The format is password.[JNDI name]=[password] For example: com.prismt.transform.runtime.exec.RunMaps password.firstConnection=pass1 password.secondConnection=pass2
Pure Java Deployment

Introduction
TM is meta-data driven, but generates pure Java code. This code may be used on any Java JM (1.3 and above) to perform data transformations. From release 2.4 onwards it has been possible to run TM run in the 'short-lived' or 'long-lived' mode. The main difference is that in the 'short-lived' mode TM objects are freshly created for particular transforms, and then garbage collected, while in 'long-lived' mode TM objects are allowed to persist in memory. When running in 'short-lived' mode the principal TM class is called TMTransformer; in 'long-lived' mode it is called TMTransformExecutor. Historically, TM has been packaged for use in a stand-alone environment, and hence in 'short-lived' mode. In this mode of operation a run-time interface TMTransformer object exists only for the lifetime of the transforms. This use of TM is well documented in existing documentation and has its attractions. Where machine efficiency is not of paramount importance this type of short-lived TMTransformer can be a very neat way of packaging transforms. However, the main drawback is that it takes time to create the TMTransformer object - even when the transforms are packaged as Java. Where the transforms are packaged as XML (i.e. are supplied as XML meta-descriptions) then this time increases still further. For this reason, ETL also provide a long-lived component known as TMTransformExecutor. In addition, TMTransformExecutor gives a more fine-grained control over transform execution. Note that both TMTransformer and TMTransformExecutor extend the common interface TMTransformerCommon, and thus contain many of the most important methods. This guide begins by explaining the method by which Java Code is generated. It then describes all aspects of deploying Java transforms generated by TM at run-time, including such topics as: An Overview of TMTransformer and TMTransformExecutor Operation Sessions Divisions and Partitions used, for example, to control transactions Query Propagation used to obtain source data in response to a query in terms of the target model The Guide concludes with a set of examples using 'short-lived' and long-lived transforms.
Licensing Information
There is a method on the net.etltm.TMTransformerFactory
public static void setTMLicenseLocation(String theLocation)
When executing a transform, this call must be made on the TMTransformerFactory before the factory is used to create a TMTransformer. The location can be path to the licence file, or a URI to the location of the license file on a remote machine.
18
TM Deployment
If deploying in a non-file based environment, the call, setTMLicenseLocation must be called (as above), but passing in a special string (which will be supplied by ETL in these cases), which then stops the run-time system from checking for a licence.
Generation of Java Code

When building projects, TM takes the model meta-data and SML transforms and generates Java code. The generated Java code is compiled and the packaged in a generated jar file. Various settings affect the building of the jar file, for information see the 'SML Generation Settings' page in the online help.
TMTransformer / TMTransformExecutor: Overview of Operation

Both TMTransform and TMTransformerFactory. TMTransformExecutor are created from the factory class
Both are provided with a default constructor (zero argument) and a constructor taking an arbitrary Java Object. If an arbitrary Java object is provided, this can be used by Java functions written in TM to call back to the calling program. To do this merely requires the functions to use the TMContext:getUserReferenceObject within their body. As constructed, neither object is initialised, and must be properly initialised before it can be used to transform data. Once initialised, it is in a 'ready' state - available to take input data and transform it into output data. Initialising TMTransformer / TMTransformExecutor All TMTransformCommon objects must be provided with three pieces of data, namely: the transform project it will be running, the source adapter(s), and the target adapter(s) Providing project details Where the transform project has been generated into Java, the correct interface to use is: TMTransformExecutor:addProject(String projectName, String projectVersion). This will cause the loading of the relevant Java classes from the class path during the completeInitialisation call. Where the generated project is provided as XML the interface is: TMTransformExecutor:addXMLProject(String projectName, String projectVersion, Boolean isErrorHandler,String theXMLFileSpec, java.io.InputStream theXMLFileStream, org.w3c.dom.Document theXMLDocument). You should only provide the XML data in one of the forms. See also setXMLSMLReaderVersion(int xmlLoaderVersion) & setXMLSMLReaderTransform(String xmlLoaderProject) which are used to set up the version of transformer used to read the meta-data. More than one project may be set into a given run-time interface, TMTransformer or TMTransformExecutor. Note that the first project provided is known to the system as the 'primary project'. This is treated slightly differently from other projects in both TMTransformer and TMTransform Executor. Setting Adaptors Once the project(s) have been set, it is necessary to set source and target adapters for each project. There is little difference between using built-in adapter types or generic adapter types. Where the adapters are built-in (XML, RDBMS, Java Bean), they should be constructed as TMBuiltInAdaptorFactory 19
TM Deployment
Where the adapters are generic (i.e. user implementations of types TMGenericReaderAdapter or TMGenericWriterAdapter) then the user creates the adapter object according to their own needs Once adapters are constructed, they are set on the TMTransformerCommon adapter using: setSource(prjName,Adapter) and setTarget(prjName,Adapter) calls. There should be no need to cast the adapter to any particular type when using these calls as they are already suitably overloaded. You will also need to set the connection parameters for the adapters this is done using setInstanceConnectionData for all types of adapter. This takes a Map of keys and values. Thus, typical code for example to set up the built-in read JDBC adapter could look like:
TMAdapter srcTMAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJDBCAdapter(aTMTransformer); Map srcParams = new HashMap(); srcParams.put (TMJDBCAdapter.NAME_USER, "Scott"); srcParams.put (TMJDBCAdapter.NAME_PASSWORD, "Tiger"); srcParams.put (TMJDBCAdapter.NAME_URL, "jdbc:oracle:thin:@LOCALHOST:1521:garmon"); srcParams.put (TMJDBCAdapter.NAME_DRIVER, "oracle.jdbc.driver.OracleDriver"); srcTMAdapter.setInstanceConnectionData(srcParams); aTMTransformer.setSource (projectName,srcTMAdapter);
Normally, adapters have the same long lifetime as TMTransformer or TMTransformExecutor itself. However, the 'session' mechanism is available should the adapter wish to divide its lifetime into a series of independent periods. What happens in each period is entirely user-defined so an adapter may physically connect to different sources and sinks of data in each of these periods. Setting up TMTemplateAdapters The built-in flat file adapters implement TMTemplateAdapter. These adapters require additional information to be loaded at runtime. This is done as shown in this example:
if (anAdapter instanceof TMTemplateAdapter) { TMTemplateAdapter aTemplateAdapter = (TMTemplateAdapter) anAdapter; net.etltm.tmr.TMTemplateAdapterHelper.loadTemplate(aTemplateAdapter, false, projectName, projectMajor, 0); }
Note that in the example above, the second argument passed into the 'loadTemplate' is 'false'. This argument will change depending on whether the adapter in question is used on the source or target side. Running the Transforms and Getting Results Once setSource and setTarget have been called, the design patterns for TMTransformer and TMTransformExecutor differ. Short-Lived Mode For a TMTransformer the correct pattern is simply openProjectAndResources followed by runAllTransforms. Note that where the target is Java or XML target, you may need to ask the write adapter for the results of the transforms (i.e. the mapped data). Where the source is Java or XML target then you may need to place data into the read write adapter before runAllTransforms. Long-Lived Mode Completing Initialisation Once all sources and targets have been provided, TMTransformExecutor's completeInitialisation method should be called. This loads XML or Java descriptions of the meta-data. If the adapters are non-generic then the completeInitialisation method opens the primary sources and targets. Where the adapters are generic, the method setMetadata will be called inside each user written adapter. This provides the user with access to useful meta-data for the internal operation of the adapter. (Note that information provided depends on the setting up of setRequiresMetaData in the loader interface.)
20
TM Deployment
TMTransformExecutor does not open generic adapters at this point it is the users responsibility to open the generic adapters used with TMTransformExecutor. Note completeInitialisation method will throw exceptions if it is called prematurely or any of these methods fail. Calling transform methods is not possible until completeInitialisation has completed. The Main Loop Once initialisation is complete, the transformer is ready to serve and transform data. The main control loop that does this is explained in more detail later in this document. Closing a TMTransformExecutor The TMTransformExecutor:closeDown method should be called when no further transformations are required. This will call the finish methods inside any generic adapters and will close down any of the built-in ETL adapters. Note that both TMTransformer and TMTransformExecutor are single threaded processes and should not be called simultaneously from multiple threads. However, there is nothing to stop generic adapters from using separate threads internally for example to send data across a network, or simply to queue data for efficiency. In addition TMTransformExecutor may know of 'stalled' items that are necessarily waiting for further items in the input pipelines. Thus, if closeDown is called at the 'wrong time', generic adapters may wish to throw a TMException, while TMTransformExecutor may throw a TMException if the pool of 'stalled' items is not empty. The TM Test Tool The TM Test Tool is simply an interface onto the short-lived TMTransformer. In addition to testing and interactive transform development (in conjunction with the TM Design Tool), it can be used as a primitive deployment tool and can be run from the command line. It communicates with the run-time session itself by executing a Java class called 'com.prismt.transform.runtime.exec.RunMaps'. It picks up the required parameters to run from a file called 'Exec.properties' in the current directory. It is possible to use this mechanism to run transforms without writing any Java code at all. JNDI Passwords If a project contains lookups which require passwords for JNDI database connections, they are specified as shown in the following example: try { net.etltm.helper.DefaultTMConnectionLocatorConfiguration.configure(ne t.etltm.helper.DefaultTMConnectionLocator.getInstance()); String[] sArgs = {"password.World=" + "fred", "password.AnotherConnection=" + "frank"}; //World is name of the connection used in the lookup, //fred is the actual password //multiple connection / password pairs can be set as shown in this example net.etltm.helper.TMPasswordHolder.setPasswords(sArgs); } catch (Exception e) { // exception handling code goes here }
21
TM Deployment
TMTransformExecutor - Main Loop

Basic pattern Because transforms are called on an instance-by-instance basis, it is easy to implement the basic loop design pattern:
////either built-in adapter code TMDHIterator anIt= getObjectsOfInterest(aTMSourceQuery); ////or generic user code TMDHIterator anIt= user code returning an iterator of TMReadElement's while (true) { TMReadElement aInputDh = anIt.getNext(); if (null==aInputDh) { break; } TMWriteElement aOutputDh = (TMWriteElement )myTMTransformExecutor.transform(aInputDh); // further user processing such as: myUserWrittenTarget.consumeHndle(aOutputDh); //consumeHndle is a user written function }
This allows for fine control of the transform process. Note that simple transforms will be stateless outside the call to "transform". However, transforms that are more complex will require that part TMTransformExecutor retains "stalled" handles and also references to fkey/primary key relationships. Variations Simplest Pattern The easiest sort of generic source to target the loop pattern is:
while (true) { TMReadElement aInputDh = (TMReadElement)anIt.getNext(); if (null==aInputDh) { break; } myTMTransformExecutor.transform(aInputDh); } // then some call to myWriteAdapter.'getData.'
This pattern is probably more common should the target be a non-generic adapter. Here the return value from the each call to 'transform' is simply ignored. We can do this because the write side handle is already 'included' in the relevant write adapter. For RDBMS systems, this means that data will eventually be flushed to disc, while for other types of adapter some kind of getData method is available to getJava or SAX events after the loop has finished. More Complex Patterns However, it is also possible to use the return value from the transform within the group for increased flexibility. The principal reason you might wish to use these options with the built-in Java or XML adapters is to allow the transformation process to scale beyond limitations of memory. For databases, on the other hand, this approach is more likely to be adopted because the user wishes a more intimate involvement in transaction control and JDBC optimisations. (Note that by default the ETL database handler is fully scalable anyway). Thus, with the built-in Java adapter as target, the loop could read:
while (true) { TMDH aInputDh = anIt.getNextDH();
22
TM Deployment
if (null == aInputDh) { break; } TMDH aDH = executor.transform(aInputDh); // Now calling aDH.getJavaObject() will get us the objects in the user defined classes. }
Thus, with the built-in XML adapter as target, the loop could read:
while (true) { TMReadElement aInputDh = anIt.getNext(); if (null==aInputDh) { break; } TMDH aDH = (TMDH)myTMTransformExecutor.transform(aInputDh); }
Thus, with the built-in RDBMS adapter as target, the loop could read:
while (true) { //maybe rdbms starting transactions/batching etc TMReadElement aInputDh = anIt.getNext(); if (null==aInputDh) { break; } TMDH aDH = (TMDH )myTMTransformExecutor.transform(aInputDh); // rdbms stopping/committing transactions/batching etc }
Note that there may be many calls to getObjectsOfInterest followed by particular types of processing. Controlling what is returned by ObjectsOfInterest The only parameter to getObjectsOfInterest is a TMSourceQuery. This is, potentially, a very complex structure, which allows the user to perform a very complex set of selections on the source. (In conjunction with TMQueryPropagator this selection can even be defined in target terms - see TMQueryPropagator). Simple uses include: Getting One Domain Using the static factory class method in TMQueryObjectsFactory to describe a selection of all objects from one domain.
TMSourceQuery aTMSourceQuery = net.etltm.qp.TMQueryObjectsFactory.createSourceQuery(domain)); anIterator=adapter.getObjectsOfInterest(aTMSourceQuery);
Getting Several Domains Using the static factory class method in TMQueryObjectsFactory to describe a selection of many objects (e.g. 'Account', 'Name', 'Transaction') from several domains.
Vector v = new Vector(); v.add("Account"); v.add("Name"); v.add("Transaction"); TMSourceQuery aTMSourceQuery = net.etltm.qp.TMQueryObjectsFactory.createSourceQueryV(v)); anIterator=adapter.getObjectsOfInterest(aTMSourceQuery);
Selecting attributes Limiting the selection to certain attributes on a domain (e.g. 'payer', 'payee' and 'date') on any particular class (e.g. 'transaction')
TMSourceQuery aTMSourceQuery = net.etltm.qp.TMQueryObjectsFactory.createSourceQuery(transaction));
23

aTMSourceQuery.addAttribute("payer"); aTMSourceQuery.addAttribute("payee"); aTMSourceQuery.addAttribute ("date")
TM Deployment
anIterator=adapter.getObjectsOfInterest(aTMSourceQuery); Getting all domains This can be done with:

anIterator=adapter.getObjectsOfInterest(null);
But note that some adapters cannot implement this call - the JDBC built-in adapter, for example, does not implement this call - you need to know what domains (i.e. tables) you are selecting. Convenience Classes There are number of convenience classes defined for this system mostly in the package 'net.etltm.helper'; TMDHListIterator The class TMDHListIterator found in net.etltm.helper is a simple implementation of a TMDHIterator based on a Java list. If your underlying architecture allows you to represent your iterated objects in this fashion then this class provides a convenient way of building the TMDHIterators required by the above methods direct from a Java list. AbstractTMReadAdapter Extending the AbstractTMReadAdapter is the easiest way of building a generic read adapter. This approach is suitable if you are building an adapter to handle instances of moderate size. Thus, if you can reasonably expect the adapter to hold all input objects in memory this is the recommended choice. AbstractTMWriteAdapter Extending the AbstractTMWriteAdapter is the easiest way of building a generic write adapter. This approach is suitable if you are building an adapter to handle instances of moderate size. Thus, if you can reasonably expect the adapter to hold all output objects in memory, before serialising them or sending them to another component, this is the recommended choice. Notes on the Transform Methods These basic transform calls are very simple each takes a source item and returns a target item. TMTransformExecutor:transform(TMReadElement) TMTransformExecutor:transform(TMReadElement, targetDomainList, insist) Both these calls return a DataHandle, i.e. single items of type TMDH (which can be cast to TMWriteElements or your own types as required). If you have designed a transform so that it directly "unNests" then it may create a target array rather than a single TMDH. In this case, the form to use is: TMTransformExecutor:transformM(TMReadElement). This will, potentially, return a list of TMDHs. If you use the transform method and your transform does "unNest" then no exception will be thrown, but you will only ever get the first element created. Note the single parameter call runs any defined transform from TMReadElement's source domain. If there are two or more independent transforms from the same source domain (a design pattern that is not encouraged, but not excluded) then it will simply choose one of them randomly. (Future versions of the product may support mode and priority to help make this process more determinable). The second method resolves this ambiguity and attempts to run the defined transform from TMReadElement's source domain to the specified list of target domains. The names in TargetDomainList should be in priority order. If "insist" is true, then this is the only operation it will attempt - if there were no such defined transform it will simply throw an error. If "insist" is false and if it fails to find any successful transforms to the domains in the target Domain it will continue and perform any transform from TMReadElement's source domain. If targetDomainList is null then insist is ignored and the behaviour is as the single parame ter call.
24
ETL Transformation Manager Using TM adapters outside TM
TM Deployment
All TM adapters, including the built-in adapters, can be used outside TM itself and within your own software architecture. They are simply re-usable components.
Sessions Divisions and Partitions

TMTransformExecutor is a long-lived component. It is possible to open and close adapters owned by a TMTransformExecutor during its lifetime. In addition, the lifetime of each open adapter may be divided into sets of what we call 'sessions'. These divide its lifetime into a series of open sessions during which "blocks" of data are being transformed. There is no limit to the depth of this subdivision. The precise activity performed by each user adapter as each session (of any particular level) begins and ends is entirely user-defined. Typical 'beginSession' actions could include 'connecting' to a new data source, or beginning some kind of transaction, or doing nothing at all. Typical 'endSession' activities might be 'disconnecting' or ending a transaction. The only requirement is that sessions should be properly nested i.e. that each 'beginSession' is followed by an 'endSession' to end normally or another 'beginSession'. Calling a second 'beginSession' at the same level as an already open Session is used to force the creation of a new session and the loss of the previous session at that level. Thus, there is no need for an abort Session operation. However, these semantics cannot be enforced by TM except for its own concept of 'partitions' and 'divisions' (see below). Note finally that TM's session control is not restrictive generic adapters may implement any of these operations outside the interfaces required by TM. Once these user operations have been defined, all TMTransformExecutor's implementation of 'beginSession' and 'endSession' does is to call through to 'beginSession' and 'endSession' on each of its current open adapters in turn. The adapters are ordered in order of adapter opening. While 'endSession' takes no parameters, other than the level of session it is ending, 'beginSession' also takes a java.util.Map. This map is keyed into whatever string data is required to "begin a session" at that level - again this is user-defined. Where a generic adapter has been implemented to co-operate with TMs GUI interfaces, then it is possible, using the call 'getBeginSessionNames' to discover the keys required for any particular adapter see the interface TMGenericInstance for further details. Finally, note that 'beginSession' returns a Boolean. This should be false if no action is taken, true if any action is taken. This allows the Transformer itself finally to return a result for its 'beginSession' operation, which will only be 'true' if any action has been taken by any adapter. "Partitions" and "Divisions" TM itself may be set to respond to two levels of session. These are termed "partition" and "division". These are not hard coded to any particular level of session. For flexibility, TMTransformExecutor contains methods to set and get the levels at which 'beginSession' and 'endSession' are to be interpreted internally as instructions to begin or end 'partitions' or 'divisions'. For example, the call setRecognisedDivisionLevel(3) tells TMTransformExecutor to interpret beginSession(3) and endSession(3) as the beginning and ending a TMTransformExecutor "division." Note that 'beginSession' and 'endSession' events are still also called on all open adapters even if they are interpreted by TMTransformExecutor. In the case of 'beginSession', place any special processing within TMTransformExecutor after 'beginSession' has been called on the other adapters. In the case of endSession, TMTransformExecutor's own processing takes place before that in the adapters. Transformation Architecture with Partitions and Division In order to understand what 'divisions' and 'partitions' actually do it is necessary to have some understanding of the internal architecture of the transformation engine at least when it is using generic adapters and configured in long-life mode. When internal adapters are in use, other caching patterns and optimisations take place. In long-life mode, every call to TMTransformExecutor's transform method produces zero, one or a number of data handles. Before returning to control to the user TMTransformExecutor will attempt to "flush" these handles out to the user's generic implementation by calling the finalize method. However, for reasons of efficiency it will only flush out non-identified handles - handles that have identification 25
TM Deployment
are retained internally. The lifetime of this internal retention is the lifetime of a partition. Thus, ending a partition will have the effect of flushing identifying handles through the user layer. Thus, by ending, or by not ending, partitions, the user may control how often identifying handles are flushed out to the user's implementation of the generic adapter. Note that all handles are always flushed out when an adapter closes. Thus, 'partitions' are optional. The concept of the 'division' however, is quite different. One of the advanced features of TM is its treatment of batched data. TM understands that batched data may not necessarily arrive in the right order, or may refer to other pieces of data in other batches. To do this, it holds lightweight references between batches. TM chooses to retain this data based on its analysis of value-based relationships. The lifetime of this lightweight data is known as a "division". Ending a division thus removes from memory any lightweight references that may be used between batches. Note that divisions are only meaningful if TMTransformExecutor::setSupportsCrossSessionHandles has been called. This call simply turns on TM's ability to remember references. Otherwise, such references are never remembered. Stalled Handles When TM is holding forward references, some of its data handles may be placed into a state known as "stalled." These stalled handles cannot be written out through the user layer because they contain unresolved mandatory references. In relational database terms, these are best thought of as lacking data for mandatory foreign key columns. Such mandatory references may, of course, arrive in later transforms - if they do, then the stalled handle status is cleared and the handle itself automatically written out as appropriate. However, stalled handles may persist until the adapter closes, at which point their continued existence may prove a problem. Thus, three methods exist to help handle the pool of stalled handles. The method getStalledHandleDescriptions returns descriptions of those handles that are stalled including the domains, and the reason for their stalled status. The method TMTransformExecutor:destroyStalledHandles may be used to remove these from memory at any time the data in the handles is simply discarded. The method clearStalledHandleFlags may be used to remove the stalled status of these handles - the next call to end a partition will then attempt to flush them through to the user layer (this may result in faults at the user layer level as it attempts to deal with unresolved mandatory references). If an adapter is closed and stalled handles are present, all normal handles are first flushed through to the user layer - then an exception will be thrown to describe what is stalled. Built-in Adapters and Sessions Note that the built-in adapters for databases Java and XML currently take no special action on the beginning or ending of sessions at any level. This will, however, change.
Clearing a Project
The method removeXMLProject(String projectName, String projectVersion) may be used to remove a project from the TMTransformExecutor. Once cleared another set of transforms may be loaded. There is no corresponding equivalent for Java generated code.
Transforms from $Document, $PreDocument and $PostDocument

Outside XML, these transforms principally exist in order to allow the SML code to perform system functions directly. These are only automatically run in the short-lived TMTransformer - but can be explicitly run in a TMTransformExecutor (.e.g. using the methods ' runPostDocumentTransform', 'runPreDocumentTransform' and ' runDocumentTransform'). In the short-lived TMTransformer, the calls are run as follows: $PreDocument transforms are always called before built-in adapters are physically connected. For the generic adapters this means just before the call back to 'setMetaVerificationOn'.
26
ETL Transformation Manager $Document transforms are called just after the call to 'setMetadata'. $PostDocument transforms are called after just after the call to 'finish'.
TM Deployment
Note that generic adapters have no access to any type of $Document object - this object is internally created and destroyed entirely within the TMTransformCommon.
Running the Samples

TM is supplied complete with a set of samples in the directory "samples\\deployment\\examples". (This is a relative path from the TM install directory - thus if you have chosen to install TM in "X:\\Program Files\\TM" then the complete path is "X:\\Program Files\\TM\\samples\\deployment\\examples"). To run the samples you will need to follow the following steps: 1) Open TM Design Tool and build each of the projects in turn. They have already been set to generate the jar code to "samples\\deployment\\lib". When generation has finished the directory "samples\\deployment\\lib" should contain the files:
1. ETL_BasicXMLErcHandler_1_0.jar 2. ETL_GenericToJava_1_0.jar 3. ETL_JavaToGeneric_1_0.jar 4. ETL_JavatoJavaEmbededXML_2_0.jar 5. ETL_JavaToJavaWithError_2_0.jar 6. ETL_JavaToJava_2_0.jar 7. ETL_JavaToJDBC_2_0.jar 8. ETL_JavaToXML_2_0.jar 9. ETL_JDBCToJavaFilter_1_0.jar 10. ETL_JDBCToJava_2_0.jar 11. ETL_LLJavatoJavaEmbededXML_1_0.jar 12. ETL_XMLToJava_3_0.jar 13. ETL_XMLtoXML_1_0.jar
2) You will also need to build the supplied Java code. The directory "samples\\deployment\\examples" contains the actual Java samples themselves - the main file here is "ExamplesInJava". This contains a number of very simple utility procedures for displaying the results of the transforms, and also the static strings defining the location of the various sources and targets used in the examples. Note that for reasons of simplicity, these read and write only from the supplied example MS Access data bases using JDBC/ODBC, not from Oracle or Sybase (as is described in the text). There is, of course, no need to have MS Access actually installed in your system. The directories tgt and src contain the actual Java instance models used in the examples. All the Java code must be built for the system to run. This may vary from system to system. Using JBuilder, (recommended) the steps you need to perform are: Create a new JBuilder project for your examples: 1. 2. 3. 4. Set the source path to something like " X:\\TM\\samples\\deployment\\examples" Add the packages "examples" and "examples.tgt" and "examples.src" to your product. Ensure that the libraries include both ofetsr.jar and the generated jar files listed above. Generate and run the Java code. Running in the JBuilder debugger is highly recommended as it allows you to focus on items of interest.
27
TM Deployment
Short-Lived Deployment Examples

The code included in this section is taken from the standard examples distributed with the product. Note that ALL possible combinations of source and target are legal it is only for conciseness that this document limits the examples to the 16 described here. Thus, it is quite legal to transform between XML and XML or between database and database. Simple Example File Setup The example code is built up in a single Java file. Step 1 File Creation We start with the outline of a java file containing the main building blocks of any application. Begin by defining the package and class for your deployed code.
package shortlived; public class ShortLived { public ShortLived() { } }
Step 2 - Main As the entire example will be self-contained in a single file we also add a main method in order to run the application. The main method below contains a single line to create an instance of the ShortLived object.
package shortlived; public class ShortLived { static public void main(String[] args) { ShortLived shortLived = new ShortLived(); } public ShortLived() { } }
Now these steps are complete, the actual methods to execute the transforms can be considered. Running the Transformation System As running transformations in Short Lived mode is straightforward, we will only require a single method that will run the transforms.
For the purpose of this example, the source will be taken from an XML file, with the data also being written to a target XML file. Step 3 Initialise Add the outline of the method runTransform. 28

package shortlived; public class ShortLived { static public void main(String[] args) { ShortLived aShortLived = new ShortLived(); aShortLived.runTransform(); } public ShortLived() { }
TM Deployment
public void runTransform() { System.out.println("ETL solutions Short Lived XML to XML Deployment Example"); } }
Step 4 New Transformer The first step is to create a new Short-lived TMTransformer object. This is the main object used to carry out the transformations that have been created using the TM Design Tool.
package shortlived; import net.etltm.*; public class ShortLived { static public void main(String[] args) { ShortLived aShortLived = new ShortLived(); aShortLived.runTransform(); } public ShortLived() { } public void runTransform() { System.out.println("ETL solutions Short Lived XML to XML Deployment Example"); try { //Set the licence file location TMTransformerFactory.setTMLicenseLocation( "d:\\mydeployment\\licence\\etltm.lic"); //Short-lived TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,"shortLivedProj",1); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
1.
Add the import line at the top of the file. This allows the use of the TMTransformer object. 29
ETL Transformation Manager 2. 3. 4.
TM Deployment
Add a call to set the licence file. This must be made on the TMTransformerFactory before the factory is used to create a TMTransformer. A Try-Catch block is required around the creation of the TMTransformer. The static factory method used to create the TMTransformer is described in detail within the API documentation found within the installation directory \Program Files\TM\docs\api-docs\index.html.
The 3 parameters passed in are: 1. userRefObject This can be any java object. It may be retrieved later in any of the java functions defined in the SML project. 2. Project Name The name of the project for the transformations. 3. The Version The major version of the project for the transformations. Step 5 Source Adapter The next step is to set up the source adapter. The source adapter is used to access the source instance data for transformation.
package shortlived; import net.etltm.*; public class ShortLived { static public void main(String[] args) { ShortLived aShortLived = new ShortLived(); aShortLived.runTransform(); } public ShortLived() { } public void runTransform() { System.out.println("ETL solutions Short Lived XML to XML Deployment Example"); try { //Short-lived TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,"shortLivedProj",1); //Create SOURCE Adapter TMXMLAdapter aXMLSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource("shortLivedProj",aXMLSourceAdapter); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
1.
The source XML adapter is created in a single line directly from TMs own adapter factory by simply calling makeNewReadTMXMLAdapter from the 30
ETL Transformation Manager TMBuiltInAdaptorFactory class. documentation. 2.
TM Deployment Further detail can be found in the API
The second line entered sets the source adapter for the TMTransformer object created in Step 4. Hence, the TMTransformer will be able to use the TMXMLAdapter when accessing the source data.
Step 6 Target Adapter The target adapter is created in a similar fashion to the source adapter. 1. The target XML adapter is created in a single line directly from TMs own adapter factory by simply calling makeNewReadTMXMLAdapter from the TMBuiltInAdaptorFactory class. Further detail can be found in the API documentation. The second line entered sets the target adapter for the TMTransformer object created in Step 4. Hence, the TMTransformer will be able to use the TMXMLAdapter when writing the target data.
2.
package shortlived; import net.etltm.*; public class ShortLived { static public void main(String[] args) { ShortLived aShortLived = new ShortLived(); aShortLived.runTransform(); } public ShortLived() { } public void runTransform() { System.out.println("ETL solutions Short Lived XML to XML Deployment Example"); try { //Short-lived TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,"shortLivedProj",1); //Create SOURCE Adapter TMXMLAdapter aXMLSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource("shortLivedProj",aXMLSourceAdapter); //Create TARGET Adapter TMXMLAdapter aXMLTargetAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget("shortLivedProj",aXMLTargetAdapter); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
31
ETL Transformation Manager Step 7 Setting the Source XML File
TM Deployment
The xml file to be used as the source file can now be specified for the source TMXMLAdapter.
package shortlived; import net.etltm.*; import java.util.*; public class ShortLived { static public void main(String[] args) { ShortLived aShortLived = new ShortLived(); aShortLived.runTransform(); } public ShortLived() { } public void runTransform() { System.out.println("ETL solutions Short Lived XML to XML Deployment Example"); try { //Short-lived TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,"shortLivedProj",1); //Create SOURCE Adapter TMXMLAdapter aXMLSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource("shortLivedProj",aXMLSourceAdapter); //Create TARGET Adapter TMXMLAdapter aXMLTargetAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget("shortLivedProj",aXMLTargetAdapter); //Set the XML file the source adapter will read in Map param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\source.xml"); aXMLSourceAdapter.setInstanceConnectionData(param); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
1. 2. 3.
Add a new import line at the top of the file in order to use hash maps in the form of java.util. Create a new HashMap which will be used to pass parameters into the source TMXMLAdapter. Add the location and file name of the source xml file to the parameter hashMap, using the pre-defined static key TMXMLAdapter.NAME_URL for identification.
32
ETL Transformation Manager 4.
TM Deployment
Use the setInstanceConnectionData method on the source adapter, in order to add the parameter.
Step 8 Setting the Target XML File The setting of the target XML file is done in a similar fashion to the source file described above.
package shortlived; import net.etltm.*; import java.util.*; public class ShortLived { static public void main(String[] args) { ShortLived aShortLived = new ShortLived(); aShortLived.runTransform(); } public ShortLived() { } public void runTransform() { System.out.println("ETL solutions Short Lived XML to XML Deployment Example"); try { //Short-lived TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,"shortLivedProj",1); //Create SOURCE Adapter TMXMLAdapter aXMLSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource("shortLivedProj",aXMLSourceAdapter); //Create TARGET Adapter TMXMLAdapter aXMLTargetAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget("shortLivedProj",aXMLTargetAdapter); //Set the XML file the source adapter will read in Map param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\source.xml"); aXMLSourceAdapter.setInstanceConnectionData(param); //Set the XML file the target adapter will write to param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\target.xml"); aXMLTargetAdapter.setInstanceConnectionData(param); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
1.
Create a new HashMap which will be used to pass parameters into the target TMXMLAdapter.
33
ETL Transformation Manager 2. 3.
TM Deployment
Add the location and file name of the target xml file to the parameter hashMap, using the pre-defined static key TMXMLAdapter.NAME_URL for identification. Use the setInstanceConnectionData method on the target adapter, in order to add the parameter.
Step 9 Initialise the Data in TMTransformer Now the Source and Target Adapters have been created and the source and target xml files have been specified to the respective adapters, the TMTransformer object can open the required information.
package shortlived; import net.etltm.*; import java.util.*; public class ShortLived { static public void main(String[] args) { ShortLived aShortLived = new ShortLived(); aShortLived.runTransform(); } public ShortLived() { } public void runTransform() { System.out.println("ETL solutions Short Lived XML to XML Deployment Example"); try { //Short-lived TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,"shortLivedProj",1); //Create SOURCE Adapter TMXMLAdapter aXMLSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource("shortLivedProj",aXMLSourceAdapter); //Create TARGET Adapter TMXMLAdapter aXMLTargetAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget("shortLivedProj",aXMLTargetAdapter); //Set the XML file the source adapter will read in Map param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\source.xml"); aXMLSourceAdapter.setInstanceConnectionData(param); //Set the XML file the target adapter will write to param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\target.xml"); aXMLTargetAdapter.setInstanceConnectionData(param); //Open the project information along with the source //and target adapter information aTMTransformer.openProjectAndResources(); } catch (TMException aTMException) {
34

aTMException.printStackTrace(); return; } } }
TM Deployment
This is simply done in a single line that opens the transformation project information and the source and target file information. Step 10 Run the Transforms The final line to be added is the line that executes the transformations taking the data from the source and writing it to the target following the business logic defined using the TM Design Tool.
package shortlived; import net.etltm.*; import java.util.*; public class ShortLived { static public void main(String[] args) { ShortLived aShortLived = new ShortLived(); aShortLived.runTransform(); } public ShortLived() { } public void runTransform() { System.out.println("ETL solutions Short Lived XML to XML Deployment Example"); try { //Short-lived TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,"shortLivedProj",1); //Create SOURCE Adapter TMXMLAdapter aXMLSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource("shortLivedProj",aXMLSourceAdapter); //Create TARGET Adapter TMXMLAdapter aXMLTargetAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget("shortLivedProj",aXMLTargetAdapter); //Set the XML file the source adapter will read in Map param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\source.xml"); aXMLSourceAdapter.setInstanceConnectionData(param); //Set the XML file the target adapter will write to param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\target.xml"); aXMLTargetAdapter.setInstanceConnectionData(param); //Open the project information along with the source //and target adapter information
35

aTMTransformer.openProjectAndResources(); // run the transform aTMTransformer.runAllTransforms(); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
TM Deployment
Execution Step 11 Running the Program In order to run the java file created using the above steps, ensure you have the following in your classpath. 1. ofetsr.jar (Which can be found in the installed Program Files \TM\lib directory) 2. ETL_shortLivedProj_1_0.jar (The jar file created from your TM Design Tool mappings, located by default in Program Files\TM\com\prismt\transform\maps\shortLivedProj\V1\m0) Changes for Database Connections If the source, target or both were databases, the following changes would be required. In the following example, we will describe the changes to the target side connection. The changes required to the source side would be similar. Adapter Changes Where currently the adapter is created as :
TMXMLAdapter aXMLTargetAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer);
For a database connection simply changing the method to refer to a JDBC adapter as shown below will allow this :
TMJDBCAdapter aTMJDBCAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJDBCAdapter(myTMTransformer);
Connection Changes Currently connection to the target side information is done simply by specifying the xml file for transformation as shown below:
//Set the XML file the target adapter will write to param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, "d:\\xml\\target.xml"); aXMLTargetAdapter.setInstanceConnectionData(param);
For a database connection simply specify the USER, PASSWORD, URL and DRIVER as required, using the pre-defined static keys available with the TMJDBCAdapter.
Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_USER,"root"); params.put(TMJDBCAdapter.NAME_PASSWORD,"root"); params.put(TMJDBCAdapter.NAME_URL,"jdbc:odbc:mySQLTest"); params.put(TMJDBCAdapter.NAME_DRIVER,"sun.jdbc.odbc.JdbcOdbcDriver");
36

aTMJDBCAdapter.setInstanceConnectionData(params);
TM Deployment
Java to Java Example Notes This simply transforms a populated array of Java objects (of type "Player") found in the Game object into an array of "Person" objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. the directory in which the path examples.src.*class may be found). the examples.tgt path (i.e. the directory in which the path examples.tgt.*class may be found). Code
Try { TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJavaAdapter aSrcTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aSrcTMJavaAdapter); TMJavaAdapter aTgtTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTgtTMJavaAdapter); aTMTransformer.openProjectAndResources(); // add some data to the read adapter - this can be done before // this point if required // here we could use aReadAdapter.addObject(aGame) but then the // only objects known to the read adapter by name are the Game // objects. // if you want to transform from Person then we need to tell about // the PERSON objects // we could add them one a time using addObject or better still add // the whole array using addObjects aSrcTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); // run the transfrom and get the result object aTMTransformer.runAllTransforms(); // the file xmltarget.xml will have been created SimpleUtilities.showJavaObjects(aTgtTMJavaAdapter, "Player"); } catch (TMException aTMException) { System.out.println(aTMException.toString()); return; }
Java to XML Example ( as serialised file) This example of Java to XML (as a serialised file) simply transforms a populated array of Java objects (of type "Player") found in the Game object into an XML file. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. the directory in which the path examples.src.*class may be found.) 37
ETL Transformation Manager In this example, the file "xmltarget.xml" will be created in the current directory. Code
TM Deployment
TMXMLAdapter aTMXMLAdapter = null; try { TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter); aTMXMLAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMXMLAdapter); Map params = new HashMap(); params.put(TMXMLAdapter.NAME_URL, AllExamplesInJava.EXAMPLE_TGT_PATH + "xmltarget.xml"); aTMXMLAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); // add some data to the read adapter aTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); aTMTransformer.runAllTransforms(); // the file xmltarget.xml will have been created } catch (TMException aTMException) { aTMException.printStackTrace(); return; }
Java to XML Example (as DOM in memory) This example simply transforms a populated array of Java objects (of type "Player") found in the Game object into an org.w3c.dom.Document object (or DOM). To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. the directory in which the source classes may be found.) The transform returns an org.w3c.dom.Document object. Code
TMTMXMLAdapter aTMXMLAdapter = null; Document aDocument = null; try { TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter); // Here we DON'T specify a file to write to as we are going to ask // for the DOM output aTMXMLAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMXMLAdapter); aTMTransformer.openProjectAndResources(); // add some data to the read adapter aTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); aTMTransformer.runAllTransforms(); aDocument = aTMXMLAdapter.getXMLDocument();
38

} catch (TMException aTMException) { aTMException.printStackTrace(); return; }
TM Deployment
Java to XML Example (as serialised memory output) This example transforms a populated an array of Java objects (of type "Player") found in the Game object into a serialised version of the XML text. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. to the directory in which the path examples.src.*class may be found.) The output returns as an OutputStream. Code
TMXMLAdapter aTMXMLAdapter = null; try { TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter); // but we DON'T specify a file to write to as we are going to ask // for the serializeStream output aTMXMLAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMXMLAdapter); aTMTransformer.openProjectAndResources(); // add some data to the read adapter aTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); aTMTransformer.runAllTransforms(); aTMXMLAdapter.serializeStream(System.out); } catch (TMException aTMException)
XML (as file) to Java Example This XML (as serialised file) to Java example simply transforms an XML file into an array of "Person" objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. to the directory in which the path examples tgt.*class may be found.) Code
TMJavaAdapter aTMJavaAdapter = null; try { TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJDBCAdapter aTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJDBCAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJDBCAdapter); aTMJavaAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJavaAdapter);
39
TM Deployment
Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_URL,theURL); params.put(TMJDBCAdapter.NAME_DRIVER,TMC.sDC_ACCESS_DRV); aTMJDBCAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); aTMTransformer.runAllTransforms(); }
Source file The XML source file for the examples looks like:
<Game> <Weapon <Weapon <Weapon <Weapon <Weapon <Weapon <Weapon <Weapon <Room <Room <Room <Room <Room <Room <Room <Room <Person <Person <Person <Person <Person <Person <Person </Game>
name name name name name name name name = = = = = = = =
="ROPE" colour = "BROWN"/> ="REVOLVER" colour = "SILVER"/> ="REVOLVER" colour = "BLACK"/> ="SPANNER" colour = "METALLIC"/> ="KNIFE" colour = "RED"/> ="BASEBALL BAT" colour = "WOOD-EFFECT"/> ="FLICK-KNIFE" colour = "GREEN TORTOISE"/> ="BRICK" colour = "ORANGE"/> "BILLIARD ROOM"/> "DRAWING ROOM"/> "KITCHEN"/> "DINING ROOM"/> "LOUNGE"/> "BEDROOM"/> "BATHROOM"/> "LIVING ROOM"/> "Miss" name ="SCARLET" man ="false" roomName = "BILLIARD ROOM"/> "Colonel" name ="MUSTARD" man ="true" roomName= "DRAWING ROOM"/> "Prof" name = "PLUM" man ="true" roomName = "KITCHEN"/> "Mrs" name = "PEACOCK" man ="false" roomName = "DINING ROOM"/> "Mrs" name = "WHITE" man ="false" roomName = "LOUNGE"/> "Rev" name = "GREEN" man ="true" roomName = "BEDROOM"/> "Dr" name = "BLACK" man ="true" roomName = "LIVING ROOM"/>
name name name name name name name name
title title title title title title title
= = = = = = =
XML (as W3C Document ) to Java Example This XML (as W3C Document ) to Java example simply transforms an XML file into an array of "Person" objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. to the directory in which the path examples.tgt.*class may be found.) The code must also provide a populated org.w3c.dom.Document called aDocument; Code
TMJavaAdapter aTgtTMAdapter=null; try { TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,projectName,3); TMAdapter aSrcTMAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTgtTMAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aSrcTMAdapter); aTMTransformer.setTarget(projectName,aTgtTMAdapter); Document aDocument = SimpleUtilities.getDomFromFile(AllExamplesInJava.EXAMPLE_SRC_PATH+"xmlsource.xml");
40
TM Deployment
Map srcConnectionData = new HashMap(); srcConnectionData.put(TMXMLAdapter.NAME_ORG_W3C_DOM_DOCUMENT,aDocument); aSrcTMAdapter.setInstanceConnectionData(srcConnectionData); aTMTransformer.openProjectAndResources(); aTMTransformer.runAllTransforms();
XML (as inputStream ) to Java Example This XML (as inputStream) to Java example simply transforms an XML file into an array of "Person" objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. to the directory in which the path examples.tgt.*class may be found.) The code must also provide an InputStream containing a valid relevant XML serialization; Code
try { FileInputStream aFileInputStream = new FileInputStream(AllExamplesInJava.EXAMPLE_SRC_PATH + "xmlsource.xml"); TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this,projectName,3); TMXMLAdapter aTMXMLAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMXMLAdapter); aWriteAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aWriteAdapter); Map param = new HashMap (); param.put(TMXMLAdapter.NAME_JAVA_IO_INPUTSTREAM,aFileInputStream); aTMXMLAdapter.setInstanceConnectionData(param); aTMTransformer.openProjectAndResources(); aTMTransformer.runAllTransforms(); } catch (TMException
Java to Database URL Example This Java to Database URL example simply transforms a populated array of Java objects (of type "Player") found in the Game object into a database identified by URL, name and password. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. the directory in which the path examples.src.*class may be found). Code
TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJavaAdapter aTMJavaAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter ); TMJDBCAdapter aTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMJDBCAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJDBCAdapter);
41
TM Deployment
Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_URL,theURL); params.put(TMJDBCAdapter.NAME_DRIVER,TMC.sDC_ACCESS_DRV); // most data bses also require // params.put(TMJDBCAdapter.NAME_USER ... // params.put(TMJDBCAdapter.NAME_PASSWORD ... aTMJDBCAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); aTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.runAllTransforms(); // the RDBMS adapter will have flushed the data and closed the connection
Java to Open RDBMS Connection Example This Java to Open RDBMS example simply transforms a populated array of Java objects (of type "Player") found in the Game object into a database via an open database connection. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. to the directory in which the path examples.src.*class may be found. Code
tConnection aConnection = SimpleUtilities.getConnection(theURL); try { // set the system up to run TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJavaAdapter aTMJavaAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter); TMJDBCAdapter aTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMJDBCAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJDBCAdapter); Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_SQL_CONNECTION,aConnection); aTMJDBCAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); // add some data to the read adapter aTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.runAllTransforms(); } catch (TMException ...
Database URL to Java Example This example transforms from a database to an array of Java objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. the directory in which the path examples.tgt.*class may be found). The URL, name and password must refer to a database accessible via JDBC. Code
42
TM Deployment
TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJDBCAdapter aTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJDBCAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJDBCAdapter); aTMJavaAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJavaAdapter); Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_URL,theURL); params.put(TMJDBCAdapter.NAME_DRIVER,TMC.sDC_ACCESS_DRV); aTMJDBCAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); aTMTransformer.runAllTransforms();
Open RDBMS Connection to Java Example This Open RDBMS Connection to Java example simply transforms from the player items found via the open JDBC connection to an array of "Person" objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. to the directory in which the path examples.tgt.*class may be found". The connection must be an open JDBC Connection object/. Code
Connection aConnection = SimpleUtilities.getConnection(theURL); TMJavaAdapter aTMJavaAdapter; try { TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJDBCAdapter aTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJDBCAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJDBCAdapter); aTMJavaAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJavaAdapter); Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_SQL_CONNECTION,aConnection); aTMJDBCAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); aTMTransformer.runAllTransforms(); } catch (TMException
Java to Java (with Error Handler) Example This example is the same as the first Java to Java example except that it introduces an error handler. If no error handlers are defined, then errors are regarded as unhandled and sent to the general-purpose output text stream. Once an error handler has been added, the error is transformed by the code in the error handler and placed into the output device specified by the error handler (which maybe XML, Java or a database). The error will not be sent to the general-purpose output text stream. Any number of error handlers may be defined. Every error is handled by every error handler in turn. The only exception to this is if an error should occur within an error handler. In this case, the error is not handled by the error handler which threw the exception but by all the other error handlers.
43
TM Deployment
It is important not to confuse the concept of an error handler - that is to say a project dedicated to the transformation of errors from TMError into any output target - with the code contained within an SML catch error block. The two concepts are quite separate. Thus, if both an SML catch error block and an error handler exist, then error is processed in the following manner: The error is thrown and a event is created within a TM exception. The code in the catch error is executed (this may transform the error event.) The code in the error handler(s) is executed (this may transform the error event.) Note also that Error Handlers of registered with a given TMTransformer object. Thus, where the TMTransformer object creates other TMTransformer object during the course of the execution (see next example) then those new TMTransformer objects will not inherit currently registered Error Handlers. Code
TMJavaAdapter aTgtTMJavaAdapter = null; try { TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 2); TMJavaAdapter aSrcTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aSrcTMJavaAdapter); aTgtTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTgtTMJavaAdapter); // add an error handler - it only needs a target aTMTransformer.addProject(errorHandlerProjectName, 1, true); TMXMLAdapter errorHandlerOutput= TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget(errorHandlerProjectName,errorHandlerOutput); Map params = new HashMap(); params.put(TMXMLAdapter.NAME_URL,"errorExample.xml"); errorHandlerOutput.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); // add some data to the read adapter TMJavaAdapter aReadAdapter = (TMJavaAdapter) aTMTransformer.getReadTMAdapter(); // here we could use aReadAdapter.addObject(aGame) but then the only objects aReadAdapter.addObjects(aGame.getPlayers()); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); aTMTransformer.runAllTransforms(); } catch (TMException...
Java to Java (with embedded objects) Example This example expands the Java to Java example to show how an embedded XML stream may be forwarded from one transformation to another. This is typical of the techniques used to gain access to any embedded XML document placed within a JMS message. Basic Java Code This is identical to the Java to Java example. The only difference in the distributed examples is that the "game" object contains a populated XML string in addition to its other data.
SML Code
44
TM Deployment
The SML code contained logic to discover the presence of the XML String and to send it to a Java function defined within translation builder. This is the SML code:
TRANSFORM (Location) <- (Weapon) IF (myName='XML-Data') THEN X:=23; LOGS('Calling out!'); myName := MakeDOM(myColour); ELSE myName := myColour+CHARAT(myName,2); END_IF;
The function makeDOM has the following body. The header section in italics is automatically generated by TM Design Tool. The rest of the function is boilerplate code to get the initial user object passed into to the first transformation and then invoke a further transformation upon at code.
public static java.math.BigDecimal MakeDOM( TMContext theTMContext, java.lang.String P1) throws Exception { // get the theTMTransformer that started this transform TMTransformer theTMTransformer = theTMContext.getTMTransformer(); // now get the user's object set when they started the TMTransformer // cast it to the type we set JavatoJavaEmbeddedXML aUserObjct = (JavatoJavaEmbeddedXML) theTMTransformer.getUserReferenceObject(); // get the current write adapter TMJavaAdapter aTMJavaAdapter=theTMTransformer.getWriteTMJavaAdapter(); // start another transform - with same target and the source is the // String containing the embedded XML aUserObject.runExampleForXML(P1,aTMJavaAdapter); //user code return null; }
The method runExampleForXML is now able to create another TMTransformer object in the conventional manner. Apart from using the method setTargetAdapterFromExistingAdapter to allow it to use the existing target adapter there is nothing special about this Java code - it follows the same pattern as the previous Java code. Note that any TMExceptions that are thrown from this transformation will be sent back to the calling project and may be caught by the Error Handlers defined in the calling project.
Code
public void runExampleForXML(String P1, TMJavaAdapter aTMJavaAdapter) { String projectName = "XMLToJava"; TMJavaAdapter aWriteAdapter = null; try { TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 3); StringReader aStringReader = new StringReader(P1); Document aDocument = makeDomObj(P1); TMXMLAdapter aTMXMLAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMXMLAdapter); Map param = new HashMap (); param.put(TMXMLAdapter.NAME_ORG_W3C_DOM_DOCUMENT,aDocument);
45

aTMXMLAdapter.setInstanceConnectionData(param);
TM Deployment
aTMTransformer.setTargetAdapterFromExistingAdapter(null, aTMJavaAdapter); aTMTransformer.openProjectAndResources(); aTMTransformer.runAllTransforms(); } catch (TMException aTMException) { System.out.println(aTMException.toString()); return; } }
Java to Generic Example This example transforms from an array of Java objects to a set of CSV files. The CSV is written using the class 'samples.openinterfaces.csv.write.TmExampleCSVWriteAdapter' which is also supplied in the distribution. Code
TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 1); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter); TMAdapter aTMAdapter = new samples.openinterfaces.csv.write.TmExampleCSVWriteAdapter(); aTMTransformer.setTarget(projectName,aTMAdapter); Map params = new HashMap(); params.put(samples.openinterfaces.csv.write.TmExampleCSVWriteAdapter. DIRECTORY, AllExamplesInJava.EXAMPLE_TGT_PATH+"csv"); aTMAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); // add some data to the read adapter - this can be done before this // point if required aTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); aTMTransformer.runAllTransforms();
Generic to Java Example This example transforms from a set of CSV files to an array of Java objects. The CSV is read using the class 'samples.openinterfaces.csv.read.TmExampleCSVReadAdapter', which is also supplied in the distribution. Code
TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 1); TMAdapter aTMAdapter = new samples.openinterfaces.csv.read.TmExampleCSVReadAdapter(); aTMTransformer.setSource(projectName,aTMAdapter); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJavaAdapter); Map params = new HashMap(); params.put(samples.openinterfaces.csv.read.TmExampleCSVReadAdapter. DIRECTORY, AllExamplesInJava.EXAMPLE_SRC_PATH+"csv"); params.put(samples.openinterfaces.csv.read.TmExampleCSVReadAdapter. EXTENSION, "csv"); aTMAdapter.setInstanceConnectionData(params);
46

aTMTransformer.openProjectAndResources(); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); aTMTransformer.runAllTransforms();
TM Deployment
Flat File to Flat File example (using the built in flat file adapters) This example transforms using the built in flat file adapters to both read and write. Note that when using the flat file adapters the file for each element is specified using the convention url_[elementName] When deploying using the built-in flat file adapters, a call to loadTemplate() has to be explicitly made as shown in the example:
TMTransformer aTMTransformer = TMTransformerFactory.getNewTMTransformer(this, projectName, 1); TMTemplateAdapter aSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadCSVAdapter(aTMTransformer); TMTemplateAdapterHelper.loadTemplate(aSourceAdapter, true, projectName, 1, 0); Map sourceParams = new HashMap(); sourceParams.put("url_fixed1WithHdrDotted", "D:\\regressiontests\\csv\\source\\unaltered\\fixed1WithHdrDotted.txt"); sourceParams.put("Scaling", "N"); aSourceAdapter.setInstanceConnectionData(sourceParams); aTMTransformer.setSource(projectName, aSourceAdapter); TMTemplateAdapter aTargetAdapter = TMBuiltInAdaptorFactory.makeNewWriteCSVAdapter(aTMTransformer); TMTemplateAdapterHelper.loadTemplate(aTargetAdapter, false, projectName, 1, 0); Map targetParams = new HashMap(); targetParams.put("url_fixed1WithHdrDotted", "D:\\out.txt"); aTargetAdapter.setInstanceConnectionData(targetParams); aTMTransformer.setTarget(projectName, aTargetAdapter); aTMTransformer.openProjectAndResources(); aTMTransformer.runAllTransforms();
Long-Lived Deployment Examples

The code included in this section is taken from the standard examples distributed with the product. Running these samples is identical to running the short-lived examples. See the notes on 'Running the Samples' in the previous section Simple Example File Setup This simple example will show how to create the java deployment code for a long-lived adapter that continually reads files from a specific directory at 10-second intervals. Code snippets displayed in bold indicate new lines that have been added to the example. Step 1 File Creation We start with the outline of a java file containing the main building blocks of any application. Begin by defining the package and class for your deployed code.
package longlived; public class LongLived {
47

public LongLived() { } }
TM Deployment
Step 2 - Main As the entire example will be self-contained in a single file we also add a main method in order to run the application. The main method below contains a single line to create an instance of the LongLived object.
package longlived; public class LongLived { static public void main(String[] args) { LongLived longLived = new LongLived(); } public LongLived() { } }
Now these steps are complete, the actual methods to execute the transforms can be considered. Initialising the Transformation System The first method to be created is the initialisation method. This sets up the connections to the source and target data sources. For the purpose of this example, the source will be taken from an XML file, with the data being written to a target Database. Step 3 Initialise Add the outline of the method initSystem.
package longlived; public class LongLived { static public void main(String[] args) { LongLived longLived = new LongLived(); longLived.initSystem(); } public LongLived() { } public void initSystem() { System.out.println("Long Lived XML to Database Deployment Example"); } }
Step 4 New Transformer
48
TM Deployment
The first step of the initialisation is to create a new Long Lived TMTransformExecutor object . This is the main object used to carry out the transformations which have been created using the TM Design Tool. Within this example the TMTransformExecutor object will be used in both the initialisation method (to configure its source and targets) and secondly in a method to run the transformations, and therefore be created as a class member.
package longlived; import net.etltm.*; public class LongLived { //Class Members TMTransformExecutor myTMTransformer; static public void main(String[] args) { LongLived longLived = new LongLived(); longLived.initSystem(); } public LongLived() { } public void initSystem() { System.out.println("Long Lived XML to Database Deployment Example"); try { //Set the licence file location TMTransformerFactory.setTMLicenseLocation( "d:\\mydeployment\\licence\\etltm.lic"); //Long Lived myTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this,"longLivedProj",1); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
First add the import line at the top of the file. Declare myTMTransformer as a global variable of type TMTransformExecutor. A Try-Catch block is required around the creation of the TMTransformExecutor. Add a call to set the licence file. This must be made on the TMTransformerFactory before the factory is used to create a TMTransformer. The static factory method used to create the TMTransformExecutor is described in detail within the API documentation found within the installation directory \Program Files\TM\docs\api-docs\index.html. The 3 parameters passed in are: 1. 2. 3. userRefObject This can be any java object. It may be retrieved later in any of the java functions defined in the SML project. Project Name The Version The name of the project for the transformations. The major version of the project for the transformations. 49
ETL Transformation Manager Step 5 Source Adapter
TM Deployment
The next step is to set up the source adapter. The source adapter is used to access the source instance data for transformation. As the source will be required in later methods also for this example, it is set as a global variable.
package longlived; import net.etltm.*; public class LongLived { //Global Variables TMTransformExecutor myTMTransformer; TMXMLAdapter myTMXMLSourceAdapter; ...<code ommited due to space>. public void initSystem() { System.out.println("Long Lived XML to Database Deployment Example"); try { //Long Lived myTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this,"longLivedProj",1); //Create SOURCE Adapter myTMXMLSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(myTMTransformer); //Set the source adapter for the TMTransformExecutor myTMTransformer.setSource("longLivedProj",myTMXMLSourceAdapter);
} catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
The source adapter is created in a single line described in further detail within the API documentation. The second line entered sets the source adapter for the TMTransformExecutor object created in Step 4. Hence, the TMTransformExecutor will be able to use the TMXMLAdapter when accessing the source data. Step 6 Target Adapter The target adapter is created in a similar fashion to the source adapter. As the target for the transformation will be a database, a TMJDBCAdapter is created. myTMTransformer has the setTarget method called to indicate to the transformer which adapter will be used for the writing of information to the target. In order to pass the database connection information into the adapter a hash map of values is used. An Additional import is required for java.util. Common database parameters are added to the hash map using pre-defined static variables (such as NAME_USER, NAME_PASSWORD, NAME_URL and NAME_DRIVER) which can be found in TMJDBCAdapter.
50
TM Deployment
Note: As a mySQL database is being used as the target, an ODBC connection has to be set up to the database (Data Sources (ODBC) from the Administrative Tools within Windows).
package longlived; import net.etltm.*; import java.util.*; public class LongLived { //Global Variables TMTransformExecutor myTMTransformer; TMXMLAdapter myTMXMLSourceAdapter; ...<code ommited due to space>. public void initSystem() { System.out.println("Long Lived XML to Database Deployment Example"); try { //Long Lived myTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this,"longLivedProj",1); //Create SOURCE Adapter myTMXMLSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(myTMTransformer); //Set the source adapter for the TMTransformExecutor myTMTransformer.setSource("longLivedProj",myTMXMLSourceAdapter); //TARGET TMJDBCAdapter aTMJDBCAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJDBCAdapter(myTMTransformer); //Set the source adapter for the TMTransformExecutor myTMTransformer.setTarget("longLivedProj",aTMJDBCAdapter); //Set up database connection properties Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_USER,"root"); params.put(TMJDBCAdapter.NAME_PASSWORD,"root"); params.put(TMJDBCAdapter.NAME_URL,"jdbc:odbc:mySQLTest"); params.put(TMJDBCAdapter.NAME_DRIVER,"sun.jdbc.odbc.JdbcOdbcDriver"); aTMJDBCAdapter.setInstanceConnectionData(params); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
Step 7 Initialise TMTransformExecutor The resource and target information are opened separately within the TMTransformExecutor, as the source will be handled separately for the files being received. Finally add the loop that will run the transformations every 10 seconds.
51

package longlived; import net.etltm.*; import java.util.*; public class LongLived { //Global Variables TMTransformExecutor myTMTransformer; TMXMLAdapter myTMXMLSourceAdapter; ...<code ommited due to space>. public void initSystem() { System.out.println("Long Lived XML to Database Deployment Example");
TM Deployment
try { //Long Lived myTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this,"longLivedProj",1); //Create SOURCE Adapter myTMXMLSourceAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(myTMTransformer); //Set the source adapter for the TMTransformExecutor myTMTransformer.setSource("longLivedProj",myTMXMLSourceAdapter); //TARGET TMJDBCAdapter aTMJDBCAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJDBCAdapter(myTMTransformer); //Set the source adapter for the TMTransformExecutor myTMTransformer.setTarget("longLivedProj",aTMJDBCAdapter); //Set up database connection properties Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_USER,"root"); params.put(TMJDBCAdapter.NAME_PASSWORD,"root"); params.put(TMJDBCAdapter.NAME_URL,"jdbc:odbc:mySQLTest"); params.put(TMJDBCAdapter.NAME_DRIVER,"sun.jdbc.odbc.JdbcOdbcDriver"); aTMJDBCAdapter.setInstanceConnectionData(params); //As the source will be handled differently, resources and target information //are opened seperately myTMTransformer.openOnlyProjectResources(); myTMTransformer.openPrimaryTarget(); //Loop to run the transformations every 10 seconds while (true) { runTransform(); try { Thread.sleep(10000); } catch (InterruptedException ie) { System.out.println("Iterupt"); } } }
52

catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
TM Deployment
The following steps will describe how runTransform() is created. Step 8 Accessing the XML files First, add an import line at the top of the file for java.io that will be required for the XML file handling.
import java.util.*; import net.etltm.*; import java.io.*;
The runTransform() method is located below the initSystem() method however in the interest of space only the runTransform() method is shown below. For this example, it is assumed that the directory specified will contain only XML and no extension filtering is carried out. Create an array of the files in the directory and prepare to loop over each file.
private void runTransform() { //Get a list of files in the specified directory String filename; File[] files; File f = new File("d:\\xml\\"); files = f.listFiles(); //LOOP for all files currently in the directory for(int i=0; i< files.length; i++) { filename = files[i].getAbsolutePath(); System.out.println(filename); } }
Step 9 Setting the Source File Create a hash map to contain the source file to be transformed, using the pre-defined static key TMXMLAdapter.NAME_URL for identification. Secondly a session is begun for the TMXMLAdapter with an identity of 1. The parameter hash map denoting the file to be transformed is also passed in at this time. Sessions are used to group operations together within the adapter and hence a new session will be started for each individual XML file. Further code will be added in Step 10 to run the transformation, which is followed by the ending of the session for the TMXMLAdapter.
private void runTransform() { //Get a list of files in the specified directory String filename; File[] files; File f = new File("d:\\xml\\"); files = f.listFiles(); //LOOP for all files currently in the directory
53

for(int i=0; i< files.length; i++) { filename = files[i].getAbsolutePath(); System.out.println(filename); try { //Add the source XML file to the hash map Map param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, filename); //begin the session myTMXMLSourceAdapter.beginSession(1,param); //CODE FOR STEP 10 //end the session myTMXMLSourceAdapter.endSession(1); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } } }
TM Deployment
Step 10 Transforming the Data For this example the source instance data follows the BOOKS dtd structure within the Program Files\TM\Samples\XML directory. A new import is required at the top of the file, for the TMDHIterator described below.
package longlived; import import import import java.util.*; net.etltm.*; java.io.*; net.etltm.qp.TMQueryObjectsFactory;
First a TMDHIterator is created to iterate through the instances of BOOK wi thin the XML file. This is created by getting the objects of interest from the source TMXMLAdapter which in turn allows the creation of a query set (similar to a where clause in SQL) in this case passing in the parameter BOOK the item to be transformed.
private void runTransform() { //Get a list of files in the specified directory String filename; File[] files; File f = new File("d:\\xml\\"); files = f.listFiles(); //LOOP for all files currently in the directory for(int i=0; i< files.length; i++) { filename = files[i].getAbsolutePath(); System.out.println(filename); try
54

{ //Add the source XML file to the hash map Map param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, filename); //begin the session myTMXMLSourceAdapter.beginSession(1,param);
TM Deployment
//Create an iterator to select the object for mapping TMDHIterator aTMDHIterator = myTMXMLSourceAdapter.getObjectsOfInterest (TMQueryObjectsFactory.createTMQuerySet("BOOK")); //Loop through each Book while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } //Transform the data myTMTransformer.transform(aTMDH); } //end the session myTMXMLSourceAdapter.endSession(1); } catch (TMException aTMException) { aTMException.printStackTrace(); return; } //DELETE FILES IF REQUIRED } }
A simple while loop is then used to iterate through the objects of interest retrieved within the TMDHIterator. A TMDH object is used to store each object of interest within the TMDHIterator. Finally the TMTransformExecutor (myTMTransformer) calls the transform method to transform the object of interest currently retrieved within the while loop. Additional code can be added at the end of the method to delete the files that have been processed in this run. Step 11 Running the Program In order to run the java file created using the above steps, ensure you have the following in your classpath. 1. 2. ofetsr.jar (Which can be found in the installed Program Files \TM\lib directory) ETL_longLivedProj_1_0.jar (The jar file created from your TM Design Tool mappings, located by default in Program Files\TM\com\prismt\transform\maps\longLivedProj\V1\m0)
Java to Java Example Notes This simply transforms a populated array of Java objects (of type "Player") found in the Game object into an array of "Person" objects. To run, the classpath must include: 55
ETL Transformation Manager TM Jars and dependencies (see the Classpath section in this document)
TM Deployment
the examples.src path (i.e. the directory in which the path examples.src.*class may be found). the examples.tgt path (i.e. the directory in which the path examples.tgt.*class may be found). Code
try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 2); TMJavaAdapter aSrcTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aSrcTMJavaAdapter); TMJavaAdapter aTgtTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTgtTMJavaAdapter); aTMTransformer.openProjectAndResources(); aSrcTMJavaAdapter.addObjects(aGame.getPlayers()); TMDHIterator aTMDHIterator= aSrcTMJavaAdapter.getObjectsOfInterest( TMQueryObjectsFactory.createTMQuerySet("samples.deployment.examples.src.Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } SimpleUtilities.showJavaObjects(aTgtTMJavaAdapter, "Player"); } catch (TMException ...
Java to XML (as serialised file) Example This example of Java to XML (as a serialised file) simply transforms a populated array of Java objects (of type "Player") found in the Game object into an XML file. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. the directory in which the path examples.src.*class may be found.) In this example, the file "xmltarget.xml" will be created in the current directory. Code
try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this); aTMTransformer.addProject( projectName, 2,false); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter);
aTMXMLAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer);
56
TM Deployment
aTMTransformer.setTarget(projectName,aTMXMLAdapter); Map params = new HashMap(); params.put(TMXMLAdapter.NAME_URL,AllExamplesInJava.EXAMPLE_TGT_PATH + "xmltarget.xml"); aTMTransformer.openProjectAndResources(); aTMJavaAdapter.addObjects(aGame.getPlayers()); aTMXMLAdapter.setSessionLevelUsedToConnect(0); TMDHIterator aTMDHIterator= aTMJavaAdapter.getObjectsOfInterest(net.etltm.qp.TMQueryObjectsFactory.createTMQuerySe t(null)); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.beginSession(0,null); aTMTransformer.transform(aTMDH); aTMTransformer.endSession(0); } } catch (TMException
Java to XML (as DOM in memory) Example This example simply transforms a populated array of Java objects (of type "Player") found in the Game object into a org.w3c.dom.Document object (or DOM). To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. the directory in which the source classes may be found.) The transform returns an org.w3c.dom.Document object. Code
try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 2); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter); // but we DON'T specify a file to write to as we are going to ask for the // DOM output aTMXMLAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMXMLAdapter); aTMTransformer.openProjectAndResources(); // add some data to the read adapter aTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); TMDHIterator aTMDHIterator= aTMJavaAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("samples.de ployment.examples.src.Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break;
57

} aTMTransformer.transform(aTMDH); } aTMXMLAdapter.finish(); aDocument = aTMXMLAdapter.getXMLDocument();
TM Deployment
Java to XML (as serialised memory output) Example This example transforms a populated an array of Java objects (of type "Player") found in the Game object into a serialised version of the XML text. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. to the directory in which the path examples.src.*class may be found.) The output returns as an OutputStream. Code
try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 2); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter); // the following line is very similar to the JAVA output line // but we DON'T specify a file to write to as we are going to ask for the // serializeStream output aTMXMLAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMXMLAdapter); aTMTransformer.openProjectAndResources(); // add some data to the read adapter aTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); TMDHIterator aTMDHIterator= aTMJavaAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("samples.de ployment.examples.src.Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } aTMTransformer.finish(); aTMXMLAdapter.serializeStream(System.out);
XML (as file) to Java Example This XML (as serialised file) to Java example simply transforms an XML file into an array of "Person" objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. to the directory in which the path examples tgt.*class may be found.) Code
try {
58

TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this,projectName,3); TMXMLAdapter aTMXMLAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMXMLAdapter); aWriteAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aWriteAdapter); Map param = new HashMap (); param.put(TMXMLAdapter.NAME_URL, AllExamplesInJava.EXAMPLE_SRC_PATH+"xmlsource.xml"); aTMXMLAdapter.setInstanceConnectionData(param);
TM Deployment
aTMTransformer.openProjectAndResources(); TMDHIterator aTMDHIterator= aTMXMLAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } } catch (TMException ...
XML (as W3C Document ) to Java Example This XML (as W3C Document ) to Java example simply transforms an XML file into an array of "Person" objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. to the directory in which the path examples.tgt.*class may be found.) The code must also provide a populated org.w3c.dom.Document called aDocument; Code
try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this,projectName,3); TMAdapter aSrcTMAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTgtTMAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aSrcTMAdapter); aTMTransformer.setTarget(projectName,aTgtTMAdapter); Document aDocument = SimpleUtilities.getDomFromFile(AllExamplesInJava.EXAMPLE_SRC_PATH+"xmlsource.xml"); Map srcConnectionData = new HashMap(); srcConnectionData.put(TMXMLAdapter.NAME_ORG_W3C_DOM_DOCUMENT,aDocument); aSrcTMAdapter.setInstanceConnectionData(srcConnectionData); aTMTransformer.openProjectAndResources(); TMDHIterator aTMDHIterator= aSrcTMAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("Person")); while (true) {
59

TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } } catch (TMException ...
TM Deployment
XML (as inputStream ) to Java Example This XML (as inputStream) to Java example simply transforms an XML file into an array of "Person" objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. to the directory in which the path examples.tgt.*class may be found.) The code must also provide an InputStream containing a valid relevant XML serialization: Code
try { FileInputStream aFileInputStream = new FileInputStream(AllExamplesInJava.EXAMPLE_SRC_PATH + "xmlsource.xml"); TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this,projectName,3); TMXMLAdapter aTMXMLAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMXMLAdapter); aWriteAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aWriteAdapter); Map param = new HashMap (); param.put(TMXMLAdapter.NAME_JAVA_IO_INPUTSTREAM,aFileInputStream); aTMXMLAdapter.setInstanceConnectionData(param); aTMTransformer.openProjectAndResources(); TMDHIterator aTMDHIterator= aTMXMLAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } } catch (TMException
JMS Queue to JMS Queue (With XML) The example, MessageTransformer, reads a message from a JMS queue, extracts the XML payload, transforms that XML to a target XML document, and posts that XML to a second JMS queue.
60
TM Deployment
The code for placing the XML message on the first queue, and reading it from the second queue, is not described in this document, as this is standard JMS code, not involving TM. The example uses point to point messaging (Queues), but can be easily adapted to publish/subscribe (Topics). The following diagram gives a high-level view of MessageTransformer (Controller) calling sequence.
Although it shows the MessageTransformer making an explicit call to receive a message, the example code uses a MessageListener to be notified when a message arrives and start the transform. Initialising the JMS consumer and producer. These are initialised in the usual way:
Context context = new InitialContext(); ConnectionFactory connectionFactory = (ConnectionFactory) context.lookup("jms/ConnectionFactory"); connection = connectionFactory.createConnection(); Destination readDestination = (Destination) context.lookup("jms/Queue1"); Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE); consumer = session.createConsumer(readDestination); Destination writeDestination = (Destination) context.lookup("jms/Queue2"); producer = session.createProducer(writeDestination);
This code leaves a MessageConsumer listening on jms/Queue1 and a MessageProducer writing to jms/Queue2. The JMS MessageListener and the call to start the connection is shown below. Initialising the TM runtime 61
TM Deployment
Each message received by the consumer will be transformed using the same TM transform project. This is initialised once, not each time a message is received, as the initialisation takes time. The TMTransformExecutor, the long-lived TM runtime engine, is constructed and the XML read and write adapters created. The executor is then initialised for the specified project.
// projectName and version specify the TM project the executor will run. executor = TMTransformerFactory.getNewTMTransformExecutor(null, projectName, version); // Source is XML readAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(executor); executor.setSource(null, readAdapter); // Target is XML writeAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(executor); executor.setTarget(null, writeAdapter); // Session 0 will be used to control when TM finished transforming a given message. // run will be passed in. // Session 0 starts with a call to beginSession(), the source XML for the new transform // When session 0 ends, TM makes the target available, so it can be serialised etc. readAdapter.setSessionLevelUsedToConnect(0); writeAdapter.setSessionLevelUsedToConnect(0); // Initialise executor this does things like cache the XML target metadata, which // can be expensive to create, so it is only done once not per message. executor.openOnlyProjectResources();
Listening for and Transforming Messages Finally, a MessageListener implementation is written to read and transform the XML contained in the TextMessages on jms/Queue1. An instance of this is then set as the message listener for the consumer created in JMS initialisation above. Exception handing is omitted in the code below.
class TransformMessageListener implements MessageListener { public void onMessage(javax.jms.Message message) { String transformedMessage; // Tell source adapter what the source XML is (message text) String xml = ((TextMessage) message).getText(); Map properties = new HashMap(); properties.put(TMXMLAdapter.NAME_JAVA_LANG_STRING, xml); executor.beginSession(0, properties); // Query source document for elements to transform, typically this will be the // root element name. TMQuerySet query = TMQueryObjectsFactory.createTMQuerySet(elementToTransform); // iterate over each element found and transform it TMDHIterator sourceIterator = readAdapter.getObjectsOfInterest(query); while (sourceIterator.hasNextDH()) { TMDH nextInstance = sourceIterator.getNextDH(); if (nextInstance == null) { break; } executor.transform(nextInstance); } // all elements transformed - make target DOM available executor.endSession(0); // serialise to a String and post it to jms/Queue2 StringWriter swr = new StringWriter(); writeAdapter.serialize(swr); transformedMessage = swr.toString(); producer.sendMessage(transformedMessage); } }
An instance of this class is created, and set as the message listener for the consumer constructed in JMS initialisation above, then the connection is started so that the consumer will receive messages. When a message is received on jms/Queue1, the onMessage method will be called and the XML payload transformed. 62
TM Deployment
consumer.setMessageListener(messageListener); connection.start();
Deployment Hints The above snippets contain the details required to integrate TM into JMS, but need to be fitted in to full Java classes to compile and run. The jar files required to run the transforms include the J2EE application server jar files, the TM runtime jar files, and the TM jar file containing the generated code for the required project. As the above is an XML to XML transformation, the TM jar files required are ofetsdo.jar, and tmsaxon7.jar, from the TM installation. The J2EE app server jar files will include j2ee.jar, but may require implementation specific jar files for JNDI etc to work. For the Sun reference implementation, some details can be found at: http://docs.sun.com/source/819-0079/dgacc.html#wp1022252 Other systems will require different configuration and possibly changes to the initialisation code where the JNDI InitialContext is constructed. J2EE 1.3 implementations will need to use Queue (or Topic) rather than Destination as the common Destination interface was introduced in J2EE 1.4. The TM code remains the same. Java to Database URL Example This Java to Database URL example simply transforms a populated array of Java objects (of type "Player") found in the Game object into a database identified by URL, name and password. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. the directory in which the path examples.src.*class may be found). Code
try { // set the system up to run TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 2); TMJavaAdapter aTMJavaAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter ); TMJDBCAdapter aTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMJDBCAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJDBCAdapter); Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_URL,theURL); params.put(TMJDBCAdapter.NAME_DRIVER,TMC.sDC_ACCESS_DRV); aTMJDBCAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); // add some data to the read adapter aTMJavaAdapter.addObjects(aGame.getPlayers()); TMDHIterator aTMDHIterator= aTMJavaAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("samples.de ployment.examples.src.Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH();
63

if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } } catch (TMException ...
TM Deployment
Java to Open RDBMS Connection Example This Java to Open RDBMS example simply transforms a populated array of Java objects (of type "Player") found in the Game object into a database via an open database connection. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.src path (i.e. to the directory in which the path examples.src.*class may be found. Code
try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 2); TMJavaAdapter aTMJavaAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter); TMJDBCAdapter aTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMJDBCAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJDBCAdapter); Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_SQL_CONNECTION,aConnection); aTMJDBCAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); // add some data to the read adapter aTMJavaAdapter.addObjects(aGame.getPlayers()); TMDHIterator aTMDHIterator= aTMJavaAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("samples.de ployment.examples.src.Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } } catch (TMException ...
Database URL to Java Example This example transforms from a database to an array of Java objects. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. the directory in which the path examples.tgt.*class may be found). The URL, name password must refer to a database accessible via JDBC. 64
ETL Transformation Manager Code
TM Deployment
TMJavaAdapter aTMJavaAdapter = null; try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 2); TMJDBCAdapter aTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJDBCAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJDBCAdapter); aTMJavaAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJavaAdapter); Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_URL,theURL); params.put(TMJDBCAdapter.NAME_DRIVER,TMC.sDC_ACCESS_DRV); aTMJDBCAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); TMDHIterator aTMDHIterator= aTMJDBCAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } }
Open RDBMS Connection to Java Example This Open RDBMS Connection to Java example transforms from the player items found via the open JDBC connection to an array of "Person" objects. It shows how sessions may be used to run the transform piecemeal. To run, the classpath must include: TM Jars and dependencies (see the Classpath section in this document) the examples.tgt path (i.e. to the directory in which the path examples.tgt.*class may be found". The connection must be an open JDBC Connection object/. Note that in the example given, addKeyValuePair is called. This is done because the getString() function is being used within the transformation project Code
TMJavaAdapter aTMJavaAdapter; try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this); aTMTransformer.addProject(projectName, 1,false); TMJDBCAdapter aTMJDBCAdapter=TMBuiltInAdaptorFactory.makeNewReadTMJDBCAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJDBCAdapter); aTMJavaAdapter=TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJavaAdapter); Map params = new HashMap(); params.put(TMJDBCAdapter.NAME_SQL_CONNECTION,aConnection);
65

aTMJDBCAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); readSomeItems(aTMTransformer, projectName,aTMJDBCAdapter,aTMJavaAdapter,"PLUM"); readSomeItems(aTMTransformer, projectName,aTMJDBCAdapter,aTMJavaAdapter,"MUSTARD"); readSomeItems(aTMTransformer, projectName,aTMJDBCAdapter,aTMJavaAdapter,"WHITE"); readSomeItems(aTMTransformer, projectName,aTMJDBCAdapter,aTMJavaAdapter,"GREEN"); } catch (TMException ....
TM Deployment
private void readSomeItems(TMTransformExecutor aTMTransformer, String projectName, TMJDBCAdapter theTMJDBCAdapter, TMJavaAdapter theTMJavaAdapter, String nameToGet) throws TMException { aTMTransformer.addKeyValuePair(LOOKUPNAME,nameToGet); net.etltm.qp.TMQuerySet sourceQuery=net.etltm.qp.TMQueryObjectsFactory.createTMQuerySet("Person"); boolean filtered= aTMTransformer.interpretMapFilter(projectName,sourceQuery,null); TMDHIterator aTMDHIterator= theTMJDBCAdapter.getObjectsOfInterest(sourceQuery); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } if (filtered) { System.out.println("snapshot for \""+nameToGet+"\""); SimpleUtilities.showJavaObjects(theTMJavaAdapter,"samples.deployment.examples.tgt.Play er"); System.out.println(""); } else { System.out.println("no filtering is defined"); } }
Java to Java (with Error Handler) Example This example is the same as the first Java to Java example except that it introduces an error handler. If no error handlers are defined, then errors are regarded as unhandled and sent to the general-purpose output text stream. Once an error handler has been added, the error is transformed by the code in the error handler and placed into the output device specified by the error handler (which may be XML, Java or a database). The error will not be sent to the general-purpose output text stream. Any number of error handlers may be defined. Every error is handled by every error handler in turn. The only exception to this is if an error should occur within an error handler. In this case the error is not handled by the error handler which threw the exception but by all the other error handlers . It is important not to confuse the concept of an error handler - that is to say a project dedicated to the transformation of errors from TMError into any output target - with the code contained within an SML catch error block. The two concepts are quite separate. Thus, if both an SML catch error block and an error handler exist, then error is processed in the following manner: The error is thrown and an event is created within a TM exception 66
ETL Transformation Manager The code in the catch error is executed (this may transform the error event.) The code in the error handler(s) is executed (this may transform the error event.)
TM Deployment
Note also that Error Handlers of registered with a given TMTransformer object. Thus, where the TMTransformer object creates other TMTransformer objects during the course of the execution (see next example), then those new TMTransformer objects will not inherit currently registered Error Handlers. Code
TMJavaAdapter aTgtTMJavaAdapter = null; try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 2); TMJavaAdapter aSrcTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aSrcTMJavaAdapter); aTgtTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTgtTMJavaAdapter); // add an error handler - it only needs a target aTMTransformer.addProject(errorHandlerProjectName, 1, true); TMXMLAdapter errorHandlerOutput= TMBuiltInAdaptorFactory.makeNewWriteTMXMLAdapter(aTMTransformer); aTMTransformer.setTarget(errorHandlerProjectName,errorHandlerOutput); Map params = new HashMap(); params.put(TMXMLAdapter.NAME_URL,"errorExample.xml"); errorHandlerOutput.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); // add some data to the read adapter aSrcTMJavaAdapter.addObjects(aGame.getPlayers()); aTMTransformer.setTxtLogging(System.out, TMC.eLogNONE); TMDHIterator aTMDHIterator= aSrcTMJavaAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("samples .deployment.examples.src.Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } } catch (TMException . . .
Java to Java (with embedded objects) Example This example expands the Java to Java example to show how an embedded XML stream may be forwarded from one transformation to another. This is typical of the techniques used to gain access to any embedded XML document placed within a JMS message. Basic Java Code This is identical to the Java to Java example. The only difference in the distributed examples is that the "game" object contains a populated XML string in addition to its other data. SML Code The SML code contained logic to discover the presence of the XML String and to send it to a Java function defined within translation builder.
67
ETL Transformation Manager This is the SML code:

TRANSFORM (Location) <- (Weapon) IF (myName='XML-Data') THEN X:=23; LOGS('Calling out!'); myName := MakeDOM(myColour); ELSE myName := myColour+CHARAT(myName,2); END_IF;
TM Deployment
The function, makeDOM, has the following body. The header section in italics is automatically generated by TM Design Tool. The rest of the function is basically boiler plate code to get the initial user object passed into the first transformation and then invoke a further transformation.
public static java.math.BigDecimal MakeDOM( TMContext theTMContext, java.lang.String P1) throws Exception { // get the theTMTransformer that started this transform TMTransformer theTMTransformer = theTMContext.getTMTransformer(); // now get the user's object set when they started the TMTransformer // cast it to the type we set JavatoJavaEmbeddedXML aUserObjct = (JavatoJavaEmbeddedXML) theTMTransformer.getUserReferenceObject(); // get the current write adapter TMJavaAdapter aTMJavaAdapter=theTMTransformer.getWriteTMJavaAdapter(); // start another transform - with same target and the source is the // String containing the embedded XML aUserObject.runExampleForXML(P1,aTMJavaAdapter); //user code return null; }
The method runExampleForXML is now able to create another TMTransformer object in the conventional manner. Apart from using the method setTargetAdapterFromExistingAdapter to allow it to use the existing target adapter there is nothing special about this Java code - it follows the same pattern as the previous Java code. Note that any TMExceptions thrown from this transformation will be sent back to the calling project and maybe caught by the Error Handlers defined in the calling project. Code
public void runExampleForXML(String P1, TMJavaAdapter aTMJavaAdapter) { String projectName = "XMLToJava"; TMJavaAdapter aWriteAdapter = null; try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 3); StringReader aStringReader = new StringReader(P1); Document aDocument = makeDomObj(P1); TMXMLAdapter aTMXMLAdapter = TMBuiltInAdaptorFactory.makeNewReadTMXMLAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMXMLAdapter); Map param = new HashMap (); param.put(TMXMLAdapter.NAME_ORG_W3C_DOM_DOCUMENT,aDocument); aTMXMLAdapter.setInstanceConnectionData(param); aTMTransformer.setTargetAdapterFromExistingAdapter(null, aTMJavaAdapter);
68
TM Deployment
TMDHIterator aTMDHIterator= aTMXMLAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("samples.dep loyment.examples.src.Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } } catch (TMException aTMException) { System.out.println(aTMException.toString()); return; } }
Java to Generic Example This example transforms from an array of Java objects to a set of CSV files. The CSV is written using the class 'samples.openinterfaces.csv.write.TmExampleCSVWriteAdapter', which is also supplied in the distribution.
try { TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 1); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewReadTMJavaAdapter(aTMTransformer); aTMTransformer.setSource(projectName,aTMJavaAdapter); TMAdapter aTMAdapter = new samples.openinterfaces.csv.write.TmExampleCSVWriteAdapter(); aTMTransformer.setTarget(projectName,aTMAdapter); Map params = new HashMap(); params.put(samples.openinterfaces.csv.write.TmExampleCSVWriteAdapter.DIRECTORY, AllExamplesInJava.EXAMPLE_TGT_PATH+"csv"); aTMAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources(); aTMJavaAdapter.addObjects(aGame.getPlayers()); TMDHIterator aTMDHIterator= aTMJavaAdapter.getObjectsOfInterest(TMQueryObjectsFactory.createTMQuerySet("samples.de ployment.examples.src.Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } } catch (TMException
Generic to Java Example This example transforms from a set of CSV files to an array of Java objects. The CSV is read using the class 'samples.openinterfaces.csv.read.TmExampleCSVReadAdapter', which is also supplied in the distribution.
try
69

{ TMTransformExecutor aTMTransformer = TMTransformerFactory.getNewTMTransformExecutor(this, projectName, 1); TMAdapter aTMAdapter = new samples.openinterfaces.csv.read.TmExampleCSVReadAdapter(); aTMTransformer.setSource(projectName,aTMAdapter); TMJavaAdapter aTMJavaAdapter = TMBuiltInAdaptorFactory.makeNewWriteTMJavaAdapter(aTMTransformer); aTMTransformer.setTarget(projectName,aTMJavaAdapter);
TM Deployment
Map params = new HashMap(); params.put(samples.openinterfaces.csv.read.TmExampleCSVReadAdapter. DIRECTORY, AllExamplesInJava.EXAMPLE_SRC_PATH+"csv"); params.put(samples.openinterfaces.csv.read.TmExampleCSVReadAdapter. EXTENSION, "csv"); aTMAdapter.setInstanceConnectionData(params); aTMTransformer.openProjectAndResources();
TMDHIterator aTMDHIterator= aTMAdapter.getObjectsOfInterest(net.etltm.qp.TMQueryObjectsFactory.createTMQuerySet("s amples.deployment.examples.src.Person")); while (true) { TMDH aTMDH = aTMDHIterator.getNextDH(); if (null==aTMDH) { break; } aTMTransformer.transform(aTMDH); } SimpleUtilities.showJavaObjects(aTMJavaAdapter, "samples.deployment.examples.tgt.Player"); } catch (TMException ...
Classpath
The TM software is comprised of various jars. These are installed and included in the classpath for the Design Tool. When deploying, the runtime classpath must include all necessary jar files, described in the following table: File Name ant.jar ant-launcher.jar bcel-5.2.jar commons-cli-1.2.jar connectionpool.jar db2java.zip derby.jar dom3-xml-apis.jar Required at runtime No No No No Yes Yes If reading / writing to / from DB2 database Yes If reading / writing to / from derby database Yes
70
ETL Transformation Manager dom4j.jar Yes - If reading / writing using the Excel adapter. If using Excel '91 - 2003 format files (*.xls), only poi.jar is required. No Yes Yes Yes Yes No No Yes Yes Yes No Yes - If reading / writing using the Excel adapter. If using Excel '91 - 2003 format files (*.xls), only poi.jar is required. Yes - If reading / writing using the Excel adapter. If using Excel '91 - 2003 format files (*.xls), only poi.jar is required. Yes - If reading / writing using the Excel adapter. If using Excel '91 - 2003 format files (*.xls), only poi.jar is required. Yes Yes - If reading / writing using the Excel adapter. If using Excel '91 - 2003 format files (*.xls), only poi.jar is required. No Yes No Yes Yes Yes Yes No Yes Yes - If reading / writing using the Excel adapter. If using Excel '91 - 2003 format files (*.xls), only poi.jar is required.
TM Deployment
dtdparser121.jar EccpressoAll.jar flexlm.jar fscontext.jar jdom.jar jhall.jar jlfgr-1_0.jar log4j-1.2.8.jar ofetsdo.jar ojdbc.jar packagerunner.jar poi.jar
poi-ooxml.jar
poi-ooxml-schemas.jar
providerutil.jar stax-api.jar
swing-layout-1.0.jar tmglobals.jar tmhelp.jar tmsaxon.jar tmtext.jar tmutils.jar tmvalues.jar tools.jar xml-apis.jar xmlbeans.jar
The deployment classpath must also include: The generated project jar The generated project jar of any error-handling projects (if using) User Defined Functions (udf) jars (if using) Additional classes needed by the TM Java Adapter (if reading from and/or writing to Java)
71
TM Deployment
72

TM Deployment Guide

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

TM Deployment Guide

Hochgeladen von

Copyright:

Verfügbare Formate

Real Data Integration Solutions

ETL Solutions Limited 2010 ETL Solutions Ltd.

ETL Transformation Manager

Part Number: TM418-TMD Doc. Issue: 02 (September 2010)

ETL Transformation Manager

Properties Editor Guide

Using and Creating Transform Packs

TM Runtime Error Handling

Building TM Projects Program matically

ETL Transformation Manager

ETL Transformation Manager

ETL Transformation Manager

ETL Transformation Manager

ETL Transformation Manager

ETL Transformation Manager

Using TM Deployment Tool

Figure 1 Deployment Tool Dialog

ETL Transformation Manager

ETL Transformation Manager Target Type Select the target type.

ETL Transformation Manager

Figure 8 New Task Error Handlers 6

ETL Transformation Manager

Figure 9 Error Handlers Project Name

Specify the target model type:

Figure 10 Error Handlers Target Type

ETL Transformation Manager

Specify the target configuration:

Figure 11 Error Handlers Target Configuration

Figure 12 Error Handlers Project Display

Press Next to go to the Task Name page.

ETL Transformation Manager

Task Name Enter a name for the Task.

ETL Transformation Manager

ETL Transformation Manager

Task Name Page Supply a name for the Task Name.

Completing the New Main Method Task Wizard

ETL Transformation Manager

ETL Transformation Manager

ETL Transformation Manager

ETL Transformation Manager

Figure 24 Task History

ETL Transformation Manager

ETL Transformation Manager

Command Line Deployment

ETL Transformation Manager

Pure Java Deployment

ETL Transformation Manager

Generation of Java Code

TMTransformer / TMTransformExecutor: Overview of Operation

ETL Transformation Manager

ETL Transformation Manager

ETL Transformation Manager

TMTransformExecutor - Main Loop

ETL Transformation Manager

ETL Transformation Manager

anIterator=adapter.getObjectsOfInterest(aTMSourceQuery); Getting all domains This can be done with:

ETL Transformation Manager Using TM adapters outside TM

Sessions Divisions and Partitions

ETL Transformation Manager

Transforms from $Document, $PreDocument and $PostDocument

Running the Samples

ETL Transformation Manager

Short-Lived Deployment Examples

ETL Transformation Manager