Beruflich Dokumente
Kultur Dokumente
10G
AND SALESFORCE.COM
INTEGRATION
PART 1
Written by:
Christian Screen
ArtOfBI.com (The Art of Business Intelligence)
Version 1 (2010/03/22)
The Art of BI (http://www.artofbi.com)
Oracle Data Integrator (ODI) and SalesForce.com Integration
Table of Contents
ABOUT THIS TUTORIAL .......................................................................................................................................... 4
PREFACE .............................................................................................................................................................. 4
OBJECTIVE ............................................................................................................................................................ 4
SOFTWARE DEPENDENCIES ....................................................................................................................................... 4
PROJECTED COMPLETION TIME .................................................................................................................................. 4
PRE-REQUISITES ..................................................................................................................................................... 4
INSTALL THE ODI 10.3.5.5 PATCH ........................................................................................................................... 6
DIRECTION ............................................................................................................................................................ 6
INSTALL THE JAVA JDK ........................................................................................................................................... 7
DIRECTION ............................................................................................................................................................ 7
FILE DOWNLOAD & INSTALLATION .............................................................................................................................. 7
SETTING-UP THE SAMPLE DATA SOURCE ............................................................................................................... 8
DIRECTION ............................................................................................................................................................ 8
TUTORIAL FILE DOWNLOAD & INSTALLATION................................................................................................................. 8
SETTING-UP SALESFORCE.COM 3RD PARTY SOFTWARE........................................................................................... 9
DIRECTION ............................................................................................................................................................ 9
SOFTWARE DOWNLOADS.......................................................................................................................................... 9
INSTALL TUTORIAL FILES / DATA PROCEDURE .............................................................................................................. 10
TEST SALESFORCE.COM API LOGIN CREDENTIALS ......................................................................................................... 10
CREATE WORK REPOSITORY DATABASE SCHEMA ................................................................................................ 13
DIRECTION .......................................................................................................................................................... 13
CREATE SCHEMA .................................................................................................................................................. 13
SETUP TOPOLOGY MANAGER .............................................................................................................................. 14
DIRECTION .......................................................................................................................................................... 14
CREATE REPOSITORY CONNECTION LOGIN .................................................................................................................. 14
CREATE WORK REPOSITORY .................................................................................................................................... 15
ADD SALESFORCE.COM TECHNOLOGY ........................................................................................................................ 17
CREATE AN AGENT ............................................................................................................................................... 18
CREATE A CONTEXT ............................................................................................................................................... 19
CREATE A LOGICAL AGENT ...................................................................................................................................... 20
CONFIGURE THE AGENT ....................................................................................................................................... 22
DIRECTION .......................................................................................................................................................... 22
CONFIGURE AGENT BATCH FILE ............................................................................................................................... 22
CREATE AGENT START/STOP BATCH SCRIPTS ............................................................................................................... 23
SETUP DATA SERVERS & DATA STORES ................................................................................................................ 25
ADD FILE DATA SERVER ......................................................................................................................................... 25
ADD SALESFORCE.COM TECHNOLOGY DATA SERVER ..................................................................................................... 26
PREPARE THE INTEGRATION IN DESIGNER ........................................................................................................... 28
DIRECTION .......................................................................................................................................................... 28
INITIAL DESIGNER CONNECTION ............................................................................................................................... 28
CREATE A PROJECT................................................................................................................................................ 29
IMPORT THE KNOWLEDGE MODULES ......................................................................................................................... 29
CREATE A FILE SYSTEM LOG FOLDER .......................................................................................................................... 30
CONFIGURE RKM OPTIONS .................................................................................................................................... 30
CONFIGURE IKM OPTIONS ..................................................................................................................................... 30
CREATE THE MODELS ............................................................................................................................................ 31
GO, GO INTEGRATION.......................................................................................................................................... 35
DIRECTION .......................................................................................................................................................... 35
ADD FILE DATASTORE ............................................................................................................................................ 35
CREATE THE INTERFACE .......................................................................................................................................... 36
VERIFY SALESFORCE.COM CONTACTS LOAD......................................................................................................... 40
DIRECTION .......................................................................................................................................................... 40
VERIFY ............................................................................................................................................................... 40
CONCLUSION ....................................................................................................................................................... 42
REFERENCES......................................................................................................................................................... 42
OBJECTIVE
This tutorial was developed for users with an intermediate understanding of ODI, data
warehousing, ETL (or ELT as ODI labels itself), and Salesforce.com. During this exercise a
single comma-separated values (CSV) file containing sales contacts will be interfaced with ODI
and pushed into the user’s SFDC contacts list. The most important part of this tutorial is getting
the set-up steps configured correctly. There may seem to be a lot of initial steps but without
them the integration is not possible. Please follow the first several steps very closely.
SOFTWARE DEPENDENCIES
In order to use this tutorial the following software must be installed and functioning properly:
• Windows Server 2003
• Oracle database (Oracle XE recommended)
• Oracle Data Integrator (ODI) Client/Server
• SalesForce.com Software
PRE-REQUISITES
The SalesForce.com KMs provided by ODI are rough, initial releases and have not had much in
the way of updates since 2006. Due to this lapse, the following items must be secured prior to
continuing in this exercise. The instruction outlined in this tutorial was created for a user on a
Windows Server 2003 environment. By design the SalesForce.com KMs utilize a cmd.exe
execution in the KM XML files to run JAR files. Clearly this can be updated by the developer
and saved for another environment such as Linux outside of this tutorial but this tutorial lives in
a Windows environment. The following pre-requites must be fulfilled before continuing:
• Installed Oracle Data Integrator (ODI Quality or Profiler not necessary)
o Assumes a Master Repository is already configured
• Java JDK 1.5 or Greater
• Download and install the ODI 10.3.5.5 patch
o Patch available on Oracle Support or Mirrored here
o The most important part of this patch is the SalesForce.com RKM
• A SalesForce.com Developer Edition Account (Free)
o Registration Page http://www.developerforce.com/events/regular/registration.php
o Obtain (reset) security token for API login access
Setup > Reset Security Token
As part of the prerequisites for this tutorial your ODI environment should be setup with a
confirmed connection to a Master Repository. It is assumed that you as the conductor of this
exercise have Administrator-like rights and privileges to all environments surrounding the
interaction. Although full Administrator rights are not necessary for all steps, many items such
as setting folder security, etc. will require a high-level of privileges.
1. Download and install the JDK 1.5 or greater on the same machine as the ODI installation
from, http://java.sun.com/javase/downloads/widget/jdk6.jsp
2. Open your computer’s System Properties prompt by right-clicking on My Computer from
Windows Explorer.
3. Select the Advanced tab
4. Click the Environment Variables button
5. In the System Variables section, locate ODI_JAVA_HOME
6. Change its value to the path of the JDK that you just installed but do not include the bin
path. For example if the JDK installation path is C:\Program Files\Java\jdk1.6.0_16 the
enter C:\Program Files\Java\jdk1.6.0_16. Do not include the bin folder in this path as the
JAVA_HOME system variable does. ODI adds the bin path when it references the Java
executables.
7. Restart the server to ensure that this environment variable change is set.
4. Open the new folder’s permission and security tab and set “Modify” permissions on the
folder to Everyone.
5. Regardless of whether or not you were able to download the ODI 10.3.5.5 patch from
Oracle support you will need to use the adjusted SalesForce.com RKM contained in the
sample code for this article as it fixes a naming issue the patch did not. This file,
KM_RKM Salesforce.com.xml, is located in the ODI\oracledi\impexp\ folder when the
sample code zip file is extracted.
a. Make a back-up copy of the original file in the ODI directory before you
overwrite it.
b. Copy and paste the KM_RKM Salesforce.com.xml file from the sample code
extract directory into the <ODI_HOME>\oracledi\impexp\ folder of your ODI
installation. This is the default location for all ODI KMs.
The Quickstart.jar file provides driver access to ODI and will be downloaded and placed in the
driver path of the ODI install directory.
As an ancillary tool to test that your SalesForce.com API login credentials are connecting
properly you will download the ApexDataLoader.exe from SalesForce.com and install it on your
local machine. Unfortunately, there is no way to test your SalesForce.com login credentials from
within ODI. The credentials are only passed during runtime execution. This SFDC tool will
allow you to test that your login is working. It also has other extraction benefits outside of this
tutorial’s scope.
SOFTWARE DOWNLOADS
SForceDataLoader is an open-source Java based application from SFDC and can be downloaded
freely at http://sourceforge.net/projects/sforce-app-
dl/files/SforceDataLoader/7.0%20zip%20with%20upsert/sforcedataloader70.zip/download
2. Click OK.
3. From the file menu, choose Insert.
4. On the login prompt enter your SalesForce.com username which is usually your email
address. For your password you will need your concatenated SalesForce.com password
and security token. For example if your password is “password1” and your security
token is “ABCDEFGH” then your API combination password would be
“password1ABCDEFGH”. Please reset your security token via the SalesForce.com
dashboard if you do not have it ready. It will then be sent to you via email.
Once your API password has been entered, click Log in to confirm your connection. If the login
is successful you will know that your API credentials are working. These will be the credentials
used for the SalesForce.com Technology Data Server in ODI.
CREATE SCHEMA
Connect to your Oracle XE database instance using your favorite IDE or SQL*PLUS using a
system user and create a new user, ODI_WORK with a password of odiwork. Be sure that
CONNECT and RESOURCE roles are granted to this user.
For the Oracle Data Integrator Connection section please enter the following:
• Login Name: ODI_SFDC_DEMO.
• User: SUPERVISOR
• Password: SUNOPSIS
For the Database Connection (Master Repository) section please enter the database connectivity
credentials for the relational db/schema that holds your ODI Master Repository data.
Click the Test button to confirm your credentials are accurate. Click OK if a successful
connection is made.
Now on the login prompt, enter the username and password you just created for the repository
credentials. Click OK to login.
Find the tab for Repositories “AB” at the bottom and click on it.
1. In the left-pane right-click on “Work Repositories” and click Insert Work Repository.
2. You are prompted to enter information for a new Data Server connection. Enter the
following information:
• Name: ODI_SFDC_DEMO_DS
• Technology: [select the DB type of your Work Repository (ex: Oracle)]
• Instance / dblink (Data Server): [the name of your database server, ex: localhost]
• User: ODI_WORK
• Password: odiwork
3. Click the JDBC tab and enter the following based on the database and SID of your
instance:
• JDBC Driver: oracle.jdbc.driver.OracleDriver
• JDBC Url: jdbc:oracle:thin:@demomachine:1521:xe
4. Click the Test button to test the connection. When prompted with selection for an Agent
just click Test. If successful then the connection to the database is good.
5. Click OK. A prompt appears and requires you to enter the Work Repository information
which creates the ODI reference.
1. Expand Technologies.
a. If the SalesForce.com Technology is already available in your list of
Technologies you may skip this section and proceed to the next section.
2. Right-click Technologies, click Open in New Window. The new window opens as a
window pane on the right-hand side of the IDE.
3. Click the second icon “Import Technology”.
CREATE AN AGENT
An agent is required by ODI to execute most KMs and needs a server process to run under. If no
agent currently exists then one must be created. There are two types of agents. We will be
created a non-scheduler agent.
In the Physical Architecture tab, right-click Agents and click Insert Agent.
You cannot test the agent yet as we must first set up a context and a logical agent.
Click OK.
CREATE A CONTEXT
As something akin to grouping like objects, ODI has a context object. We will create one
specific to the SFDC integration. Towards the bottom of the left-pane click the Contexts tab
with the icon that looks like a mountain with a flag atop it.
• Code: ODI_SFDC_CONTEXT
4. Click OK.
The section under “rem Repository Connection Information” must be edited in order to link the
newly created Master and Work Repositories to the Agent. Change the following settings as
outlined below based on your installation and server:
set ODI_SECU_DRIVER=oracle.jdbc.driver.OracleDriver
set ODI_SECU_URL=jdbc:oracle thin:@demomachine:1521:xe
set ODI_SECU_USER=ODI_MASTER
set ODI_SECU_ENCODED_PASS=fDyXMwUP4Dv5W89gwI,p
set ODI_SECU_WORK_REP=ODI_SFDC_DEMO_WORK_REP
set ODI_USER=SUPERVISOR
set ODI_ENCODED_PASS=LELKIELGLJMDLKMGHEHJDBGBGFDGGH
The variable ODI_SECU_USER should contain the value for the Master Repository schema
user. The ODI_SECU_ENCODED_PASS variable should contain the value for the Master
Repository schema user password in an encrypted format To encrypt the password you must run
the ODI Agent encode function from the command-line. To do so, launch the MS-DOS
command prompt from Start > Run, change the directory to <ODI_HOME>\oracledi\bin\ and
execute the following:
• Agent encode [Your Password]
o The screenshot below shows several passwords being encrypted on the machine
this tutorial was written on.
The ODI_SECU_WORK_REP variable should contain the name of the Work Repository. And,
the ODI_USER and ODI_ENCODED_PASS should contain the respective credentials to log in
to the ODI Repository. If you did not stray from this tutorial using (SUPERVISOR/SUNOPSIS)
you can leave these as they are.
Create a folder in the root of your ODI_HOME path called _Agent_Scripts and save the scripts
here.
Create the first batch file script with the following code and save as start_sfdc.bat:
H:
cd\
CD ODI\oracledi\bin
agent "-PORT=20910" "-NAME=ODI_SFDC_AGENT"
Create the second batch file script with the following code and save as stop_sfdc.bat:
H:
cd\
CD ODI\oracledi\bin\
agentstop "-port=20910"
Run the startAgent.bat file you just created. You may now return to the Topology Manager
Physical Agent and test that the Agent’s connection is working.
5. Click Apply.
The Physical Schema prompt will appear. On the Definition tab the root file Directory where the
files reside must be defined. Enter the folder path that you created earlier and copied the tutorial
sample data files into. Please note that there is no browse tool so you will need to click inside
the dropdown and type in the path. Pay close attention and ensure that you enter a forward slash
(not backslash) for the path name, ex: H:/FilesStorage. Be sure to spell it correctly.
9. Click OK twice.
Please note an undocumented caveat of ODI is that the password field only accepts 35
characters in the password field. For the API password (password + security token) this
means that you may need to change your SalesForce.com password and shorten it so that it
is less than 11 characters in length as the security token is 24 characters long.
The JDBC does need to be modified as it is not necessary for this type of technology.
6. Click OK.
CREATE A PROJECT
In Designer, on the Projects tab of the left pane, click the first icon at the top for Insert Project.
For the new project prompt enter ODI_SFDC_PROJECT for the Name and Code fields.
Click OK.
On the Import Knowledge Modules… prompt, make sure that the File Import Directory is set to
the ODI_HOME\Oracledi\impexp folder. At this point select the SalesForce.com KMs from the
list below. Hold down the ctrl key to select multiple KMs in one sitting. Select the following
files from the list:
• IKM File to Salesforce (Upsert)
• RKM Salesforce.com
Click OK.
below and modify its default values replacing ODI_HOME with your ODI installation directory
which if installed on the C:\ drive will be C:\ODI:
• ERROR_FILE: ODI_HOME\_Logs\sforcedataloader.error
• JVM_PATH: ODI_HOME\jre\1.4.2\bin
• LOG_FILE: ODI_HOME\_Logs\sforcedataloader.log
• RESULT_FILE: ODI_HOME\_Logs\extract.csv
• SF_PATH: C:\\sforcedataloader
• SUCCESS_FILE: ODI_HOME\_Logs\sforcedataloader.success
Please note that in the SF_PATH value that the backslash is escaped. This is necessary due to
the programming of the KM.
In the Models view click the Insert Model Folder icon and in the new model folder prompt enter
SFDC_MODELS in the Name field. Click OK.
Now, we will create two models to be stored inside the SFDC_MODELS folder. One to hold the
flat file objects and one to hold the SalesForce.com objects.
Click the icon to Insert Model (4th yellowish icon with a plus). In the new model prompt
Definition tab enter the following:
• Name: SFDC_MODEL
• Code: SFDC_TARGET
• Technology: Salesforce.com
• Logical Schema: SFDC_SCHEMA
Last Saved: 10/8/2010 11:34 PM http://www.artofbi.com
31
The Art of BI (http://www.artofbi.com)
Oracle Data Integrator (ODI) and SalesForce.com Integration
On the Reverse tab, click the Customized radio button. Choose the following values from the
respective dropdown lists:
• Context: ODI_SFDC_CONTEXT
• Local Agent: ODI_SFDC_LOCIGAL_AGENT
• Select your KM: RKM Salesforce.com.ODI_SFDC_PROJECT
Now at the very bottom of the Reverse tab, click the Reverse button. This will begin your initial
connection via the SalesForce.com API and pull down all existing SalesForce.com objects
available under your user account.
Click “OK” or “Yes” for all prompts that follow after clicking the Reverse button.
This can take up to 5 minutes. Click OK to close the new model window.
To create the flat files model, navigate to the Models tab of the left-pane. Click the icon to
Insert Model. In the fields and dropdowns select or enter the following values:
• Name: SFDC_FLAT_FILE
• Code: SFDC_FLAT_FILE
• Technology: File
• Logical Schema: SFDC_FILE_SCHEMA
On the Reverse tab of the SFDC_FLAT_FILE model change the Context dropdown value to
ODI_SFDC_CONTEXT.
Click OK to finish and verify that your two models have been created. If the models don’t
immediately line-up under the SFDC_MODELS folder simply click each model and drag it into
the parent folder.
Go, Go Integration
DIRECTION
Establishing a project, setting up the models and reverse engineering SalesForce.com objects has
the ODI Designer prepped and ready to. Now, we will add our two data stores and create an
interface to load data into SalesForce.com.
On the Files tab, configure the settings with the following values:
• File Format: Delimited
• Heading: 1
• Field Separator: , (comma)
• Text Delimiter: “ (double quote)
You should now see the columns names from the file and the associated metadata.
Click OK to finish.
In the new Interface prompt enter i_push_contacts_to_SFDC for the Name field value and ensure
that the ODI_SFDC_CONTEXT is selected.
Check the Staging Area Different From Target checkbox and choose SFDC_FILE_SCHEMA
from the dropdown.
Click the Diagram tab. Click OK on the default help prompt if it appears.
In the Diagram we need to access the source and target datastores that were just created. Leave
the Interface prompt alone for a second and change the tab on the left-pane to the Models tab.
You will see that the new Interface prompt does not disappear.
Expand the SFDC_MODELS parent folder to expose the two SFDC models.
Expand the SFDC_FLAT_FILE model to expose the Contacts datastore. Click on the Contacts
object and drag it into the sources section of the new Interface window.
Expand the SFDC_Model to expose the underlying SalesForce.com objects. Click on the
Contact object and drag it to the Target Datastore section of the new Interface window.
When the warning prompt appears and asks “Do you want to perform an Automatic Mapping?”
click Yes. The mapping will be performed and should line-up the columns nicely.
Click the Flow tab and notice the lineage from source to target has been created. Click on the
header of the Target box to reveal its properties. Ensure that the IKM dropdown box has IKM
File to Salesforce (Upsert) selected as its value.
To execute the interface and see data loaded into your SalesForce.com account, click the
Execute button at the bottom right of the Interface window. A prompt will appear to choose the
Agent and context under which the interface should execute. Choose ODI_SFDC_CONTEXT
as the Context and ODI_SFDC_LOGICAL_AGENT as the Agent.
VERIFY
Open the ODI Operator from the Start menu by All Programs > Oracle > Oracle Data Integrator
> Operator.
On the left side of the window under Session List expand the Physical Agent node >
ODI_SFDC_AGENT. Expand the session based execution of the interface you just launched. If
all icons glow green then the execution was successful.
To view the log files, navigate to the folder ODI_HOME\_Logs and open the file prefixed with
extract.csv.i_push_contacts_to_SFDC followed by a session number using your favorite text
editor. Any errors or successes will be articulated in this file.
Finally, log into your SalesForce.com account, click on the contacts tab or navigate to your
contacts object to view the updated list.
Conclusion
Oracle Data Integrator is a great ETL (ELT) to for moving data between heterogeneous data
sources. This tutorial touched on very small portions of the tremendous power that ODI has.
This tutorial allowed an intermediate to advanced user to learn more about ODI knowledge
modules, learn more about SalesForce.com, integrate the two via a data flat file and verify its
success. This is the first part in a two part series. The next part in the series will demonstrate
how to pull data from SalesForce.com into a local data source which is quite a bit more complex
as it involves modifying the Knowledge Module itself. All technical issues and follow-up
conversation can be found on the Art of Business Intelligence blog at http://www.artofbi.com.
References
(http://www.oracle.com/technology/products/oracle-data-
integrator/10.1.3/htdocs/documentation/oracledi_setup.pdf).