What is Golden Gate?

Oracle GoldenGate enables the exchange and manipulation of data at the transaction level among multiple, heterogeneous (Oracle, DB2, SQL Server,
Ingres, MySQL) platforms across the enterprise. Oracle Goldengate can be used as a replication tool, ETL, and even as a DR solution. GoldenGate
enables us to extract and replicate data across a variety of topologies as shown the diagram below as well as the exchange and manipulation of data at the
transactional level between different databases.

Oracle GoldenGate is composed of the following components:

● Extract: The Extract process runs on the source system and is the extraction (capture) mechanism of Oracle GoldenGate.

● Data pump: A data pump is a secondary Extract group within the source Oracle GoldenGate configuration. The data pump reads this trail and sends the
data over the network to a remote trail on the target. DATA PUMP is an optional extract.

● Replicat: The Replicat process runs on the target system. Replicat reads extracted data changes and DDL changes (if supported) that are specified in
the Replicat configuration, and then it replicates them to the target database

● Trails or extract files: To support the continuous extraction and replication of database changes, Oracle GoldenGate stores the captured changes
temporarily on disk in a series of files called a trail. A trail can exist on the source or target system, or on an intermediary system, depending on how you
configure Oracle GoldenGate. On the local system it is known as an extract trail (or local trail). On a remote system it is known as a remote trail.
Trail files are created as needed during processing, and they are aged automatically to allow processing to continue without interruption for file
maintenance. By default, each file in a trail is 10 MB in size.

● Checkpoints: Checkpoints store the current read and write positions of a process to disk for recovery purposes. These checkpoints ensure that data
changes that are marked for synchronization actually are extracted by Extract and replicated by Replicat, and they prevent redundant processing. They
provide fault tolerance by preventing the loss of data should the system, the network, or an Oracle GoldenGate process need to be restarted.

● Manager: Manager is the control process of Oracle GoldenGate. Manager must be running on each system in the Oracle GoldenGate configuration
before Extract or Replicat can be started, and Manager must remain running while those processes are running so that resource management functions
are performed. Manager performs the following functions:

● Collector: Collector is a process that runs in the background on the target system. Collector receives extracted database changes that are sent across
the TCP/IP network, and it writes them to a trail or extract file. Typically, Manager starts Collector automatically when a network connection is required.
When Manager starts Collector, the process is known as a dynamic Collector, and Oracle GoldenGate users generally do not interact with it. However, you
can run Collector manually. This is known as a static Collector. Not all Oracle GoldenGate configurations use a Collector process.
Section-0

In few lines, How to do describe GoldenGate?

1. It is a replication tool.

2. Very similar to Oracle Data Guard.

3. However, Oracle Data Guard will only replicate from "Oracle to Oracle" databases.

Where as Oracle GoldenGate is "heterogeneous" replication. So, what does "heterogenous" means?

Well, it mean cross platform which means, "Oracle to SQL Server", "SQL Server to Oracle", "Oracle to DB2", "Oracle to MySQL" is possible with
Oracle GoldenGate

4. Oracle GoldenGate can also be used as a DR Site.

The following is the high level documentation of Oracle 11g R2 GoldenGate installation on 64-bit RHEL 5 u3. As time progress, I will be updating this
document. Please refer to server 210 and 211 for your reference.

So what do you need to Install Oracle GoldenGate 11g R2 to test it out?

1. VMware installed on a server
2. Install OS (CentOS 64-bit)
3. Install Packages for Oracle Database
4. Create oracle username
5. Create oinstall group
6. Configure Oracle 11g R2 Software via runInstaller
7. Install Oracle 11g R2 Database using dbca

Once you have the above done, you are ready to configure Oracle 11g R2 GoldenGate on your database server. So let us start with the basics.

Part-1 Install OS

I used OS RHEL5 u3 64-bit to build two servers. However because of the rpm requirement, it is best if you work on 64-bit CentOS 5.6 as your Operating
System. My server names are Linux-VM-159 and Linux-VM-161 which was built using automatic partitioning with all the packages during the initial
installation. Later I will be naming my database as MDB1 and MDB2 respectively.

The following group of packages can be selected:

Oracle Enterprise Linux 5.6
Desktop Environment:
● GNOME Desktop Environment

Application:
● Editors
● Graphical Internet
● Text-based Internet

Development:
● Development Libraries
● Development Tools

Servers
● Server Configuration Tools

Base System
● Administration Tools
● Base
● System Tools
● X Window System
● Make sure both server has hostnames recorded in /etc/hosts.
● Make sure times are same on both servers, if not change it. Below is an example on how to change the time and sync it with BIOS time.
date -s "12/28/2010 02:32:00"
clock -w

Part-2 Install oracle 11g R2

The following technical notes shows how to install and create 11g R2 database on CentOS. You can use this a guide to install Oracle 11.2.0.2 software on
a VMWare.
http://www.mydbanotes.com/2011/05/oracle-11g-installation-on-64-bit.html

From the above document, we are assuming that

● following directories are created and
● oracle:oinstall is given the permissions to them.

If not create and grant proper permission as below:

mkdir -p /u01 /u02 /u03 /u04 /u05 /u14 /u15 /u16 /u17 /u18 /u99
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/app/oracle
chown -R oracle:oinstall /u01 /u02 /u03 /u04 /u05 /u14 /u15 /u16 /u17 /u18 /u99
chmod -R 775 /u02 /u03 /u04 /u05 /u14 /u15 /u16 /u17 /u18 /u99

Scroll down to next page for next part.

Oracle 11g GoldenGate Installation starts from here.
Part-3 Install GoldenGate on both servers (Execute the following on BOTH Database Servers)
Create Golden Gate Home Directory
mkdir -p /u01/app/oracle/product/11.2.0/gg

Create GoldenGate Software Directory

mkdir -p /u99/software/GoldenGate/11g/64-bit

Download GG
● From your local laptop browse to http://www.oracle.com/technetwork/middleware/goldengate/downloads/index.html and download 11g GG.

● We will be downloading 11.1.0.2 zip file called “ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip” from the above site. Just accept the license
and click on the file. [you may will need your metalink credentials]
● Once the file is downloaded to your client windows machine, using “winscp” send the zip file to your database server. As per the standards in my
database environment, send the file to /u99/software/GoldenGate/11g/64-bit

Or download directly from 223 Server:

scp oracle@192.168.0.223:/u99/software/GoldenGate/11g/64-bit/ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip /u99/software/GoldenGate/11g/64-bit/.

Unzip the GG software

cd /u99/software/GoldenGate/11g/64-bit/
unzip /u99/software/GoldenGate/11g/64-bit/ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip
 Copy tar file to GG home
cp /u99/software/GoldenGate/11g/64-bit/fbo_ggs_Linux_x64_ora11g_64bit.tar /u01/app/oracle/product/11.2.0/gg/.

untar the file

cd /u01/app/oracle/product/11.2.0/gg/
tar -xvof fbo_ggs_Linux_x64_ora11g_64bit.tar

Add the GG settings in oracle’s .bash_profile at the end of the file.

vi ~/.bash_profile
export GGATE=/u01/app/oracle/product/11.2.0/gg/
export PATH=$PATH:$GGATE

Re-execute the .bash_profile

. ~/.bash_profile

Start Golden Gate

cd $GGATE
./ggsci

Create directories for GoldenGate Working directories (GoldenGate Working Directories = Golden Gate Engine)
create subdirs
Exit out of Golden Gate
exit

Section-1 Verify DB is in archivelog mode are in archivelog mode (Execute on both Source and Target)

sqlplus / as sysdba

SQL>archive log list

Database log mode Archive Mode
Automatic archival Enabled
Archive destination /u99/PrimeDG/Archive
Oldest online log sequence 1830
Next log sequence to archive 1832
Current log sequence 1832

If DB is not in archivelog log mode, see “How enable the archivelog mode”.
http://www.mydbanotes.com/2011/07/how-to-enable-archivelog-mode.html

Section-2 Add supplemental Log data for all columns in database. (Execute on both Source and Target)
SELECT SUPPLEMENTAL_LOG_DATA_MIN FROM V$DATABASE;
-- it must be YES or IMPLICIT

--IF not, enable supplemental logging by executing the following

ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;

Section-3 Turn off Recycebin (Execute on both Source and Target)

alter system set recyclebin=off scope=spfile; → This is not required in 11g

Section-4 Restart Database (Execute on both Source and Target)

shutdown immediate;
exit;
sqlplus / as sysdba
startup;
exit

Section-5 Create GGS_ADMINSchema for Oracle GoldenGate objects (Execute on both Source and Target)
sqlplus / as sysdba

create user GGS_ADMIN identified by abc123 default tablespace users temporary tablespace temp;
grant dba to GGS_ADMIN;
grant connect,resource,unlimited tablespace to GGS_ADMIN;
grant execute on utl_file to GGS_ADMIN;

Section-6 Run scripts for creating all necessary objects for support DDL replication (Execute on both Source and Target)
cd $GGATE ← Very important to be in the GG_HOME
sqlplus / as sysdba
spool /tmp/gg_setup.rtf
#1
@$GGATE/marker_setup.sql
Enter GoldenGate schema name: GGS_ADMIN

So what is marker_setup.sql script?

● It is a installation script for Oracle GoldenGate Marker table
● This script drops marker table first (if it exists), so all previous data is lost, use caution if not the first time installation
● You will be prompted for the name of a schema for the Oracle GoldenGate database objects
● The following grants will be given to GoldenGate schema
○ GRANT CONNECT, RESOURCE, DBA TO &gg_user;
● It will create a table called marker_table_name

#2
@$GGATE/ddl_setup.sql
Enter Oracle GoldenGate schema name: GGS_ADMIN
Enter mode of installation:INITIALSETUP

So what is ddl_setup.sql script?

● It is a Installation script for Oracle GoldenGate DDL Replication trigger and package
● You will be prompted for the name of a schema for the Oracle GoldenGate database objects.
● When executing this script, all sessions issuing DDL must close and reconnect including Oracle GG ones such as replicate.

Why we use the “INITIALSETUP”?

● To install or reinstall DDL replication, enter INITIALSETUP
● To upgrade DDL replication, enter NORMAL

#3
@$GGATE/role_setup.sql
Enter Oracle GoldenGate schema name: GGS_ADMIN

The above will generate SQL to create a role. Simply run the following as we have only one user GGS_ADMIN.
grant GGS_GGSUSER_ROLE to GGS_ADMIN;

So what is role_setup.sql script?

 ● It is a installation script for GoldenGate Security Role

#4
@$GGATE/ddl_enable.sql

So what is ddl_enable.sql script?

● This script will enable Goldengate DDL Replication trigger

#5
@$GGATE/ddl_pin.sql GGS_ADMIN

So what is ddl_pin.sql ?
● It pins GoldenGate DDL Replication code in shared pool
○ exec dbms_shared_pool.keep(...........);

exit;

Step-10 Enable additional logging at the table level (This needs more info)

cd $GGATE
./ggsci
DBLOGIN USERID GGS_ADMIN, PASSWORD abc123
add trandata <owner>.<tablename>
exit

At this point initial GoldenGate installation is done on the Source server.

Section 10 - Task:
Our taskis to replicate a schema called “sender” to the destination database schema called “receiver”.

Section 11 - Create schema called “Sender” on Source DB:

sqlplus / as sysdba

create user sender identified by abc123 default tablespace users temporary tablespace temp;
 grant connect,resource,unlimited tablespace to sender;

Section 12 - Create schema called “Receiver” on Destination DB:

sqlplus / as sysdba

create user receiver identified by abc123 default tablespace users temporary tablespace temp;
grant connect,resource,unlimited tablespace to receiver;

Section-14 Edit and start MANAGER process on Source DB and Destination DB

The Manager process must be running on both the source as well as target systems before the Extract or Replicat process can be started and performs a
number of functions including monitoring and starting other GoldenGate processes, managing the trail files and also reporting.
cd $GGS_ADMIN
./ggsci

Check the status of Manager process. It should be stopped.

info all

Now, it is the time to edit the parameters for MANAGER process.

Add the following to the manager process

● Add the port#
● Set the ORACLE_SID
● were or not to start Extract/Replicat process automatically, and
● how to purge old extracts for manager to start the process.
edit params mgr
PORT 7809
autostart ER *
PURGEOLDEXTRACTS ./dirdat/*, USECHECKPOINTS, MINKEEPHOURS 4
 --GoldenGate Admin username/password is required for DDL Support.
USERID ggs_admin, PASSWORD abc123

Now start the MANAGER process

Start manager
The USERID and PASSWORD parameter and required if you enable GoldenGate DDL support and this is the Oracle user account that we created for
the Manager(and Extract/Replicat) as described in the earlier tutorial.
DBLOGIN USERID GGS_ADMIN, PASSWORD abc123
DBLOGIN USERID sender, PASSWORD abc123
exit

Verify Manager process is started.

info all

Section-14.1 Create Extract on Source DB:

Now Create and extract called “ext1” and edit it’s parameter file.
edit params E1
--Extract name
extract ext1

--Set the ORACLE_SID

SETENV (ORACLE_SID=PrimeDG)

--Connection to database--
userid GGS_ADMIN, password abc123

--Hostname and port for trail--

rmthost Linux-224, mgrport 7809

--Path and name for trail--

 rmttrail /u01/app/oracle/product/11.2.0/gg/dirdat/lt

--DDL support
ddl include mapped objname Moid.T1;

--DML
table Moid.T1

Add extract called ext1

add extract ext1, tranlog, exttrailsource ./dirdat/sa, begin now, threads 1
add extract ext1, tranlog, begin now, threads 1
Note:
add extract ext1 → name of the extract is ext1
tranlog → tells GG to extract data from source’s online redo log.
exttrailsource → defines a Data Pump process for this extract.
Begin now → tells GG to start data replication immediately.
Threads 1 → In a RAC environment, it must be set to # of database instances. For single standalone server, it must be set to 1.

Add Extract TRAIL

add exttrail ./dirdat/sa, extract ext1, megabytes 50
add exttrail ./dirdat/ma, extract e1 megabytes 50
ADD EXTTRAIL ./dirdat/Ra, EXTRACT E1

Section-14.2 Create REPLICAT on DestinationDB:

--add checkpoint table to the destination database
cd $GGATE
./ggsci
edit params ./GLOBAL
 GGSCHEMA GGS_ADMIN
CHECKPOINTTABLE ggs_ADMIN.Checkpoint

--login to database from ggsci to create the check point table.

add checkpointtable GGS_ADMIN.checkpoint

--Create Replication Group

add replicat R1, exttrail ./dirdat/Ra, checkpointtable ggs_admin.checkpoint

--Create Parameter file for replicat. Add the following lines in the parameter file.

--Replicat group Name

replicat rep1

--source and target definitions

ASSUMETARGETDEFS

--target database login --

userid GGS_ADMIN, password abc123

--file for dicarded transaction --

discardfile /u01/app/oracle/product/11.2.0/gg/discard/rep1_discard.txt, append, megabytes 10

--ddl support
DDL

--Specify table mapping ---

map sender.*, target receiver.*;

Start extract process on Source:

--------------------------------
ggsci
start extract ext1
info all
Start Replicate process on Destination:
---------------------------------------
ggsci
start replicat rep1
info all

Verification:
-------------

--On Source Server

sq
create table sender.emp as select * from scott.emp;

DECLARE
v_count number := 0;
BEGIN
LOOP
EXIT WHEN v_count > 16;
INSERT INTO sender.emp
SELECT * FROM sender.emp;
COMMIT;
v_count := v_count + 1;
END LOOP;
END;
/

select count(*) from sender.emp;

--On Destination Server

sq
select count(*) from receiver.emp;

Numbers should match.

I have the following:

--On Source:
SQL> select count(*) from receiver.emp;

COUNT(*)
----------
7340032

SQL> !hostname

Linux-VM-GG-211

--On Destination:
SQL> select count(*) from receiver.emp;

COUNT(*)
----------
7340032

SQL> !hostname

Linux-VM-GG-211

That's it. You just created Uni-Directional Replication using Oracle GoldenGate.
commands
---------
history
show all
info all
start manager
DBLOGIN USERID ggs_owner, PASSWORD ggs_owner
stop manager
fc --> same like fc -l of unix?
help add extract
help add replicat

stop replicat rep1

stop stop extract ext1
SEND EXTRACT EXT1, FORCESTOP
stop manager

edit params mgr

edit params ext1
edit params rep1

Error logs
view ggsevt
vi $GGS_ADMIN/ggserr.log
tail -f $GGS_ADMIN/ggserr.log
$GGS_ADMIN/dirrpt/EXT1.rpt --Extract logs are here.

view report mgr

view report ext1
view report rep1
References:
Oracle Main GoldenGate Documentation:
http://www.oracle.com/technetwork/middleware/goldengate/documentation/index.html

Oracle GoldenGate Documentation:

http://download.oracle.com/docs/cd/E18101_01/index.htm

GG Administrator’s Guide
http://download.oracle.com/docs/cd/E18101_01/doc.1111/e17341.pdf

GG installation on SQL Server:

http://download.oracle.com/docs/cd/E18101_01/index.htm

GG installation on Oracle:
http://download.oracle.com/docs/cd/E18101_01/doc.1111/e17799.pdf