White Paper: Migrating an Oracle Applications 10gR2 Database from Solaris to Linux

Author: Tim Watson, June 2009

Overview
This document covers the process of migrating an Oracle 10gR2 database from the Solaris operating system to the Linux operating system. The database that this process was executed on supported Oracle Applications 11.5.10.2. There were additional steps required to accommodate the Apps environment, but much of this process can be applied to any Oracle 10gR2 database. Many of the scripts are provided by patches applied to the Source application system. These scripts are useful even if not running an Apps database. The patches can be downloaded and extracted (unzip). Once extracted, directories under the patch number will contain the scripts which can be leveraged.

Background
There isnt a migration utility (script or DBUA) to perform a cross platform migration. To change platforms require the database instance be re-built and / or the data moved using one of the following methods: 1. 2. 3. Export/Import or Datapump facilities. All versions support Export/Import but for Datapump, 10.1.0.2 or higher is required Transportable Tablespaces 10G or Later RMAN Convert Database functions. 10G or Later

The choices available will depend on BOTH the OS and Oracle versions of both the source and destination. The RMAN Convert Database functions only work when the Endian format of the source and target are the same. Solaris has an Endian format of Big and Linux has an Endian format of little, so this was not an option. Transportable Tablespaces has a convert function to convert from one Endian format to another, but there are limitations (metalink note 371556.1). Because of the complexity of the data within an oracle applications database and because of some of limitations, Transportable Tablespaces was ruled out as a viable option. That left option 1 and since the database is 10gR2, the datapump utilities will be used.

Environment
Source: Operating System: Endian Format: Database: Apps Version: Size: Storage: Target: Operating System: Endian Format Database: Apps Version: Size: Storage: RHEL 5, 2.6.18-53.el5, 64-bit Little 10.2.0.4 11.5.10.2 90GB NAS (NetApp) SunOS 5.8, 64-bit Big 10.2.0.2.0 11.5.10.2 90 GB DAS

1. Prepare the Source System

1.1 Perform the steps in the "Before the Database Installation" subsection of Section 1 of the Oracle Applications Release 11i with Oracle Database 10g Release 2 (10.2.0) Interoperability Notes. Do not export the OLAP analytical workspaces. Verify that you have at least 1.5 GB of free SYSTEM tablespace. Deregister the current database server (didn't execute this until ready to execute autoconfig on the new linux server) perl $ORACLE_HOME/appsutil/bin/adgentns.pl appspass=apps contextfile=$CONTEXT_FILE -removeserver Update application tier context file with new database listener port number s_dbhost New database hostname s_dbdomain New database domain name s_db_serv_sid New database SID s_dbport New database listener port s_apps_jdbc_connect_descriptor NULL Don't execute autoconfig at this time, only update context file. Autocofig will be executed later. 1.2 1.3 Apply the Applications consolidated export/import utility patch (4872830) - adpatch Apply latest Applications database preparation scripts patch (7225862) - do not use adpatch, Copy the files located under $PATCH_TOP/admin directory to your $APPL_TOP/admin directory Generate target database instance creation script adcrdb.sql SQL> sqlplus system/<system password> @$AD_TOP/patch/115/sql/adclondb.sql 10 script generates adcrdb.sql and adpostcrdb.sql Record Advanced Queue settings Copy the auque1.sql script from the $AU_TOP/patch/115/sql directory on the source administration server node to the source database server node SQL> connect / as sysdba; SQL> @auque1.sql This will generate auque2.sql in the current directory. Will be used later on target. Remove rebuild index parameter in spatial indexes SQL> select * from dba_indexes where index_type='DOMAIN' and upper(parameters) like '%REBUILD%'; SQL> alter index <index name> rebuild parameters <parameters> where <parameters> is the original parameter set without the rebuild_index parameter Purge recycle bin Apps environments, by nature, create and drop tables during normal processing. Over time, 1000s of tables can end up in the recyclebin. It is recommened to purge the recyclebin before export to improve the export/import performance and to reduce required storage. SQL> purge dba_recyclebin; 1.8 Fix potential import issue During test import runs, you might find storage issues that can be corrected before export. Below is an example of one issue that we encountered. alter table applsys.DR$FND_LOBS_CTX$I storage(maxextents unlimited); -had to do this step again during import.

1.4

1.5

1.6

1.7

1.9

Datapump Import Errors ORA-39125 in KUPW$WORKER.PUT_DDLS with LPX-00007 (Note: 554701.1) We encountered an error when importing triggers that turned out to be a bug in 10.2.0.2. The issue has to be corrected in the source database before export. We applied patch 6026088 for 10.2.0.2 mentioned in note 554701.1. This patch did not fix the issue. The released patch 6753556 for 10.2.0.3 was confirmed to be valid for a 10.2.0.2 database, so we applied that patch and it did fix the issue. (Issue was not encountered in a 10.2.0.4 database). Applied patch 6026088 following steps in the readme Issue was not fixed Applied patch 6753556 following steps in the readme Issue was fixed Recommendation is to upgrade to 10.2.0.4 before performing export to avoid bugs.

2. Prepare the Target Database System

2.1 Verify RHEL 5 installation requirements for 10gR2 (Metalink note 421308.1) Physical Memory: minimum 1GB RAM, grep MemTotal /proc/meminfo Swap Space: 2GB RAM = 1.5 times RAM, 8GB RAM = RAM, >8GB RAM = 0.75 times RAM, grep SwapTotal /proc/meminfo Disk: 400 MB of disk space in the /tmp directory, 1.5 GB - 3.5 GB of disk space for Oracle software Kernel: 2.6.18-8 (or later), uname -r SELinux: Must be disabled, /usr/sbin/getenforce, /usr/sbin/sestatus, /usr/sbin/selinuxenabled; if [ $? -ne 0 ]; then echo "DISABLED"; else echo "ENABLED"; fi Change the SELINUX value to "SELINUX=disabled", reboot. rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n" RPM: binutils-2.17.50 (x86_64) compat-db-4.2.52 (x86_64) compat-libstdc++-296(i386) compat-libstdc++-33-3.2.3 (x86_64) compat-libstdc++-33-3.2.3(i386) control-center-2.16.0 (x86_64) gcc-4.1.1 (x86_64) gcc-c++-4.1.1 (x86_64) glibc-2.5-12 (x86_64) glibc-2.5-12 (i686) glibc-common-2.5-12 (x86_64) glibc-devel-2.5-12 (x86_64) glibc-devel-2.5-12(i386) glibc-headers-2.5-12 (x86_64) ksh-20060214-1.4 (x86_64) libaio-0.3.96 (x86_64) libgcc-4.1.1(i386) libgcc-4.1.1(x86_64) libgnome-2.16.0 (x86_64) libgnomeui-2.16.0 (x86_64) libgomp-4.1.1 (x86_64) libstdc++-4.1.1 (x86_64) libstdc++-devel-4.1.1 (x86_64)

libXp-1.0.0-8 (i386) make-3.81 (x86_64) sysstat-7.0.0 (x86_64) Kernel Parms: /etc/sysctl.conf , sysctl -p kernel.shmall = 2097152 kernel.shmmax = 2147483648 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 262144 net.core.wmem_default = 262144 net.core.wmem_max = 262144 net.ipv4.tcp_wmem = 262144 262144 262144 net.ipv4.tcp_rmem = 4194304 4194304 4194304 fs.file-max = 65536

semmsl 250 /proc/sys/kernel/sem semmns 32000 semopm 100 semmni 128 shmall 2097152 /proc/sys/kernel/shmall shmmax Half the size of physical memory (in bytes) /proc/sys/kernel/shmmax shmmni 4096 /proc/sys/kernel/shmmni file-max 65536 /proc/sys/fs/file-max ip_local_port_range Minimum:1024,Maximum: 65000 /proc/sys/net/ipv4/ip_local_port_range rmem_default 262144 /proc/sys/net/core/rmem_default rmem_max 262144 /proc/sys/net/core/rmem_max wmem_default 262144 /proc/sys/net/core/wmem_default wmem_max 262144 /proc/sys/net/core/wmem_max Create Oracle users and groups Verify mounts and ownership Limits: /etc/security/limits.conf oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 /etc/pam.d/login - session required pam_limits.so /etc/profile if [ $USER = "oracle" ]; then ulimit -u 16384 ulimit -n 65536 fi /etc/nsswitch.conf verify using files, check hostname, check domainname (should return none), /etc/hosts fully qualified name

DNS:

OUI:

./runInstaller -ignoreSysPrereqs <---required because Oracle 10.2 expects OS to be RHEL 3 or RHEL/OEL 4 Warnings: ignore warnings about prereqs and un-certified as long as previous steps have been followed.

2.2

Create target Oracle 10g Oracle home (disk install and patch to current target level) Download software from edelivery.oracle.com, 10.2.0, Companion CD, 10.2.0.4 (6810189) ./runInstaller -ignoreSysPrereqs Choose the Enterprise Edition installation type select the options not to upgrade an existing database and to install the database software only. You may ignore the errors when running utlu102i.sql as documented in 370825.1. Install Oracle Database 10g Products from the 10g Companion CD (./runInstaller) Perform 10.2.0.4 patch installation tasks unzip and extract the 10.2.0.4 (6810189) SELECT version FROM v$timezone_file (returned 3, need to follow note 553812.1) (executed utltzpv4.sql, indicated nothing to do) ./runInstaller cd $ORACLE_HOME/install, ./changePerm.sh Create nls/data/9idata directory, perl $ORACLE_HOME/nls/data/old/cr9idata.pl Create Native compile directories cd $ORACLE_HOME/plsql mkdir nativelib cd nativelib i="0" while [ $i -lt 150 ]; do mkdir d$i; i=$[$i+1]; done Create the target initialization parameter file and CBO parameter file copy init<sid>.ora and ifilecbo.ora from source to target ($ORACLE_HOME/dbs) refer to note 216205.1 for init settings update initialization parameters involving the db_name, control_files, and directory structures added: *._b_tree_bitmap_plans = FALSE event='10298 trace name context forever, level 32' -fixes issues with NFS and data pump removed: *.query_rewrite_enabled='true' *.undo_retention=900 Comment out the parameters undo_tablespace and undo_management (these will be added after running adcrdb.sql)

2.3

2.4

Create the target database instance Copy the adcrdb.sql script, generated in Section 1, from the source administration server node to the target database server node. Update the script with any necessary changes to the directory structures for the log file(s), data file(s), or tablespaces, reflecting the layout of the target database server node changed directories changed National Language characterset to UTF8

changed PCTINCREASE 0 changed TEMP location and increase to 6GB Verify ORACLE_HOME, ORACLE_SID, and ORA_NLS10 are set to target export ORACLE_HOME=/u01/app/oracle/TEST/product/10.2.0 export ORACLE_SID=TEST export ORA_NLS10=/u01/app/oracle/TEST/product/10.2.0/nls/data/9idata export PATH=$ORACLE_HOME/bin:$PATH . oraenv SQL> connect / as sysdba; SQL> spool adcrdb.log; SQL> startup nomount; SQL> @adcrdb.sql SQL> exit; Add the parameters undo_tablespace and undo_management to the initialization parameter file Add the parameters that pertain to the native compilation where specified if not already there Bounce database 2.5 Copy database preparation scripts to target Oracle home Copy from the $APPL_TOP/admin directory on the source to $ORACLE_HOME/appsutil/admin on the target (can get these directly from patch 7225862) addb1020.sql adsy1020.sql adjv1020.sql admsc1020.sql The remarks section at the beginning of each script contains additional information Each script creates a log file in the current directory Set up the SYS schema sqlplus "/ as sysdba" @$ORACLE_HOME/appsutil/admin/addb1020.sql Set up the SYSTEM schema sqlplus system/<system password> @$ORACLE_HOME/appsutil/admin/adsy1020.sql Install Java Virtual Machine sqlplus system/<system password> @$ORACLE_HOME/appsutil/admin/adjv1020.sql Install other required components sqlplus system/<system password> @$ORACLE_HOME/appsutil/admin/admsc1020.sql FALSE SYSAUX TEMP Run adpostcrdb.sql script Copy the adpostcrdb.sql script, generated in Section 1, from the source to the target sqlplus system/<system password> @adpostcrdb.sql Note: system didn't have rights - had to execute as sysdba Disable automatic gathering of statistics Copy $APPL_TOP/admin/adstats.sql from the source to the target sqlplus "/ as sysdba" SQL> shutdown normal;

2.6

2.7

2.8

2.9

2.10

2.11

SQL> startup restrict; SQL> @adstats.sql SQL> exit; 2.12 Back up the target database instance (get a backup before starting import in case you have to start over)

3. Export the Source Database

3.1 Create the export parameter file Copy $AU_TOP/patch/115/import/auexpdp.dat from the source to the directory on the database server node where the export dump files are to be created Modify the file to reflect the source environment and other customized parameters. Parameter Description Template Value directory directory where the export dump files will be created dmpdir dumpfile export dump file name(s) aexp%U.dmp filesize export dump file size 1GB log log file name expdpapps.log interMedia, OLAP, and Data Mining schemas are not exported Ensure that the schema names in the exclude parameters reflect those in your database. Create a directory in the system schema that corresponds to the directory specified in the template. sqlplus system/<system password> SQL> create directory dmpdir as '/patches/linux_migration/export/TEST'; Comment out or remove the transform parameter. It is used only for the import process. Do not change the other parameters. 3.2 3.3 3.4 Shut down Applications server processes (stop all application processes but leave database and listener up) Back up AZ table data (may need to upgrade to AZ.H.DELTA.1) Note: 402785.1 Grant exempt access policy privilege to source system schema SQL> grant EXEMPT ACCESS POLICY to system; Export the Applications database instance expdp system/<system password> parfile=<export parameter file name> Revoke exempt access policy privilege from source system schema SQL> revoke EXEMPT ACCESS POLICY from system;

3.5

3.6

4. Import into the Target Database

4.1 Create the import parameter file Copy the export parameter file you created in Section 1 from the source to target Remove the exclude parameters. Remove the filesize parameter. Change the name of the log file. Uncomment the transform parameter. Create a directory in the system schema with the name set to the directory specified in the template $ sqlplus system/<system password> SQL> create directory dmpdir as '/patches/linux_migration/export/TEST'; Copy the export dump files (copy from source to target if needed)

4.2

4.3

Import the Applications database instance $ impdp system/<system password> parfile=<import parameter file name> Issue encountered with import: ORA-01631: max # extents (505) reached in table APPLSYS.DR$FND_LOBS_CTX$I alter table applsys.DR$FND_LOBS_CTX$I storage(maxextents unlimited); ALTER USER "PORTAL30_PUBLIC" GRANT CONNECT THROUGH "PORTAL30" ALTER USER "PORTAL30_SSO_PUBLIC" GRANT CONNECT THROUGH "PORTAL30_SSO" KOREAN_LEXER (created but failed) sqlplus connect / as sysdba @?/ctx/sample/script/drkorean.sql The following domain indexes where still invalid after executed the drkorean.sql script: "CS"."CS_FORUM_MESSAGES_TL_N3" "CS"."CS_FORUM_MESSAGES_TL_N4" "CS"."CS_INCIDENTS_ALL_TL_N1" "CS"."CS_KB_ELEMENTS_TL_N2" "CS"."CS_KB_SETS_TL_N3" "CS"."CS_KB_SOLN_CAT_TL_N1" "CS"."SUMMARY_CTX_INDEX" "ICX"."ICX_QUES_CTX" Steps to correct invalid domain indexes: cd $CS_TOP/patch/115/sql sqlplus apps/<appspass> @cskbctxp.sql APPS CS CTXSYS Login as System Administrator. Submit the concurrent request "Knowledge Management Forum Messages Index Synchronization" using mode "DR". Also submitted the following jobs using mode DR Knowledge Management Category Index Synchronization Knowledge Management Solution Index Synchronization Knowledge Management Statement Index Synchronization @cssrcrix.sql APPS CS CTXSYS @cssrctxp.sql APPS CS CTXSYS @cssrtxtx.sql CS Can Ignore Errors with triggers below - tables are obsolete and not used. RA_ADDRESSES_T1_VEH RA_ADDRESSES_BRIU

4.4

5. Finishing Tasks on the Target System

5.1 Reset Advanced Queues Copy auque2.sql that was generated in Section 1 from the $ORACLE_HOME/appsutil/admin directory of the source to the target. $ sqlplus /nolog

SQL> connect / as sysdba; SQL> @$ORACLE_HOME/appsutil/admin/auque2.sql 5.2 Perform post-import steps outlined in the Interoperability document (362203.1)x Start the new database listener if not already started Run adgrants.sql copy $APPL_TOP/admin/adgrants.sql from apps tier to the database tier sqlplus "/ as sysdba" @adgrants.sql <APPLSYS schema name> Grant create procedure privilege on CTXSYS Copy $AD_TOP/patch/115/sql/adctxprv.sql from the apps tier to the database tier $ sqlplus apps/<APPS password> @adctxprv.sql <SYSTEM password> CTXSYS Implement and run AutoConfig edit context file on apps tier if you haven't already to reflect new linux host create tnsnames.ora and listener.ora for rdbms and start listner (need to do this until autoconfig is enabled) exec FND_NET_SERVICES.REMOVE_SERVER('<SID>', '<sid>_<HOST>') execute autoconfig on the apps tier adgendbc.sh will fail until autoconfig is executed on the dbtier. configure autoconfig on the db tier On the Application Tier (as the APPLMGR user):x Log in to the APPL_TOP environment (soxurce the environment file) Create appsutil.zip file perl <AD_TOP>/bin/admkappsutil.pl This will create appsutil.zip in $APPL_TOP/admin/out . On the Database Tier (as the ORACLE user): Copy or FTP the appsutil.zip file to the <RDBMS ORACLE_HOME> cd <RDBMS ORACLE_HOME> unzip -o appsutil.zip Review database context file and make any necessary edits perl adbldxml.pl tier=db edit context file OUTBOUND_DIR /u01/app/oracle/TEST/product/10.2.0/appsutil/outbound/TEST execute adchkcfg.sh contextfile=<CONTEXT> if you want a report of what changes will be made by running autoconfig execute autoconfig on db tier cd <RDBMS ORACLE_HOME>/appsutil/bin adconfig.sh contextfile=<CONTEXT> edit "s_apps_jdbc_connect_descriptor" with NULL verify /etc/hosts is correct with domain for database server. execute autoconfig on the apps tier again and verify adgendbc.sh completed successfully. Gather statistics for SYS schema $ sqlplus "/ as sysdba" SQL> shutdown normal; SQL> startup restrict; SQL> @adstats.sql SQL> shutdown normal; SQL> startup; SQL> exit; Re-create custom database links (checks links for changes in host name) Add test, prod, etc to tnsnames (compare to tnsnames from source system) Apply Oracle Receivables patch 5753621 (adpatch)

5.3 5.4

Restore AZ table data (Note: 402785.1) Create OWA_MATCH package Create subdirectory $ORACLE_HOME/appsutil/admin/OWS. Download Patch 3835781 to any patch directory and unzip it. Copy the following files from this patch directory to $ORACLE_HOME/appsutil/admin/OWS. patch.sql modplsql/owa/pubmat.sql modplsql/owa/privmat.sql Set the environment to point to ORACLE_HOME on the database server. From directory $ORACLE_HOME/appsutil/admin/OWS execute the following commands: $ sqlplus /nolog SQL> connect / as sysdba SQL> @patch.sql Create ConText and Spatial objects (script generates driver file that you name) perl $AU_TOP/patch/115/driver/dpost_imp.pl <driver file> apply <driver file> using adpatch Compile invalid objects $ sqlplus sys/<sys password> @$ORACLE_HOME/rdbms/admin/utlrp.sql Maintain Applications database objects Using adadmin: Compile flexfield data in AOL tables Recreate grants and synonyms for APPS schema Start Applications server processes Create DQM indexes Create DQM indexes by following these steps: Log on to Oracle Applications with the "Trading Community Manager" responsibility Click Control > Request > Run Select "Single Request" option Enter "DQM Staging Program" name Enter the following parameters: Number of Parallel Staging Workers: 4 Staging Command: CREATE_INDEXES Continue Previous Execution: NO Index Creation: SERIAL Click "Submit" Re-create DBMS jobs A feature of 10gR2 is that during import, all dbms_jobs are created under the user that is used for the import process. In this case, the system user will own all the jobs. The DBMS_JOB.USER_EXPORT procedure can be used to create a call to DBMS_JOB.ISUBMIT to recreate the jobs. Below is an example for one job: set array 1 var job number var jobstring VARCHAR2(2000) col jobstring format a1000 word_wrap col what format a25 word_wrap

5.5

5.6

5.7

5.8 5.9

5.10

col interval format a20 ALTER SESSION SET NLS_DATE_FORMAT='YYYY:MM:DD:HH24:MI:SS'; BEGIN DBMS_JOB.USER_EXPORT(2482,:jobstring); END; / print jobstring connect SYSTEM/<password> exec dbms_job.remove(2482); connect <original job owner>/<password> execute jobstring printed above 5.11 Re-Size Datafiles The script provided by the application patches that creates the database shell will size the datafiles exactly as they are on the source system. The import process should tighten up the extents within the tablespace, so there may be additional space that can be reclaimed by resizing the datafiles. The following script will indicate which files can be resized: set verify off column file_name format a50 word_wrapped column smallest format 999,990 heading "Smallest|Size|Poss." column currsize format 999,990 heading "Current|Size" column savings format 999,990 heading "Poss.|Savings" break on report compute sum of savings on report column value new_val blksize select value from v$parameter where name = 'db_block_size' / select file_name, ceil( (nvl(hwm,1)*&&blksize)/1024/1024 ) smallest, ceil( blocks*&&blksize/1024/1024) currsize, ceil( blocks*&&blksize/1024/1024) ceil( (nvl(hwm,1)*&&blksize)/1024/1024 ) savings from dba_data_files a, ( select file_id, max(block_id+blocks-1) hwm from dba_extents group by file_id ) b where a.file_id = b.file_id(+) / column cmd format a75 word_wrapped select 'alter database datafile '''||file_name||''' resize ' || ceil( (nvl(hwm,1)*&&blksize)/1024/1024 ) || 'm;' cmd from dba_data_files a, ( select file_id, max(block_id+blocks-1) hwm

from dba_extents group by file_id ) b where a.file_id = b.file_id(+) and ceil( blocks*&&blksize/1024/1024) ceil( (nvl(hwm,1)*&&blksize)/1024/1024 ) > 0 / 5.12 Unlock stats on AQ tables select owner, table_name, stattype_locked from dba_tab_statistics where stattype_locked is not null; exec dbms_stats.unlock_table_stats('APPLSYS', 'AQ$_WF_CONTROL_P'); 5.13 DB consolse setup: If you arent using Enterprise Manager Grid Control, you may want to setup the DB console for monitoring in your new database $ $ $ $ $ emca -repos create emca -config dbcontrol db emctl start dbconsole emctl stop dbconsole emctl status dbconsole

Summary
Again, many of these steps are specific to an Oracle Applications 11i database, but many of the scripts and procedures can be leveraged for any 10gR2 database. The important thing to remember is that there are a lot of little things that need to be verified that might be specific to your environment. Counts of objects and the number of invalid objects should be compared between the source and target. Also, full functional regression testing should be performed to verify that the application is functioning properly. For timing purposes, the export of this 90 GB database took 6 hours, while the import took 7 hours.

References
733205.1 243304.1 371556.1 331221.1 554701.1 - Migration of Oracle Database Instances Across OS Platforms - 10g : Transportable Tablespaces Across Different Platforms How to move tablespaces across platforms using Transportable Tablespaces with RMAN - 10g Export/Import Process for Oracle Applications Release 11i - Datapump Import Errors ORA-39125 in KUPW$WORKER.PUT_DDLS with LPX-00007

About the Author

Tim Watson is Principal Consultant for SageLogix, Inc. He was previously with Oracle Consulting Services. He has worked with Oracle since version 7, and has been consulting to Oracle clients for over 10 years. His primary consulting focus is on implementing and maintaining Oracle Applications environments.