OCFS

Oracle Cluster FileSystem for Linux

Users Guide
Table of Contents
Q. Can I install the Oracle Distribution in an OCFS partition?...........21
Q. My partitions don't mount automatically during boot. What's
wrong?..............................................................................................22
Q. When running fsck.ocfs, it returns the error “WARNING: nonzero
bytes after the disk header structure”. What does it mean?............22
Q. What are the most appropriate (recommended) tool IO and capa-
city analysis of OCFS?.......................................................................22
Q. Can I use ocfs on a NAS (Network Attached Storage) device like
NetApp?............................................................................................22
Q. Can I use LVM or MD to create my OCFS filesystem on top of it?.22
Q.I want to perform some tests using RAC/OCFS, but I don't want to
spend too much money on hardware. Do I have any other solution?
.........................................................................................................22
Q.Why do I have to include the option _netdev in the fstab on Red-
Hat?..................................................................................................23
Q.What is the best way to archive the logs to an OCFS directory?...23
Q.Is OCFS supported on 64 bit platform like Itanium?......................23
Q. Can I run ocfs on non enterprise Linux distributions like RedHat 9?
.........................................................................................................23
Q.Do I need any specific version of United Linux to run OCFS?........23
Q. Can I run the latest OCFS on a plain RedHat AS 2.1, without any
errata applied?..................................................................................23
Q.What is the advantage of running ocfs against raw devices?.......24
Q.What happen if I have to change the IP Address of my systems?. 24
Q.My Network Interface Card had to be replaced. Do I need to do
something?.......................................................................................24
Q.I have a partition that is not mounted. How do I know if it is an
ocfs partition or not?.........................................................................24
Q.Can I use my OCFS partition to store regular files?.......................24
Q.How much do I lose in terms of performance compared to raw
devices?............................................................................................24
Q.How do I enable async I/O on Oracle using OCFS?........................25
Q.How do I backup my OCFS files? Can I use tar or other OS com-
mand?...............................................................................................25
Q.Is it possible to resize an existing OCFS partition?........................25
Q.I'm having problems with OCFS. How can I debug OCFS?.............25
Q.Can I run OCFS in a stand-alone system? What are the advantages
of running it?....................................................................................25
Q.I have a database running on OCFS in a stand-alone node. Why it
is so slow compared to other stand-alone systems running on ext3?
.........................................................................................................26
Q.How can I obtain more information about OCFS?..........................26
Q.I have a customized RedHat AS kernel on my system. Does Oracle
support OCFS and the RDBMS on it?................................................26
Q.How do I know if my OCFS version is officially supported by
Oracle?..............................................................................................26
Q.Is OCFS is mountable on Linux and Windows 2000 simultaneously?
.........................................................................................................26
 Introduction.

OCFS is a shared disk cluster filesystem. The current

version (version 1) released for Linux is specifically de-
signed to alleviate the need for managing raw devices.
It can contain all the oracle datafiles, archive log files
and controlfiles. It is however not designed as a gen-
eral purpose filesystem.
This document describes the steps required to install
OCFS on Linux and will also give guidelines for optimiz-
ations and some more in depth understanding of how
the filesystem works.

Downloading OCFS.

OCFS can be downloaded from

“http://oss.oracle.com/projects/ocfs” for the following
distributions : RedHat Advanced Server 2.1 and United
Linux 1.0 (Conectiva, SuSe, TuboLinux and SCO). Or-
acle officially supports the Oracle database on OCFS if
it is installed from the binary packages that are avail-
able for download.
If the user decides to download the source code and
compile it, then there will be no formal support
provided by Oracle.
In addition to the OCFS binaries, we also provide a col-
lection of utilities (cp, dd, tar and textutils) that en-
able O_DIRECT. The updated tools are recommended to
be used as they make more efficient use of the operat-
ing system in conjunction with OCFS.
Binary distributions for ia32 and ia64 can be found un-
der each one of the supported platforms.
There are basically three rpm packages to download in
See “Appendix
A” for a
order to install OCFS. Those packages are:
complete list of
files and
•OCFS-Support
description of
each one of the •OCFS-Tools
•OCFS Module
Before downloading the OCFS Module, make sure it is
compatible with the kernel version in use (uname -a).
 Installing OCFS

Installing OCFS is an easy process. After downloading

The rpm the packages, issue the following command on the dir-
command in
the right will ectory where the packages were downloaded:
install the OCFS # rpm -Uhv ocfs*.rpm
packages if
they are not This will install the support tools, the actual kernel
installed yet
and will module for the filesystem and a graphical configura-
upgrade them if tion tool.
they were

Automatically mount OCFS during boot.

After installing the OCFS packages, verify that the

module will be properly initialized on startup using the
command:
# chkconfig –list |grep -i ocfs

If the output looks like :

ocfs 0:off 1:off 2:off 3:on 4:on 5:on 6:off

Then, no action is required, but if the output doesn't

show “on” on 3,4 and 5 (rc levels), issue the following
command to enable automatic startup of the ocfs dur-
ing boot:
# chkconfig ocfs on

Configuring OCFS.

OCFS depends on a node specific configuration file.

This file is named ocfs.conf and it is located in the /etc
directory; it can be generated automatically or manu-
ally using the ocfstool. During the next sections, the
/etc/ocfs.conf file and both methods of configuring will
be described in detail. This file is needed on every
node in the cluster and it is highly recommended to
use ocfstool to configure each node.

1.1.1 - /etc/ocfs.conf file.

The /etc/ocfs.conf file can have the following paramet-

ers:
ip_address – Specify the IP Address to be used by the
OCFS DLM. The server must be able to reach all nodes
 participating on the cluster through the interface re-
lated to the IP Address specified in this field.
ip_port – Specify the port to be used by the OCFS DLM
to communicate with the other nodes in the cluster.
The port must be the same on all nodes in the cluster.
Node_name – Specify the server hostname associated
to the IP Address specified in the ip_address paramet-
er.
comm_voting – Specify which method for voting is go-
ing to be used by OCFS. If set to 0 (default), it means
that OCFS will be voting using the disk, if set to 1, it
will be using the network to vote. If the OCFS is set to
use network to vote and it becomes unavailable for
some reason, it will automatically (and transparently)
fall back to disk. Enabling comm_voting will drastically
increase performance for regular filesystem operations
such as rm, mv, mkdir etc.
guid – This parameter is automatically filled by the
ocfs_uid_gen utility and should never be manually
changed.

1.1.2 - Automatic Configuration using ocfstool.

In order to perform the automatic configuration with

ocfstool, it is necessary that the GUI environment is
properly set and enable.
Open a GUI session as root and execute the command
“ocfstool”. Make sure the DISPLAY variable is set be-
fore starting the tool.
When the ocfstool window(See Illustration 1) open, in-
voke the Generate Config(See Illustration 7) task by
either pressing the key sequence <CTRL-G> or using
the menu, clicking on Tasks>Generate Config . For
more information on the parameters, check the item
Generate Config in the ocfstool section.

1.1.3 - Manual configuration

Although the manual configuration is provided, Oracle

strongly recommends the usage of the ocfstool since it
does provide a reliable, consistent and easy way to
 properly configure OCFS.
To manually configure the OCFS, create the file /etc/ocf-
s.conf based on the sample below on each one of the
nodes participant of the cluster. Make sure the para-
meters are consistent among nodes.
# ocfs config
if using a # Ensure this file exists in /etc directory #
specific node_name = ca-test2.us.oracle.com
network, make ip_address = 10.0.0.1
sure all nodes ip_port = 7000
will have
comm_voting = 1
hostname and
IP addresses
After the file is created, execute the utility ocfs_uid_gen
with the -c argument as root in order to generate the
unique identification key necessary for the OCFS to
identify itself in the cluster. After the generation of the
uid key, the /etc/ocfs.conf file should looks like:
#
# ocfs config
# Ensure this file exists in /etc#
node_name = ca-testt2.us.oracle.com
ip_address = 10.0.0.1
ip_port = 7000
comm_voting = 1
guid = 9B2996991BCB25DF4CBB0003470CFE75

Loading ocfs.

The /etc/init.d/ocfs startup script is provided in the

package and automatically loads the OCFS module if
there is an entry for it in /etc/fstab. Using this startup
script is the preferred method to load OCFS as it does
all the verification before loading the module and
mounting the partitions.
If for some reason there still is a need of manually load
the OCFS module, just issue the command load_ocfs as
root. If the process is successfully executed, it should
show a message like:
# load_ocfs

/sbin/insmod ocfs node_name=ca-test2.us.oracle.com ip_address=10.0.0.1

ip_port=7000 cs=1859 guid=9B2996991BCB25DF4CBB0003470CFE75

Using /lib/modules/2.4.9-e-enterprise-ABI/ocfs/ocfs.o

Formatting an OCFS Partition.

Similar to configuring the OCFS, there are two ways to

 format an OCFS partition. One is using the “ocfstool”
command and the GUI environment, and the other is
using the “mkfs.ocfs” command from the shell prompt.
Both commands needs to be executed as root.
None of the ocfs utilities will partition the disk at any
time. So, before formatting, choose the utility of your
preference and partition the disk according to the
needs of your implementation, and make sure the
disk/partition is not being used by anything else to
avoid data loss.
With OCFS you have to format a partition only once on
one node, after that every node will be able to mount
this filesystem. (of course the device needs to be vis-
ible on every node in the cluster)

1.1.4 - Format using ocfstool.

Start by invoking the ocfstool command like described

in the section “3.2.2 Automatic configuration using
ocfstool”. After getting the ocfstool window(See illus-
tration 6), press either the sequence key <CTRL-F> or
from the menu, choose Tasks>Format. Fill in all fields
and click the “OK” button. For more information on the para-
meters, check the format item under ocfstool section.

1.1.5 - Format using mkfs.ocfs.

For people who do not have a GUI available, use mkf-

s.ocfs. If a GUI is available, the preferred method to
format an OCFS partition is to use the ocfstool. The mkf-
s.ocfs command has the following syntax:
# mkfs.ocfs

usage: mkfs.ocfs -b block-size [-C] [-F] [-g gid] [-h] -L volume-label

-m mount-path [-n] [-p permissions] [-q] [-u uid] [-V] device

-b Block size in kilo bytes

-C Clear all data blocks

-F Force format existing OCFS volume

-g GID for the root directory

-h Help
 -L Volume label

-m Path where this device will be mounted

-n Query only

-p Permissions for the root directory

-q Quiet execution

-u UID for the root directory

-V Print version and exit

When using the mkfs.ocfs command, the user has to

provide all the information that is prompt by the ocf-
stool utility.

The usage of the “-C” argument will force the mkfs.ocfs

to clear all blocks. Depending on the size of the parti-
tion, it may be a long process.
The “-F” argument should be used only if the partition
was previously formatted as an OCFS.
The “-b” argument specify the blocksize that the parti-
tion will be formatted. The blocksize specifies the max-
imum size of the partition that can be mounted. It goes
from 4k to 1M and allows volumes from 32Gb up to
8Tb. Format with 128kb blocksize is optimal size. Sizes
between 4kb and 1mb are supported. The smaller
blocksizes will have a performance penalty, but will be
useful for the future when we will support regular files.
128kb blocksize means that every file created with
content uses up a minimum of 128kb space on disk.
Even if there is only 1 byte of data in the file. The
filesystem will allocate chunks of space in <blocksize>
chunks.

The example below show an ordinary partition being

formatted and its output.
# mkfs.ocfs -F -b 128 -g dba -u oracle -L /u01 -m /u01 - p 775 /dev/sdb1

Checking heart beat on volume ..........

Clearing volume header sectors...Cleared volume header sectors

Clearing node config sectors...Cleared node config sectors

Clearing publish sectors...Cleared publish sectors

Clearing vote sectors...Cleared vote sectors

Clearing bitmap sectors...Cleared bitmap sectors

Clearing data block...Cleared data block

 Writing volume header...Wrote volume header

Mount OCFS partitions.

1.1.6 - Mounting manually.

At least for the first time, it is good to have the parti-

tions mounted manually instead of automatically.
That's because the user has control over all the pro-
cess and can check if everything works fine.
To mount the new OCFS partition, use the mount com-
mand with the “-t” argument specifying “ocfs” in front
of it. The example below shows how an OCFS partition
is mounted:
# mount -t ocfs /dev/sdb1 /u01

1.1.7 - Mounting automatically

To mount the OCFS partitions automatically, just add

the partition information to the /etc/fstab file. The ex-
ample below show an entry for RedHat AS 2.1.
/dev/sdb1 /u01 ocfs _netdev
The _netdev /dev/sdd1 /u02 ocfs _netdev
option is
supported on
RedHat, but
may not be for
UL distributions.
Check the
mount man
pages for

Tools

ocfstool

is a GUI frontend for managing and debugging

ocfstool
OCFS volumes on the system; and also the preferred
method for managing OCFS. One can mount and un-
mount volumes, format partitions, view information
and individual files, see the current node map, and block
bitmap.
After starting the tool you are presented with a window con-
 sisting of 2 sections or segments.
The top portion maps all known partitions that are OCFS
formatted and allow users to mount and unmount these parti-
tions. The mount operation will try to mount the filesystem to
the mountpoint specified during the format operation. Like
any other filesystem the umount operation will only succeed if
there is no process using it.
The bottom portion has a series of folders that are divided by
areas of management and browsing. It will show the informa-
tion related to the device selected in the top portion of the
screen.
The information that can be obtained is divided in :
The preferences
menu has effect General – Contain general information about the filesystem
only to the (See Illustration 1). Like ocfs version that formatted the
general folder,
allowing the device, mountpoint, size of the filesystem, number of extents,
user, group and userid and groupid with privileges on the filesystem and the
protection fields
appropriate permission.
to be updated
when in File Listing – Will show file information about the filesystem
(See Illustration 2). Selecting a file or directory, one can see
its information, like size, allocation unit, ownership and pro-
tection. One can also see which nodes have the partition
mounted at that specific point in time.
Configured Nodes – On this folder, one can see which nodes
have the selected partition (See Illustration 3).
Bitmap View – On this folder, one can see the bitmap alloca-
tion for the selected partition (See Illustration 4).
Free Space – On this folder, one can see a list of free space for
the selected partition (See Illustration 5). This list will show
the size and the bit# of the free space.
Illustration 1

Illustration 2
Illustration 3

Illustration 4
 Illustration 5
In addition to the two portions showed, there are the
tasks that can be reached by the menu. The available
tasks on the menu are:

1.1.8 - Format.

There are two ways to invoke the format window, one is press-
ing the <CTRL+F> key sequence and the other is invoking by
selecting the menu Tasks>Format.
When the format window is invoked, the following options will
be available:
Device – From the pull-down menu, select the device that is
going to be OCFS formatted. Make sure the device is not in
use by any other application or filesystem before proceeding.
Blocksize – Select the blocksize on which the partition is go-
ing to be formatted. Valid values are 4k, 8k, 16k 32k, 64k,
128k, 256k, 512k and 1024k. The optimal size suggested by
Oracle is 128k. Smaller blocksize can be selected but they will
carry some performance penalty. This is also going to limit the
size of the partition that can be used when formatting with
OCFS (32Gb to 8Tb).
Volume Label – This will specify the volume label. This is
 useful if the user wants to use the volume label to mount the
filesystem.
Mountpoint – Specify the location where the partition is go-
ing to be mounted. The mount point must exist on all nodes in
the cluster that will share the device.
User – Specify the user that will own the filesystem. When the
filesystem is mounted, it is automatically owned by the user
specified in this field.
Group – Specify the group that the filesystem will belong to.
Similar to the User field.
Protection – Sets the default permission of the filesystem
when mounted. Usually set to 0755.
Clear all Data Blocks – When checked, this option will make
ocfs format block by block, zeroing all the filesystem. This op-
tion will increase the time necessary to format the partition
considerably. (SLOW)
Force – This options needs to be checked if the partition
to be formatted was previously formatted by OCFS.

Illustration 6

1.1.9 - Generate Config.

Invoking this task will promptly generate the /etc/ocfs.-

conf file according to the server configuration. Mandat-
ory fields will be automatically filled with the server in-
formation. There is no need to do any further steps as
the process will automatically generate the guid in the
configuration file..
 When the Generate Config window is invoked, the following
options will be available:
Interface – Select the interface that is going to be used by
the OCFS to generate unique identification. This interface will
also be used by the OCFS Distributed Lock Manager (DLM) in
future implementation. Make sure that all nodes that will be
sharing the OCFS can be reached through the selected inter-
face.
Port – Select the port number that is going to be used to com-
municate to the other nodes. Any non-used port can be as-
signed to OCFS. Make sure all nodes have the same port con-
figured.
Node-Name – Select the node name associated to the inter-
face selected.
Preferred Node Number – This is an optional field. If there
is a need to have an specific node assigned to a number, fill
this field, otherwise, leave it to the OCFS.

Illustration 7

Extfinder

is an utility to report a list of contiguous free

Extfinder
extents in the ocfs filesystem. It will show how frag-
mented the filesystem is after usage.

Debugocfs

The debugocfs utility is used to extract metadata inform-

ation from the OCFS partition. This information is use-
ful to help diagnose eventual problems that may arise.
The debugocfs syntax is :
debugocfs: Usage: debugocfs [-?] [-h] [-g] [-l] [-v range] [-p range]

[-d /dir/name] [-f /file/name [-s /path/to/file]] [-a range] [-A range]

[-b range] [-B range] [-r range] [-c range] [-L range] [-M range]

[-n nodenum] /dev/name

 -h: volume header

-g: global bitmap

-l: full listing of all file entries

-v: vote sector

-2: print 8-byte number as 2 4-byte numbers

-p: publish sector

-d: ocfs_dir_node structure for a given path

-f: ocfs_file_entry structure for a given file

-F: ocfs_file_entry and ocfs_extent_group structures for a given file

-s: suck file out to a given location

-a: file allocation system file

-A: dir allocation system file

-b: file allocation bitmap system file

-B: dir allocation bitmap system file

-r: recover log file system file

-c: cleanup log system file

-L: vol metadata log system file

-M: vol metadata system file

-n: perform action as node number given

/dev/name: readable device

range: node numbers to inspect (0-31), commas and dashes ok

ex. 0-3,5,14-17

Below, some examples of the debugocfs output

Volume Header.
[root@ca-test2 root]# debugocfs -h /dev/sdb12
diskheader:
version = 1.2
signature = OracleCFS
mount_point = /u01
serial_num = 0
device_size = 4194860544
start_off = 0
bitmap_off = 56320
publ_off = 23552
vote_off = 39936
root_bitmap_off = 0
data_start_off = 1368064
root_bitmap_size = 0
root_off = 2416640
root_size = 0
cluster_size = 131072
num_nodes = 32
num_clusters = 31989
dir_node_size = 0
 file_node_size = 0
internal_off = 1368064
prot_bits = 493
uid = 1011
gid = 1011
excl_mount = -1

volumelabel:
curr_master = 0
file_lock = OCFS_DLM_NO_LOCK
oin_node_map = 00000000000000000000000000000000
seq_num = 0
label = /u01
label_len = 4

[root@ca-test2 root]#

Vote Sector.
[root@ca-test2 root]# debugocfs -v 0-2 /dev/sdb12
vote0:
seq_num = 0
dir_ent = 0
open_handle = No
Vote0 = (0x00000000)
Vote1 = (0x00000000)
Vote2 = (0x00000000)

vote1:
seq_num = 0
dir_ent = 0
open_handle = No
Vote0 = (0x00000000)
Vote1 = (0x00000000)
Vote2 = (0x00000000)

vote2:
seq_num = 0
dir_ent = 0
open_handle = No
Vote0 = (0x00000000)
Vote1 = (0x00000000)
Vote2 = (0x00000000)

[root@ca-test2 root]#

NOTE: The path

specified for the ocfs_file_entry structure
file and [root@ca-test2 root]# ls /u01/oradata/smdb/dbfiles/control01.ctl
directory
/u01/oradata/smdb/dbfiles/control01.ctl
structure are
relative to the [root@ca-test2 root]# debugocfs -f /oradata/smdb/dbfiles/control01.ctl
mountpoint, not /dev/sdb12
to the root fileinfo:
filesystem. In
the example in
the right, to
obtain file entry
structure for
/u01/oradata/s
mdb/control01.
 Name = /oradata/smdb/dbfiles/control01.ctl
curr_master = 0
file_lock = OCFS_DLM_ENABLE_CACHE_LOCK
oin_node_map = 10000000000000000000000000000000
seq_num = 0
local_ext = true
granularity = -1
filename = control01.ctl
filename_len = 13
file_size = 12509184
alloc_size = 12582912
attribs = OCFS_ATTRIB_REG
prot_bits = S_IRUSR S_IWUSR S_IXUSR S_IRGRP S_IWGRP S_IXGRP
S_IROTH S_IXOTH
uid = 1011
gid = 1011
create_time = Fri Aug 29 16:49:10 2003
modify_time = Fri Aug 29 16:49:10 2003
dir_node_ptr = 2940928
this_sector = 2941440
last_ext_ptr = 0
sync_flags = OCFS_SYNC_FLAG_VALID
link_cnt = 0
next_del = 0
next_free_ext = 1
extent[0].file_off = 0
extent[0].num_bytes = 12582912
extent[0].disk_off = 4644864
extent[1].file_off = 0
extent[1].num_bytes = 0
extent[1].disk_off = 0
extent[2].file_off = 0
extent[2].num_bytes = 0
extent[2].disk_off = 0

[root@ca-test2 root]#

Support Utilities

In this section, a description of each one of the utilities

belonging to the ocfs-support package. These pack-
ages are the minimum necessary to make the OCFS
work properly.

/etc/init.d/ocfs

This script is responsible to automatically load the ocfs

module, perform a sanity check on the environment
and mount all the ocfs partitions listed in /etc/fstab.
/sbin/fsck.ocfs

Perform a filesystem check on the partition. All nodes

of a cluster that share a specific device need to have
them unmounted before running fsck.ocfs, if one of the
nodes has the partition mounted, the fsck.ocfs will fail.
The command syntax is:
usage: fsck.ocfs [-N] [-v] device

-N No write

-V Version

-v Verbose

-q Quiet

/sbin/load_ocfs

The load_ocfs script will call the insmod command and

load the ocfs module with all the correct parameters.

/sbin/mkfs.ocfs

This command is used to build a the filesystem struc-

ture in a partition. The syntax of the mkfs.ocfs command
is :
usage: mkfs.ocfs -b block-size [-C] [-F] [-g gid] [-h] -L volume-label

-m mount-path [-n] [-p permissions] [-q] [-u uid] [-V] device

-b Block size in kilo bytes

-C Clear all data blocks

-F Force format existing OCFS volume

-g GID for the root directory

-h Help

-L Volume label

-m Path where this device will be mounted

-n Query only

-p Permissions for the root directory

-q Quiet execution

-u UID for the root directory

-V Print version and exit

 /sbin/mounted.ocfs

This command mounted.ocfs is used basically to check

which nodes have a specific device mounted. The com-
mand syntax is :
usage: mounted.ocfs <device>

/sbin/ocfs_uid_gen

The ocfs_uid_gen command is usually used once during

the configuration of the OCFS after it is installed. Its
function is to generate the unique key, in the /etc/ocfs.-
conf file, that identify the node in the OCFS node man-
ager. The command syntax is :
Usage: ocfs_uid_gen -c

ocfs_uid_gen -r

-c Create a new GUID for a new node before it enters the

cluster.

-r Recover the GUID, inserting the new MAC address. This

operation is only for existing nodes whose MAC address

has changed.

Best Practices

For RedHat AS 2.1 go to

http://oss.oracle.com/projects/ocfs/dist/documentation/
RHAS_best_practices.html.
ForUnited Linux, go to
http://oss.oracle.com/projects/ocfs/dist/documentation/
UL_best_practices.txt.

Frequently asked Questions & Answers

Q. Can I install the Oracle Distribution in an

OCFS partition?
A. No. OCFS version 1 supports only database files
(datafiles, logfiles, controlfiles, archivelog files, srvm
configuration file and Oracle Cluster Manager quorum file).
 Q. My partitions don't mount automatically
during boot. What's wrong?
A. The ocfs script on /etc/init.d directory is not enable.
Use the chkconfig command to verify that and enable it.
Instructions on how to enable it are listed under item 3.1
- Automatically mount OCFS during boot.

Q. When running fsck.ocfs, it returns the error

“WARNING: nonzero bytes after the disk header
structure”. What does it mean?
A. Earlier releases of mkfs.ocfs did not clear up 1st block
entirely. We've since fixed the issue in the format
utility. As far as functioning goes, one can safely ignore
this message.

Q. What are the most appropriate (recommended)

tool IO and capacity analysis of OCFS?
A. Your usual tools (sar, vmstat, iostat) will
work, it handles io just like any other
filesystem.

Q. Can I use ocfs on a NAS (Network Attached

Storage) device like NetApp?
A. No. NAS works through network and the physical volume
and filesystem is under the fileserver control. OCFS needs
to use a direct attached storage device or SAN (Storage
Area Network) where the local server has total control over
the physical volumes.

Q. Can I use LVM or MD to create my OCFS

filesystem on top of it?
A. LVM and MD are not cluster aware and it makes it very
difficult to manage in a cluster environment. No tests have
been performed using LVM or MD. It may work, but it also
may not. Corruptions are likely to happen since they are
not cluster aware software.

Q.I want to perform some tests using RAC/OCFS,

but I don't want to spend too much money on
hardware. Do I have any other solution?
A.Yes. Users can run on firewire. Oracle does provide a
special kernel and ocfs modules to run on firewire. More
information can be found at
http://oss.oracle.com/projects/firewire/.

Q.Why do I have to include the option _netdev in

the fstab on RedHat?
A.The _netdev parameter is used when the
filesystem resides on a device that
requires network access, it is used to prevent
the system from attempting to mount these
filesystems until the network has been
enabled on the system.

Q.What is the best way to archive the logs to an

OCFS directory?
A.The ideal is to create a directory for each one
of the nodes that will be archiving on OCFS. This
way, the logs will be kept organized and there
will be no concurrency when two or more nodes
archive at the same time. This doesn't mean there
will be only archivelogs of the node on its
directory, because Oracle does allow a node to
perform an archive operation if a specific node
is down for long time to reduce recovery time.
Q.Is OCFS supported on 64 bit platform like
Itanium?
A.Yes, you can find specific packages for ia32
and ia64.
Q. Can I run ocfs on non enterprise Linux
distributions like RedHat 9?
A.OCFS is primarily developed for Enterprise
systems and therefore not officially supported on
platforms other than RedHat Advanced Server 2.1
or UL. The binaries can be compiled from source
at your own risk.
Q.Do I need any specific version of United Linux
to run OCFS?
A. Yes. The latest package of OCFS will work with
UL SP1, but Oracle strongly recommends to run on
SP2 or SP2a.
Q. Can I run the latest OCFS on a plain RedHat AS
2.1, without any errata applied?
A.No. Before the errata e.12, there was a
specific module for each one of the errata
releases. After e.12, the module is generic,
working with any of the erratas released. Oracle
strongly recommends the usage of the latest
errata available.
Q.What is the advantage of running ocfs against
raw devices?
A.There are many advantages on running OCFS. The
biggest and more important is that it looks and
feels like a regular filesystem, so users don't
have to worry with all the raw devices
administration difficulties. Also, users don't
have to create a partition for each datafile, a
few partitions or even a single partition,
depending on the needs and storage configuration,
may be able to handle big databases. It also
eliminate the restriction of 255 raw partitions
limit on Linux, giving almost endless number of
datafiles that can be created.
Q.What happen if I have to change the IP Address
of my systems?
A.Changing IP address on the same interface used
by the ocfs will require you to change the IP
Address specified in the /etc/ocfs.conf file
only.
Q.My Network Interface Card had to be replaced.
Do I need to do something?
A.Yes. Run the utility “ocfs_uid_gen -r” to update
the /etc/ocfs.conf file with the new uid.

Q.I have a partition that is not mounted. How do

I know if it is an ocfs partition or not?
A. Run the command “debugocfs -h /dev/xxxx” as root. If it
is an ocfs partition, the signature will show “OracleCFS”.
For all other type of filesystem, it will just return
something else, not necessarily the fstype.

Q.Can I use my OCFS partition to store regular

files?
A.Like Oracle Distribution, regular files are not supported
on OCFS at the moment. This is planned for OCFS 2.

Q.How much do I lose in terms of performance

compared to raw devices?
A.Not too much. Without async I/O, performance have been
fluctuating from 0 to 5% of raw devices. Considering all
the benefits of a filesystem against raw, the loss is not
that big. Using async I/O, ocfs has shown to be about 5%
faster than raw devices.

Q.How do I enable async I/O on Oracle using OCFS?

A.First of all, it is necessary to relink the Oracle RDBMS
to enable asynchronous I/O (cd $ORACLE_HOME/rdbms/lib; make
-f ins_rdbms.mk async_on). Then add to the init.ora file,
the parameters :
filesystemio_options=setall
_dbwr_async_io=TRUE
tape_asynch_io=true
disk_asynch_io=true

Q.How do I backup my OCFS files? Can I use tar or

other OS command?
A.The best way to perform an Oracle backup is to use RMAN.
Since a lot of users have their own scripts that usually
uses tar or other OS commands, we have made available a set
of commands that support direct_io, so that users can take
hot backup using tar, cp, dd. Go to
http://oss.oracle.com/projects/coreutils/files/ to get the
fileutils/coreutils appropriate to your environment. Note
that for RHAS 2.1 and SLES8, the package is called
fileutils and for RHEL3, it is called coreutils.

Q.Is it possible to resize an existing OCFS

partition?
A.Yes, tuneocfs command will do that. Please refer to the
tuneocfs manpages for additional information on syntax.

Q.I'm having problems with OCFS. How can I debug

OCFS?
A.One needs to be careful enabling debug on ocfs because it
usually fills the /var/log/messages file very quickly. To
enable OCFS tracing:
echo -1 > /proc/sys/kernel/ocfs/debug_level

echo -1 > /proc/sys/kernel/ocfs/debug_context

To disable OCFS tracing:

echo 0 > /proc/sys/kernel/ocfs/debug_level

echo 0 > /proc/sys/kernel/ocfs/debug_context

Q.Can I run OCFS in a stand-alone system? What

are the advantages of running it?
A.Yes. There is no problems on running OCFS in a stand-
 alone system since you use it for database files only
(Datafile, logfiles, controlfiles, srvm configuration file
and Oracle Cluster Manager quorum file). The advantage of
using it on stand-alone is that if one needs to upgrade to
a RAC environment, it will be a smooth transition,
basically adding a new node, recompiling the Oracle RDBMS
kernel to enable RAC mode and add a new online log thread.

Q.I have a database running on OCFS in a stand-

alone node. Why it is so slow compared to other
stand-alone systems running on ext3?
A.OCFS does direct_io, which bypass the Linux cache. Ext3
in other hands makes full usage of the Linux cache.
Depending on the memory available, it is possible that one
will mostly find the data in cache, not having to fetch
from the disk. But this condition happens only if there are
enough free memory for cache. As you use more and more
memory, performance drops significantly.

Q.How can I obtain more information about OCFS?

A.Go to http://oss.oracle.com/ocfs. You will be able to
find documents, sources, binaries and all sort of
information.

Q.I have a customized RedHat AS kernel on my

system. Does Oracle support OCFS and the RDBMS on
it?
A.No. Oracle only supports standard kernels for OCFS and
Oracle RDBMS. It would be impossible to track and reproduce
any problem without knowing all the customizations that
were performed in a specific installation.

Q.How do I know if my OCFS version is officially

supported by Oracle?
A.Check the file /var/log/messages for a message that looks
like “Oracle Cluster FileSystem 1.0.9-PROD Wed Jul 30
16:48:00 PDT 2003 (build 686e542792ebee44eaa1f)”

Q.Is OCFS is mountable on Linux and Windows 2000

simultaneously?
A.No. The on-disk format is not the same. Is similar... but
not the same.

Appendix A

Ocfs-support package Information:

 Name : ocfs-support Relocations: (not relocatable)
Version : 1.0.9 Vendor: Oracle Corporation
Release :5 Build Date: Wed 30 Jul 2003 07:48:40 PM EDT
Install date: (not installed) Build Host: ca-build1.us.oracle.com
Group : System Environment/Kernel Source RPM: ocfs-2.4.9-e-1.0.9-
5.src.rpm
Size : 698099 License: GPL
Packager : nobody <nobody@oracle.com>
URL : http://ocfs.otncast.otnxchange.oracle.com/servlets/ProjectHome
Summary : Support programs for the Oracle Cluster Filesystem
Description :
Support programs for using the Oracle Cluster Filesystem.
/etc/init.d/ocfs
/sbin/fsck.ocfs
/sbin/load_ocfs
/sbin/mkfs.ocfs
/sbin/mounted.ocfs
/sbin/ocfs_uid_gen
Ocfs-tools package information:
Name : ocfs-tools Relocations: (not relocatable)
Version : 1.0.9 Vendor: Oracle Corporation
Release :5 Build Date: Wed 30 Jul 2003 07:48:40 PM EDT
Install date: (not installed) Build Host: ca-build1.us.oracle.com
Group : System Environment/Kernel Source RPM: ocfs-2.4.9-e-1.0.9-
5.src.rpm
Size : 169381 License: GPL
Packager : nobody <nobody@oracle.com>
URL : http://ocfs.otncast.otnxchange.oracle.com/servlets/ProjectHome
Summary : Tools for managing the Oracle Cluster Filesystem
Description :
Tools to manage the Oracle Cluster Filesystem
/usr/bin
/usr/bin/cdslctl
/usr/bin/debugocfs
/usr/bin/extfinder
/usr/bin/ocfstool
/usr/share
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/cdslctl.1.gz
/usr/share/man/man1/ocfstool.1.gz
Ocfs Module Package Information:
Name : ocfs-2.4.9-e-enterprise Relocations: (not relocateable)
This example Version : 1.0.9 Vendor: Oracle Corporation
show the Release :5 Build Date: Wed 30 Jul 2003 07:48:40 PM EDT
information Install date: (not installed) Build Host: ca-build1.us.oracle.com
about the Group : System Environment/Kernel Source RPM: ocfs-2.4.9-e-1.0.9-
enterprise 5.src.rpm
module. The Size : 1104069 License: GPL
same files will
Packager : nobody <nobody@oracle.com>
be found in the
other supported URL : http://ocfs.otncast.otnxchange.oracle.com/servlets/ProjectHome
versions Summary : The Oracle Cluster Filesystem for enterprise systems.
Description :
OCFS is the Oracle Cluster Filesystem. This package is compiled for
symmetric processor kernels on machines with more than 4GB of RAM.
/lib/modules/2.4.9-e-enterprise-ABI/ocfs
/lib/modules/2.4.9-e-enterprise-ABI/ocfs-noaio
/lib/modules/2.4.9-e-enterprise-ABI/ocfs-noaio/ocfs.o
/lib/modules/2.4.9-e-enterprise-ABI/ocfs/ocfs.o
OCFS - Oracle Clustered Filesystem for Linux
Users Guide.

August 2003
Author: Marcos E. Matsunaga
Contributing Authors: Wim Coekaerts, Kurt Hackel, Sunil Mushran,
Manish Singh and Mark Fasheh.

Oracle Corporation
World Headquarters
500 Oracle Parkway
Redwood Shores, CA 94065
U.S.A.

Worldwide Inquiries:
Phone: +1.650.506.7000
Fax: +1.650.506.7200
www.oracle.com

For information about the Oracle Open Source, visit us at http://os-

s.oracle.com

Oracle is a registered trademark of Oracle Corporation. Various

product and service names referenced herein may be trademarks
of Oracle Corporation. All other product and service names
mentioned may be trademarks of their respective owners.

Copyright © 2001 Oracle Corporation

All rights reserved.