What is the difference between running

NSD and NSD -info?

Technote (FAQ)

Problem
What is the difference between running "NSD" and "NSD -info"?

Solution
The difference is in the information that is collected:

NSD -info
Running an "NSD -info" on a Domino server displays:

 system information
 notes.ini
 environment variables
 data and program directory contents
 disk drive
 system memory statistics."

Bottom line: When you want to collect only system information, run "NSD -info".

NSD
Running "NSD" captures:

 all the system information that you get with "nsd -info"
 as well as the current processes that are running

Bottom line: When you see a performance issue and need to get a sense of both the system
information and what is running, run an NSD.

Another way to position the distinction

To get the list of processes that are running, NSD uses ("touches") the PID.NBF file, which is
the Process ID table of all Notes/Domino processes.

The "NSD -info" does not touch the "pid.nbf" file.

For more information on the pid.nbf file, refer to the technote titled, "What is the PID.NBF file?"
(#1229311)

Troubleshooting Windows NSD:

When NSD is not generating a ".log" file successfully, it is a good idea to run "NSD -info".

Running NSD -info avoids touching the "pid.nbf" file. If it runs to completion, it shows that the nsd
executable is capable of running. Perhaps there is a problem with the PID.NBF file. Refer to the technote
titled, "What is the PID.NBF file?" (#1229311).

What is the PID.NBF file?

Technote (FAQ)

Problem
What is the PID.NBF file?

Solution
The PID.NBF file is the Process ID table of all Notes/Domino processes. For
the lifetime of a Notes client/server, each process start or shutdown causes
the process ID (PID) to be added to or removed from the PID.NBF.

In a Win32 environment, NSD uses the pid.nbf to determine, in case of a

crash, if the failing process is a Notes/Domino process. The PID.NBF file is
also used by Domino in a partitioned server environment. The file allows the
processes associated with each partitioned server to be identified.

The PID.NBF is located in the Domino\Data directory. The PID.NBF file can be
renamed or deleted. The file is recreated the next time the Domino server is
restarted. If the PID.NBF file is deleted while the Domino server is running,
NSD will be unable to determine the Notes processes to attach to
until the server is restarted (when the new PID.NBF is created).

How does the Compact function behave when using

Database Properties

Technote (FAQ)

Problem
In Lotus Domino®, if you use the File menu --> Database --> Properties ->
Info tab --> Compact option, how does Compact work and behave?

Solution
When triggered from the Database Properties Info panel, three kinds of
compacting can occur: copy-style compaction (-L), in-place compaction (-b),
and in-place with file size reduction compaction (-B). Note the case-
sensitivity of the commands.

NOTE: Compacting a database does not necessarily reduce its size.

Types of compacting:

-- If the Notes database requires structural changes (typically a change of

ODS [on-disk structure]). then a copy-style compaction will be performed
(equivalent to
-L).

-- If the Notes database is logged (Transactional Logging enabled), then a

simple in-place compaction will be performed (equivalent to -b).

-- If the Notes database is not logged (Transactional Logging disabled), an

in-place with file size reduction compaction will be performed (equivalent to
-B).

NOTE: In Domino 7 releases, the request from the Notes Client to compact a
database was changed. This functionality is now handled by the Update task
so that the connection to the Domino server is not tied up waiting for a mail
file to be compacted. The Update task places the Compact request in a
deferred queue and processes it when it has free cycles.
Compact operation not performed
immediately on a busy server
Technote (FAQ)

Question
You want to execute a compact operation on a Lotus Notes®/Lotus
Domino® database from within Notes. In the Notes database, from the File
menu, you choose Database --> Properties --> Info panel --> Compact. The
following message appears in the status bar:

"Your request to compact this database is now being carried out."

However, after waiting several minutes, you notice that the size of the
database reported on the Properties Info panel has not changed.

You verify that the Notes database is not logged (Transactional Logging is
enabled) and that you have access to the Compact button (manager access
in the ACL) since these could be reasons why the database size has not
reduced.

Answer
This issue was reported to Quality Engineering as SPR# JLBS6QKTAF and
has been addressed in Domino release 7.0.3.

Excerpt from the Lotus Notes and Domino Release 7.0.3 MR fix list (available
at http://www.ibm.com/developerworks/lotus):

SPR# JCHN6VM6G9 - Added the new Notes.ini variable

"Debug_Compact_Immediately" to allow compact requests from clients to be
performed immediately (like in R6) rather than queued. (Regression in 7.0.)

Refer to the Upgrade Central site for details on upgrading Notes/Domino.

Additional Information
In Domino 7 and 8 releases, the request from Notes to compact a database
was changed. This functionality is now handled by the Update (updall) task
so that the connection to the Domino server is not tied up waiting for a mail
file to be compacted. The Update task places the Compact request in a
deferred queue and processes it when it has free cycles.

On a busy Domino server, the Update task might take from minutes to hours
to process requests for compacting, especially during peak hours.

You can issue the following Domino console command (without quotes) to
see how many compact requests were executed by Update: "show stat
Update.PendingList.Processed.Compactions"

In some cases, increasing the number of Update tasks and configuring

Update to use a separate thread to do full text indexing reduces the time
involved.

To enable multiple Update tasks, add the following parameter to the server's
notes.ini file:
Updaters = # (number of desired Update tasks to run on the server).

To configure Update to use a separate thread to do full text indexing, add

the following parameter to the server's notes.ini file: Update_Fulltext_Thread
= 1.

NOTE: For the Update_Fulltext_Thread parameter, setting this variable to 1

allows use of a separate thread to do full text indexing so that long full text
indexes don't delay view updates. By default, view updates and full text
index updates are driven by the same thread. View updating will still work in
"updall."

What is a Notes database?

A Notes database is a single file that contains multiple documents. A database generally
contains information about a particular area of interest, such as the forms and policy
documents for a department, or it might contain documents of a similar type, such as e-mail
messages. In addition, some companies create "discussion databases," where employees can
post responses to particular topics. Documents in Notes databases contain rich text, pictures,
objects, and many other types of information.
Most databases are stored on one or more Domino servers, accessible by many users. These
are called shared databases. Databases that are used only by you and reside on your
computer are called local databases.
Notes comes with templates you can use to create your own databases. These templates have
the file extension NTF. The Notes databases have the extension NSF.
Notes databases also have access control lists (ACLs) that control the actions people, groups,
and servers can perform in the database. For example, one person may be able to create and
read documents in a database, where another person may be allowed only to read documents
in the same database.

Known limitations of Notes and Domino

7.x
Technote (FAQ)

Problem
What are some of the known limitations of Notes and Domino 7.x?

Please refer to this document for similar information in Notes/Domino 6.x:

"Known limitations of Notes and Domino 6.x" (#1159187).

Solution
The following tables summarize the known maximum limits of various Notes
and Domino 7.x features as noted in the Notes/Domino documentation.
Note: Specifically, the first table originated in the Notes 7 Client Help. The
second table originated in the Domino 7 Designer Help, and has been edited
to contain only unique items that are not also listed in the Notes Client table.

GETTING STARTED
Limits of Notes

Item Limit
What is the maximum size of a The maximum OS file size limit (up to
database? 64GB)
What is the maximum size of 32KB (storage); 32KB displayed in a view's
text fields? column
What is the maximum size of a Limited only by available disk space up to
rich text field? 1GB
What is the maximum size of a 64KB
single paragraph in a rich text
field?
What is the maximum amount of 64KB
Text (Summary) data per
document?
How many levels of responses in 31 levels; 300,000 documents
a hierarchical view; how many
documents at each level?
How many columns can be 64
included in one table?
How many rows can be included 255
in one table?
How many views can be added to No limit; however, as the number of views
a database? increases, the length of time to display
other views also increases
How many columns are allowed 289 ten-character columns; dependent
in a view? upon # or characters per column
How many documents can be Documents totaling at least 350K
imported into a view?
How many cascading views are 200
allowed in a database?
What is the maximum value (in 22.75
inches) you can enter for margin
size?
What is the maximum value (in 46
inches) you can enter for page
size cropping?
What is the maximum point size 250
you can select/print?
How many documents are Up to the size of the database, with a
allowed in one view? maximum of 64GB
What is the maximum number of Limited only by available disk space
documents that can be exported
to Tabular Text?
What is the maximum number of ~950 names (total ACL size is limited to
entries in an Access Control List? 32767 bytes)
What is the maximum number of 75 Roles
roles in an Access Control List?
What's the maximum password 64 bytes (63 characters for single-byte
length allowed on an ID? character sets but only 21 characters for
some double-byte languages)
What is the maximum number of 32K of names in the Members text field
contacts allowed in a group in
the Personal Address Book?
What is the maximum number of For individual names and private groups
recipients in a single mail which expand locally, 15KB; for public
message? groups which expand on a server, 5MB

APPLICATION DESIGN
Table of Notes and Domino known limits

The following table summarizes the known maximum limits of various Notes
and Domino features.
Item Maximum limit
Characters in names Database Title: 96 bytes
Filenames: On Windows® and UNIX® platforms
minimum of 255 and/or OS limits; on local
Macintosh workstation 31
Field names: 32
View names: 64
Form names: 32
Agent names: 32
Fields in a database ~ 3000 (limited to ~ 64K total length for all field
names). You can enable the database property
"Allow more fields in database" to get up to
22,893 uniquely-named fields in the database.
Forms in a database Limited only by database size.
Margin size (in inches) 46
Page cropping size (in 46
inches)
Point size to select or 250
print
Documents in a view Up to the maximum size of the database
Documents that can be Limited only by available disk space
exported to tabular text
Authorized users on a 8 users
multiple password ID
Outline entries in an ~21,000 entries
outline
Notes allows you to paste a maximum of 255 rows into a table. Exceeding
this limit will yield the error, "No more than 255 rows allowed in a table".

Related information
Known limitations of Notes and Domino 6.x

Cross Reference information

Segment Product Component Platform Version Edition
Lotus
Messaging Lotus
Domino 7.0
Applications Domino
Designer
Support & downloads >

Known limitations of Notes and Domino

6.x
Technote (FAQ)

Problem
What are some of the known limitations of Notes and Domino 6.x?

Please refer to this document for similar information in Notes/Domino 7.x:

"Known limitations of Notes and Domino 7.x" (#1222807).

Solution
The following table summarizes the known maximum limits of various Notes
and Domino 6.x features as noted in the Notes Domino documentation:
Item Maximum limit
Database size The maximum OS file size limit -- (up to 64GB)
Text field size 32KB (storage); 32KB displayed in a view's column
Rich text field size Limited only by available disk space up to 1GB
Response levels in a 31 levels; 300,000 documents
hierarchical view;
number of documents
per level
Characters in names Database Title: 96 bytes
Filenames: On Windows® and UNIX® platforms
minimum of 255 and/or OS limits; on local Macintosh
workstation 31
Field names: 32
View names: 64
Form names: 32
Agent names: 32
Fields in a database ~ 3000 (limited to ~ 64K total length for all field
names). You can enable the database property "Allow
more fields in database" to get up to 22,893 uniquely-
named fields in the database.
Columns in a table 64
Rows in a table 255
Views in a database No limit; however, as the number of views increases,
the length of time to display other views also
increases
Forms in a database Limited only by database size.
Columns in a view 289 ten-character columns; dependent upon # or
characters per column
Documents imported Documents totaling at least 350K
into a view
Cascading views in a 200
database
Margin size (in 46
inches)
Page cropping size (in 46
inches)
Point size to select or 250
print
Documents in a view Views are architecturally limited to 130GB. However,
since databases are limited to 64GB, the views cannot
grow larger than the db. So in practice, views can
grow as large as the db size limit of 64GB. **
Documents that can Limited only by available disk space
be exported to
tabular text
Entries in an Access ~950 names (ACL size is limited to 32767 bytes)
Control List (ACL)
Roles in an Access 75 Roles
Control List
Results of the First Server Setup
Running the setup of the first server modifies the notes.ini file so that it contains all
of the required information to start the server. It also creates several files, Domino
databases, and Domino documents. The following tables outline in more detail what
is created, where it is stored, and what its purpose is.
The following tables discusses some of the settings that are added by the setup
program to the notes.ini file:

Setting Example Description

NotesProgram NotesProgram=c:\progr Executable directory name.
am files\
lotus\domino
Directory Directory=c:\program Data directory name (for non-partitioned
files\lotus\ servers).
domino\data
ServerKeyFileN ServerKeyFileName=ser Server ID file name.
ame ver.id
Domain Domain=Adventure Domain name.
ServerTasks ServerTasks=Update, Tasks to start at server startup.
Replica, Router, Stats,
AMgr, Adminp, Sched,
CalConn, RunJava ISpy,
HTTP
ServerTasksAt ServerTasksAt1=Catalog, Tasks to run once per day at scheduled
x Design times.
Port TCPIP=TCP, 0, 15, 0 Configuration settings for the port driver.
Ports Ports=TCPIP Enabled ports.
DisabledPorts DisabledPorts=LAN0,SP Disabled ports.
X,COM2,COM4,COM5
$$HasLanPort $$HasLanPort=1 The server has or does not have a LAN
port.
Log Log=LOG.NSF,1,0,7,40000 Settings for the log file: file name, log
option (default is 1, log to the console), 0
(the third setting not used at this time),
number of days to retain log information,
maximum number of bytes log file can
contain.
LOG_task Log_Replication=1 Level of detail to be logged for that task.
Passthru_LogLe Passthru_LogLevel=0 Level of detail to be logged about network
vel connections, including passthrough
connections

The setup creates the following Domino databases, all stored in the data directory:
Database File Name Description
Certification certlog.ns Logs all registrations of servers and users.
Log f
Domino names.nsf Used to administer the Domino network.
Directory
Server Log log.nsf Stores the log of messages that have been displayed
on the server console in the last seven days.
Reports reports.ns Generates and stores mail usage reports.
f
Administrator's mail\msmit The first administrator's mail database is also created,
mail file h.nsf (for in a sub-folder of the data directory called mail. The
example) file name is based on the administrator's name.

The following documents are created in the Domino Directory:

Document Description
Person document Person document for the first administrator.
Server document Server document for the first server.
Group document Group document for the LocalDomainAdmins group. The name of
the first administrator is automatically added to this group.
Group document Group document for the LocalDomainServers group. The name of
the first server is automatically added to this group. Any
additional servers registered later will also automatically be
added to the group. The group type is initially set to "Multi-
purpose". This should be changed to "Servers".
Group document Group document for the OtherDomainServers group. This group
is initially empty. It is intended to list servers in other Domino
domains that connect with this domain. The group type is initially
set to "Multi-purpose". This should be changed to "Servers".
Certifier document Certifier document for the organization.
Certifier document Certifier document for the first organizational unit. This
document is only created if the option to create an OU was
selected.

The following IDs are created:

ID Stored in Default Description
the... File
Name
Organizat Data directory cert.id Certifier ID for the entire Domino
ion organization.
Organizat Domino oucert.id Certifier ID for the first OU. This file is only
ional Unit Directory, created if the option to create an OU was
attached to selected.
the Certifier
document
Server Data directory server.id Server ID for the first server.
Administr Domino user.id User ID for the first administrator.
ator Directory,
attached to
the Person
document.
May also be
stored as a
file, if that
option was
selected

The following tables describe the options you can use with the Compact server task. The first
column lists the options as they appear when you run Compact using the Task - Start tool or
the Files tab in the Domino Administrator. The second column lists the equivalent command-
line options that you use when you run Compact using a console command or using a Program
document.
Compact - Basics
Option
Compact only this database
or folder
(To specify databases to
compact using the Files tab,
select the databases in the
files pane.)
For more information on database path, see (1)Running Compact using a console command.
Compact - Options
Option
Compact database only if -S percent
unused space is greater than
x percent
Discard any built view
indexes
Keep or revert database to
previous format
Compact - Style
Command-line Description
equivalent
database path To compact a database in the
Domino data folder, enter the file
Specify any additional name, for example SALES.NSF. To
options after the compact databases in a folder within
database path. the data folder, specify the database
path relative to the data folder. For
example, to compact all databases in
the folder DATA\SALES, specify
SALES.
If you choose "Compact all
databases" (or don't specify a
database path at the command line)
Compact compacts all databases in
the data folder and in folders within
the data folder.
Command-line Description
equivalent
Compacts all databases with a
specified percent of unused space.
For example, if you specify 10,
databases with 10% or more
recorded unused space are
compacted. Note that the unused
space calculation is not always a
reliable measure of unused space.
-D Discards built view indexes. Use this
option to compact databases just
before you store them on tape, for
example. Does copy-style
compacting.
-R Compacts databases without
converting to the current release file
format of the server that stores the
databases or reverts databases in
the current release file format to the
previous release file format. For
example, on Domino 6 and more
recent servers, this option compacts
Domino 5 databases without
converting them to the Domino 6 file
format and converts Domino 6
databases to the Domino 5 file
format. This option uses copy-style
compacting.
Option Command-line Description
equivalent

In-place (recommended) -b Uses in-place compacting and

recovers unused space without
reducing the file size, unless there's
a pending structural change to a
database, in which case copy-style
compacting occurs. This is the
recommended method of
compacting.

In-place with file size -B Uses in-place compacting, recovers

reduction unused space and reduces file size,
unless there's a pending structural
change in which case copy-style
compacting occurs. If you use
transaction logging, do full database
backups after compacting completes.

Copy-style -c Uses copy-style compacting. Use this

option, for example, to solve
database corruption problems.

Copy-style: Allow access -L Enables users to continue to access

while compacting databases during compacting. If a
user edits a database during
compacting, compacting is canceled.
This is useful only when copy-style
compacting is done.

Copy-style: Ignore errors and -i Enables compacting to continue even

proceed if it encounters errors such as
document corruption. Only used for
copy-style compacting.

Compact - Advanced
The advanced compact options are not available through the Compact tool in the Files tab of
the Domino Administrator.
Option* Command-line Description
equivalent

Document table bitmap -f Disables "Document table bitmap

optimization: Off optimization" database property.
Does copy-style compacting.

Document table bitmap -F Enables "Document table bitmap

optimization: On optimization" database property.
Does copy-style compacting.

Don't support specialized -h Disables "Don't support specialized

response hierarchy: Off response hierarchy" database
property; in other words, support
specialized response hierarchy. Does
copy-style compacting.

Don't support specialized -H Enables "Don't support specialized

response hierarchy: On response hierarchy" database
property; in other words, do not
support specialized response
hierarchy. Does copy-style
compacting.

Enable transaction logging: -t Disables transaction logging.

Off

Enable transaction logging: -T Enables transaction logging. Use

On Compact - T when a database is open
or closed. If you use Compact - T on
a database that is closed, logging is
enabled but the Compact is not
logged until the database is opened;
therefore, logging is not available
until you reopen the database.

Don't maintain unread -u Disables "Don't maintain unread

marks: Off marks" database property; in other
words, maintain unread marks.

Don't maintain unread -U Enables "Don't maintain unread

marks: On marks" database property; in other
words, do not maintain unread
marks.

* Select "Set advanced properties" before you enable or disable any of these properties.
Compact - Archive
When you use the document archiving tool to archive and delete documents in a database,
you can use the following Compact options to archive documents if the database is located on
a server and you've chosen the advanced archiving option "Automatically on server."
For more information on the server archiving option, see Archiving locally or on a server if you
have installed Lotus Notes 7 Help. Or, go to www.lotus.com/ldd/doc to download or view Lotus
Notes 7 Help.
Option* Command-line Description
equivalent

Archive only -A Archives and deletes documents from

a database without compacting the
database.

Archive and then compact -a Archives and deletes documents from

a database and then compacts the
database.

Delete and then archive -j Deletes documents from a database

and then compacts the database.

*The Compact tool in the Files tab of the Domino Administrator provides only the option
"Archive database;" this option archives and then compacts.
See also
(2)
Ways to compact databases
(3)
Compacting databases
(4)
Database properties that optimize database performance
(5)
Running Compact using a console command

.TXN Files

Content
Lotus Domino supports transaction logging and recovery. With this feature enabled, the system
captures database changes and writes them to the transaction log. Then if a system or media failure
occurs, you can use the transaction log and a third-party backup utility to recover your databases
IMPORTANT: Enabling transaction logging can improve server performance in most cases.
Transaction logging saves processing time because it allows Domino to defer database updates to
disk during periods of high server activity. Transactions are recorded sequentially in the log files,
which is much quicker than database updates to random, nonsequential parts of a disk. Because the
transactions are already recorded, Domino can safely defer database updates until a period of low
server activity.

What is transaction logging?

Transaction logging keeps a sequential record of every operation that occurs to data. If a database
becomes corrupted, you can "roll back" the database to a point before it was corrupted and replay
the changes from the transaction log.

A single transaction is a series of changes made to a database on a server -- for example, a

transaction might include opening a new document, adding text, and saving the document.

Transaction logging provides three main benefits:

- In most situations, you no longer need to run the Fixup task to recover databases following a
system failure. Excluding Fixup results in quicker server restarts, since Fixup must check every
document in each database, while transaction log recovery applies or undoes only those
transactions not written to disk at the time of the system failure.

- Transaction logging saves processing time because it allows Domino to defer database updates to
disk during periods of high server activity. Transactions are recorded sequentially in the log files,
which is much quicker than database updates to random, nonsequential parts of a disk. Because the
transactions are already recorded, Domino can safely defer database updates until a period of low
server activity.

- Using transaction logging simplifies your daily backup procedure. You can use a third-party backup
utility to perform daily incremental backups of the transaction logs, rather than perform full
database backups.

IMPORTANT: Transaction logging works with databases in format ODS 41 or higher but not with
databases that use formats from earlier releases (ODS 20 will not work). After you enable
transaction logging, all databases are automatically logged. To check database formats, use the
Files tab in Domino Administrator.

NOTE: To use all of the features of transaction logging and recovery, you need a third-party backup
utility that supports Domino transaction logging.

What is considered a transaction?

A transaction is a single API call. It includes creating, modifying, reading (unread marks change) or
deleting documents. A transaction is considered COMPLETE when the change has been saved to disk
by the user. For example, if a user makes a change to the database, and does not save that change
before the server crashes, that transaction is not considered COMPLETE. The transaction would have
been COMPLETE only if the user had saved the change before the server had crashed. COMPLETE
transactions are "commited" to the transactional log.

What is a Transaction log?

A transactional log is a binary file where transactions are written. The transactional log has a .txn
file extension. These .txn files should never be deleted. The maximum size of each log extent (.txn
file) is 64 MB. You can have several .txt logs based on the size specified in the Server document.
The maximum total of .txn files is 4 GB.

What is the Database Instance ID (DBIID)

When you enable transaction logging, Domino assigns a Database Instance Identifier (DBIID) to
each Domino database. When Domino records a transaction in the log, it includes the DBIID. During
recovery, Domino uses the DBIID to match transactions to databases (it identifies which database
the changes should be applied to). The DBIID is stored in the file header, along with the database
ID and the Replica ID. Note: There is no relation to the Replica ID or the DBID.

Some database maintenance activities, such as compaction with options, cause Domino to assign a
new DBIID to a database. From that point forward, all new transactions recorded in the log use the
new DBIID; however, any old transactions still have the old DBIID and no longer match the
database's new DBIID. As a result, Domino cannot restore these old transactions to the database.

To avoid losing data, you should immediately perform a full database backup whenever a
database receives a new DBIID. When you perform this backup, you capture all the database
transactions up until that point and ensure that Domino needs only the new transactions (with the
new DBIID) to restore the database. If the DBIID changes and a backup is not taken after the fact,
the database cannot be successfully restored (backup will have the old DBIID and the transactional
log will not "know" the old DBIID.

NOTE: The DBIID has no relation to the REPLICAID or DBID.

Domino assigns a new DBIID to Domino databases when:

You enable transaction logging for the first time.

- System logging is disabled then re-enabled.
- The database is compacted using copy-style compaction.
- The database has had Fixup -J applied to it.

IMPORTANT NOTES:

• If a database is logged, the default for Compact with no switches is -b (lowercase)

• If a database is un-logged, the default for Compact with no switches is -B (uppercase).
• Compact with no switches and Compact -b (lowercase b) are the only times Compact does
not change the DBIID.
 • The DBIID changes when a database is copy-style compacted because a copy-style
essentially creates an entire new NSF with a new structure, which basically does not match
the structure in the logs for the "old" NSF anymore. Note: -L, -c, and -i are switches that
enable copy style compaction. -B at times uses copy style compaction.
• Compact -B may change the DBIID. This option uses in-place compaction unless there is a
pending structural change in which case copy-style compacting occurs. So when using this
option and transaction logging, do full database backups after compacting completes.

Note: For additional information on the compact tasks switches refer to:
Switches for COMPACT Server Task for Domino (#1084388)

• Fixup is forced on the database (fixup -j)

• You move a Notes database from one logged server to another logged server or from an
unlogged server to a logged server.

NOTE: Changing the log path or maximum log size (after initial set up and use) does not trigger a
DBIID change.

How to set up Transaction logging

1. Ensure that all databases to be logged reside in the Domino data directory, either at the
root or in subdirectories.

2. From the Domino Administrator, click the Configuration tab.

3. In the "Use Directory on" field, choose the server's Domino Directory.

4. Click Server Configuration, and then click Current Server Document.

5. Click the Transactional Logging tab.

6. Complete these fields, and then save the document.

Field Enter
Transactional Logging Choose Enabled. The default is Disabled.
Log path Path name location of the transaction log.
The default path name is \LOGDIR in the Domino data
directory, although it is strongly recommended to
store the log on a separate, mirrored device, such as a
RAID (Redundant Array of Independent Disks) level 0
or 1 device with a dedicated controller.
The separate device should have at least 1GB of disk
space for the transaction log. If you are using the
device solely for storing the transaction log, set the
"Use all available space on log device" field to Yes.
Maximum log space The maximum size, in MB, for the transaction log.
Default is 192MB. Maximum is 4096MB (4GB).
Domino formats at least 3 and up to 64 log files,
depending on the maximum log space you allocate.
Use all available space on log device Choose one:

• Yes to use all available space on the device for

the transaction log. This is recommended if
you use a separate device dedicated to storing
the log. If you choose Yes, you don't need to
enter a value in the "Maximum log space"

Automatic fixup of corrupt
databases
Runtime/Restart performance
Logging style
field.
• No to use the default or specified value in the
"Maximum log space" field.
Choose one:
• Enabled (default). If a database is corrupt and
Domino cannot use the transaction log to
recover it, Domino runs the Fixup task,
assigns a new DBIID, and notifies the
administrator that a new database backup is
required.
• Disabled. Domino does not run the Fixup task
automatically and notifies the administrator to
run the Fixup task with the -J parameter on
corrupt logged databases.
This field controls how often Domino records a
recovery checkpoint in the transaction log, which
affects server performance.
To record a recovery checkpoint, Domino evaluates
each active logged database to determine how many
transactions would be necessary to recover each
database after a system failure. When Domino
completes this evaluation, it:
• Creates a recovery checkpoint record in the
transaction log, listing each open database
and the starting point transaction needed for
recovery.
• Forces database changes to be saved to disk if
they have not been saved already.
Choose one:
• Standard (default and recommended).
Checkpoints occur regularly.
• Favor runtime. Domino records fewer
checkpoints, which requires fewer system
resources and improves server run time
performance.
• Favor restart recovery time. Domino records
more checkpoints, which improves restart
recovery time because fewer transactions are
required for recovery.
Choose one:
• Circular (default) to continuously re-use the
log files and overwrite old transactions. You
are limited to restoring only the transactions
stored in the transaction log.
• Archive (recommended) to not re-use the log
files until they are archived. A log file can be
archived when it is inactive, which means that
it does not contain any transactions necessary
for a restart recovery. Use a third-party
backup utility to copy and archive the existing
log. When Domino starts using the existing file
 again, it increments the log file name. If all
the log files become inactive and are not
archived, Domino creates additional log files.

How to disable Transaction Logging for a specific database

In most cases, disabling Transaction Logging (on a server or database level) is not recommended
because you lose all of the benefits of transaction logging (there are no ill side effects of disabling,
you simply lose the benefits). One of the benefits of transaction logging is fast server restart.
Disabling transaction logging will cause Fixup to run on the database (or all databases on the
server), creating the potential for slow restart.

After you set up transaction logging, all databases that are in Domino Release 5 or higher format
are logged. You can disable transaction logging of specific databases.

Attachments are transactionally logged; however, attachments are logged redo only. Therefore, if
the database is recovered using media recovery you will get back the last copy of the attachment
(once they are done they stay done). If, however, the server crashes with uncommited attachment
updates, they will not be undone since an undo record is never created for them .

Views are not logged, so after media recovery, you will need to rebuild views.

First, perform any of the following:

• When creating a new database, choose "Disable transaction logging" on the Advanced
Databases Options dialog.
• For an existing database, choose "Disable transaction logging" on the Database Properties
box, Beanies tab.
• In Domino Administrator, select a database on the Files tab, choose Tools - Database -
Advanced Properties, then choose "Disable transaction logging"
• Use the Compact task with the -t parameter.

Second, ensure that all users have closed the database. Next, use the DBCACHE command with the
"flush" parameter to close the database in the database cache. Finally, open the database.

How to schedule backups or Transaction logs and logged databases

Backups are essential for recovering from a media failure, which is a failure of the server's disk or
disks. If you have a third-party backup utility, you should:

- Schedule daily incremental backups of the transaction log. Use the backup utility daily to back up
the transaction log.
- Schedule archiving of transaction log files. If you use the archive logging style, use a third-party
backup utility to schedule archiving of log files.
- Schedule weekly full database backups. Each week, it is recommended to run the Compact task
with the option to reduce file size. Because this compaction style changes each database's DBIID,
you should schedule compaction with a full database backup.

How to fix corrupted databases

Corrupted databases don't occur frequently when you use Release 5 or higher databases and
transaction logging. When you use transaction logging to log changes to Release 5 or higher
databases, a server automatically uses the transaction log to restore and recover databases after a
system failure, for example after server failures or power failures. If a disk failure occurs, you use
the transaction log along with a certified backup utility to restore and recover the databases.

Using Transaction logging for recovery

Transaction logging is an integral part of recovering from system and media failures. A system
failure causes the server to stop and requires you to restart the server. During restart, Domino
automatically performs database recovery. The system uses the transaction logs to apply or undo
database transactions not flushed to disk for databases that were open during the system failure.

Domino also runs the Fixup task on databases that use formats from earlier releases, databases
that are in Release 5 or higher format but have transaction logging disabled, and on corrupt
databases if you have the "Auto fixup of corrupt databases" field in the Server document set to Yes.

Fixup -J
Causes Fixup to run on databases that are enabled for transaction logging. Fixup -j should only be
run if a database is corrupt and you have no backup of the database to roll forward from.

Without this -j option, Fixup generally doesn't run on logged databases. The Fixup task interferes
with the way transaction logging keeps track of databases. If you are using a backup utility certified
for Domino, it's important that you schedule a full back up of the database as soon after Fixup
finishes as possible.

Notes.ini parameter: Translog_Status

The TRANSLOG_Status NOTES.INI parameter is used to enable transaction logging for all
databases on the server. "0" is disabled, "1" is enabled.

Content
Transactional Logging and How it Operates

Transactional Logging is a new feature in Domino R5. It is essentially a new

method of writing out database changes to improve performance and to
ensure data integrity. Its main purpose is three-fold:

• to improve performance on the R5 server through sequential writes

to the Transactional Logs
• better data integrity by avoiding inconsistencies and data corruption
• faster server restart and crash recovery

A Transactional Log is simply a binary file where transactions are written.

The transactions are saved in log extents that have a .TXN extension. Each
log extent is 64MB and will fill before a new extent is created or a spare one
reused. Multiple log extents collectively can grow to a maximum size of 4GB
with circular logging or available disk space for archive logging.

Performance Improvement

When transactional logging is enabled on the server, the performance

improvement is mainly due to the nature of how transactional logging
operates. The writes to the Transactional Log are sequential. This is faster
since there is less head movement and there is never a need to search for a
place on the disk to write as there is in R4 or if transactional logging is not
enabled.

The Transactional Logs must be on a separate physical drive for there to be

any performance improvement. It is not sufficient to simply redirect the logs
to a separate partition or a separate logical drive. In general, if the
transactional logs are on a separate drive, a 10-20% improvement should be
seen. However, if the logs are put on the same drive, it is likely that there
will be approximately a 60% degradation.

In R4, writing to disk was time consuming. Modifications could occur across
multiple databases or different parts of one database. As a result, the head
had to move over various areas of disk to change or update data. This
means there was a significant amount of transaction time committing data to
the actual NSF (database). Without the benefit of transactional logging in R4,
fixup relies on the fact that 99.9% of the data is present in the NSF to
correct integrity problems.

In R5, when transactional logging is enabled, complete transactions are

"committed" to the Transactional Log. All writes are done to the transactional
log before they are ever written to the database. The writes are done
sequentially at least after each transaction so the Transactional Log is up to
date generally to the hundredth of a second. Again, since the writes are
sequential, there is less I/O and performance is improved. Please note that
view indexes are not transactionally logged in R5 so views may need
rebuilding after a restart or media recovery. Attachments, however, are
transactionally logged. It is important to note that attachments are logged
redo only, so if the database is recovered using media recovery, you will get
back the last copy of the attachment (once done they stay done). If,
however, the server crashes with uncommitted attachment updates, they will
not be undone since an undo record is never created for them.

When transactional logging writes transactions to the logs, an undo record

and a redo record are usually committed for each transaction. First an UNDO
log record is generated in the event of a system outage. This is done before
a change is written to a database. Before committing a transaction, a REDO
record is also generated. It is used to re-apply a transaction from the
transactional log to the database in the event that it did not get flushed to
the NSF before a server outage. Undo and redo records ensure that if a
change is half done it will be fully undone, and if a change was completely
done then it will be fully re-done to the NSF.

Flushing and Hardening

Once changes are put into the Transactional Log, the changes must also
eventually be hardened to the database. This occurs through a process called
flushing. Any open database has an in-memory version of the database that
is held in the UBM (Unified Buffer Manager). Flushing moves all changes that
were made to the database but only kept in memory (UBM), to the actual
NSF file. There is no set interval for this as the UBM determines when
flushing will occur. It is usually done when there is a lull in the server
activity. The DBIID (Database Instance Identifier) is used to correlate the
updates in the Transactional Logs and in-memory to the respective
database. It is important to note, however, that the Transactional Logs are
not read from during this process since the Transactional Logs are mainly a
write-only object. The updates are read and flushed from the UBM. They are
only read from the Transactional Logs during crash recovery. Transactional
logging is more expedient because of the fact that there are not a lot of
read/writes to it during server production. Otherwise performance would
suffer and it would defeat one of the purposes of transactional logging.

The Runtime/Restart Performance field in the Server document determines

how many MB of changes are kept in memory. The amount of space used is
bound by the "performance" level chosen in the Server document. There are
three choices: Standard (default), Favor Runtime, and Favor Restart
Recovery Time. If Standard is selected, the Redo.Limit is 49MB. This means
that checkpoints during runtime are minimal and 49MB worth of changes are
held in the UBM before they are flushed and hardened to databases. The
Favor Runtime choice has a Redo.Limit of the smaller of 500MB or half the
defined log size. This means that more information is held in the UBM and
hardened to the database less frequently. Allowing UBM to manage more of
the database changes will allow larger machines to better utilize memory
and reduce unnecessary flushing. The Favor Restart Recovery Time choice
allows for more checkpoints during runtime. There is less information held in
the UBM and data is hardened to databases more frequently. The trade-off is
that production time is slower but server restart is faster.

Crash Recovery

After a server outage the Transactional Logs are played back. The Recovery
Point is determined for each NSF requiring log updates, it is the oldest log
information that needs to be re-applied to databases. The databases are
restored to the exact moment of the outage, guaranteed to restore any data
from a completed transaction. Partial transactions will be undone and rolled
back to the last good state in an effort to avoid corruption in the database.
The partial work will be removed from the database before restart completes
and the database is made available for use.

Customers should not move databases away from a server and copy them
over from another server after a server crash. The database may be missing
a significant amount of changes that are only stored in the transaction log
and if the database is not found during restart the changes will not be
restored. Recovery restart must be performed first and can be trigged, for
example, by a request to compact a nonexistent database.

Transactional Logging NOTES.INI Parameters

Transactional logging is enabled in the Server document. All the fields in the
Server document map to specific NOTES.INI parameters.

The parameters are as follows:

TRANSLOG_AutoFixup= Tells whether autofixup is Enabled (1) or Disabled

(0)

TRANSLOG_UseAll= To use all available space or not. 0=Disable, 1=Enable

TRANSLOG_Style= Circular (0) vs. Archive (1)

TRANSLOG_Performance= Favor runtime (1), Standard (2), Favor restart

recovery (3)

TRANSLOG_Status= Whether transactional logging is Enabled (1) or Disabled

(0)

TRANSLOG_Path=XXX Specifies the path to the .TXN files

Question
What are the Checkpoint Frequencies and the Flushing Thresholds for
Transactional Logging?

Answer
Setting Checkpoint Frequency Flushing Threshold
Favor Restart 8MB of data in buffer 25MB of data in buffer
Standard 40MB of data in buffer 121MB of data in buffer
Favor Runtime 250MB of data in buffer 500MB of data in buffer

Problem
What switches are available for the COMPACT server task?

Solution
In Notes Release 5.x and later, the default compaction style is In-place
compaction.

Note: When the Compact task runs it opens the database to find out the
amount of % Used (as opposed to looking in the log.nsf).

When running COMPACT from the server console, the syntax is as follows:

load compact databasename -switch

Compact - Style

Option Command-line Description Notes/Domino

equivalent Release
Compaction style: -b Uses in-place compaction Release 5.x, 6.x,
In-place and recovers unused 7.x
(recommended) space without reducing
the file size, unless
there's a pending
structural change to a
database, in which case
copy-style compaction
occurs. This is the
recommended method of
compacting.

-b is the only switch that

will not change the DBIID
of a database if
transactional logging is
enabled.
Compaction style: -B Uses in-place Release 5.x, 6.x,
In-place with file compaction, recovers 7.x
size reduction unused space and
reduces file size, unless
there's a pending
structural change in
which case copy-style
compacting occurs. If you
use transaction logging,
do full database backups
after compacting
completes.
Compaction style: -c or -C Uses copy-style Release 5.x, 6.x,
Copy-style compaction. Use this 7.x
option, for example, to
solve database corruption
problems
Compaction style: -L Enables users to continue Release 5.x, 6.x,
Copy-style: Allow Upper or lower to access databases 7.x
access while case 'L' will during compacting. If a
compacting work. user edits a database
during compacting,
compacting is cancelled.
This is useful only when
copy-style compacting is
done.
Compaction style: -i Enables compacting to Release 4.x, 5.x,
Copy-style: Ignore Upper and continue even if it 6.x, 7.x
errors and proceed lower case 'I' encounters errors such
will work in R4, as document corruption.
but only lower Only used for copy-style
case 'i' will work compaction.
in R5.

Note: the -i
option has to be
used together
with -c or -L
otherwise you
will get the
error message
<< "Ignore
 errors" is only
allowed with
copy-style
compaction >>
Check for -o or -O If using in place Release 6.x, 7.x
overlapping obects compaction checks for
overlapping objects and
switches to copy style if
any found.

Compact - Options
Option Command-line Description
equivalent
Compact database -S percent Compacts all databases with a specified
only if unused percent of unused space. For example, if
space is greater you specify 10, databases with 10% or
than x percent more recorded unused space are
compacted. Note that the unused space
calculation is not always a reliable
measure of unused space.
Discard any built -D Discards built view indexes. Use this
view indexes option to compact databases just before
you store them on tape, for example.
Does copy-style compacting.
Keep or revert -R Compacts databases without converting
database to to the current release file format of the
previous format server that stores the databases or
reverts databases in the current release
file format to the previous release file
format. For example, on Domino 6
servers, this option compacts Domino 5
databases without converting them to
the Domino 6 file format and converts
Domino 6 databases to the Domino 5 file
format. This option uses copy-style
compacting.

Compact - Advanced
Option* Command- Description Notes/Domino
line Release
equivalent
No-Execute -e Show effective arguments Release 5.x, 6.x,
Upper or without execution since some 7.x
lower case arguments affect others. For
'E' will work. example -i only works with -c, so
by executing the compact
command first with the -e on the
end, is a good way to check
syntax without actually
compacting a database.
Document -f Disables "Document table bitmap Release 5.x, 6.x,
table bitmap optimization" database property. 7.x
optimization: Does copy-style compaction.
Off

Note: Prior to
5.0.3 this
Option
description was
incorrect in the
online
documentation
Document -F Enables "Document table bitmap Release 5.x, 6.x,
table bitmap optimization" database property. 7.x
optimization: Does copy-style compaction.
On

Note: Prior to
5.0.3 this
Option
description was
incorrect in the
online
documentation
Don't support -h Disables "Don't support Release 5.x, 6.x,
specialized specialized response hierarchy" 7.x
response database property. Does copy-
hierarchy: Off style compaction.

Note: Prior to
5.0.3 this
Option
description was
incorrect in the
online
documentation
Don't support -H Enables "Don't support Release 5.x, 6.x,
specialized specialized response hierarchy" 7.x
response database property. Does copy-
hierarchy: On style compaction.

Note: Prior to
5.0.3 this
Option
description was
incorrect in the
online
documentation
Enable -T Enables transaction logging. Release 5.x, 6.x,
transaction 7.x
logging: On
Enable -t Disables transaction logging. Release 5.x, 6.x,
transaction 7.x
logging: Off
Maintain -u Disables "Don't maintain unread Release 5.x, 6.x,
unread marks: marks" database property. 7.x
On
Maintain -U Enables "Don't maintain unread Release 5.x, 6.x,
unread marks: marks" database property. 7.x
Off
Allow for a -K Allow for a larger UNK table in a Release 5.x, 6.x,
larger UNK database past 64 K. 7.x
table NOTE : not available through the
Compact tool in the Files tab of
the Domino Administrator.
Disable large -k Removes large UNK table in Release 5.x, 6.x,
(>64 KB) UNK database. 7.x
table NOTE : not available through the
Compact tool in the Files tab of
the Domino Administrator.
No equivalent -ZU Converts document attachments Domino 6.5.6,
option. using compression type Huffman 7.0.1 and later.
to LZ1.

Note: The database property

"Use LZ1 compression for
 attachments" must be enabled
for this switch to be effective.

For Domino 6.5.6 only the

following Notes.INI parameter
must also be set:
DEBUG_ENABLE_LZ1_REPAIR=1
No equivalent -ZD Converts document attachments Domino 6.5.6,
option. using compression type LZ1 to 7.0.1 and later.
Huffman.

For Domino 6.5.6 only the

following Notes.INI parameter
must also be set:
DEBUG_ENABLE_LZ1_REPAIR=1

Compact - Archive
When you use the document archiving tool to archive and delete documents in a
database, you can use the following Compact options to archive documents if
the database is located on a server and you've chosen the advanced archiving
option "Automatically on server."
Option* Command-line Description Notes/Domino
equivalent Release
Archive only -A Archives and deletes Release 5.x, 6.x,
documents from a 7.x
database without
compacting the database.
Archive and then -a Archives and deletes Release 5.x, 6.x,
compact documents from a 7.x
database and then
compacts the database.
Delete and then -j Deletes documents from New in Release
archive a database and then 6.x, 7.x
compacts the database.
NOTE : not available
through the Compact tool
in the Files tab of the
Domino Administrator.

Domino 6 console help (produced by lo compact -?) lists the following options
which have no effect :
-v Make (v)iews big (do not compress indexes)
-V Make (V)iews small (compress indexes)
This issue was reported to Quality Engineering, through SPR # WBLD5XZHL3.
The -v and -V flags were never implemented in Domino.

Question
Mail delivered prior to Compact -c running against a database appears as
READ in the user's mail file. This mail was delivered after the user closed the
session to the server.

Answer
This issue was reported to Lotus Quality Engineering and has been
addressed in Domino 5.0.9.
Excerpt from the Lotus Notes and Domino Release 5.0.9 MR fix list (available
at http://www.notes.net):

Server-Compact
 • SPR# HHAA4WUBAS - Fixed a problem with notes being marked as
read incorrectly after the database was compacted.

To avoid this problem in Notes/Domino prior to 5.0.9, use in-place

compaction with file-space reclamation (Compact -B).

The problem occurs because of an issue related to how Profile documents are
copied into the new database during copy-style compaction. It is unrelated
to a change in the ODS of the database, and it is unrelated to purging
deletion stubs.

The problem is really caused by copy-style compaction, and will occur

anytime copy-style compaction takes place. COMPACT will perform copy-
style compaction regardless of the in-place style compact parameters if the
mail file has undergone a structural change, such as the mail file design
replacement/conversion. A copy-style compact will be used, also, under the
following conditions:

• database ODS is less than the current version

• the following Compact parameters are used: -D/d, -R/r, -M/m, -C/c,
-F/f, -U/u, -L/l
• local encryption is being enabled

Any of the advanced database properties are being changed, such as:

• Do not support specialized response hierarchy

• Document table bitmap optimization
• Do not overwrite free space
• Do not maintain unread marks
• Disable Transaction Logging

• Allow more fields in database