GETTING STARTED WITH DATABASE AUDITING

David Eddy
CONTENTS

1 INTRODUCTION................................................................................................4

2 PLANNING........................................................................................................4

3 SET UP AND CONFIGURATION..........................................................................4

3.1 Add Database Storage Areas...........................................................................4
3.2 Enable Auditing.............................................................................................5
3.3 Configure Database Options and Security..........................................................5
3.3.1 Database Options.....................................................................................5
3.3.2 Add Users to the Database........................................................................7
3.3.3 Assign DBA..............................................................................................8
3.3.4 Assign Audit Permissions...........................................................................8
3.4 Load Default Policies......................................................................................9
3.5 Build an Audit Archive Database.....................................................................11

4 PROCEDURES..................................................................................................12
4.1 Archiving Audit Data.....................................................................................12
4.2 Maintaining Audit Policy................................................................................12
4.2.1 Enabling Auditing for a Table as a Whole...................................................13
4.2.2 Enabling Auditing for Specific Fields..........................................................17
4.2.3 Committing Policy Changes......................................................................19

5 REPORTING....................................................................................................20

6 APPLICATION CONSIDERATIONS...................................................................22
6.1 Authentication and Audit Login.......................................................................22
6.2 Application Context and Event Groups............................................................25
6.3 Application Events........................................................................................27

September 2007 Page 2 of 27

Getting Started With Database Auditing David Eddy

DISCLAIMER

The information contained in this document represents the current view of Progress Software
Corporation on the issues discussed as of the date of publication. Progress reserves the right,
in its sole discretion, to modify or abandon without notice any of the plans described herein
pertaining to future development and/or business development strategies. Any reference to
third party software and/or features is intended for illustration purposes only. Progress
Software Corporation does not endorse or sponsor such third parties or software. This White
Paper is for informational purposes only. PROGRESS MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.

Copyright 2008 Progress Software Corporation. All rights reserved.

September 2007 Page 3 of 27

Getting Started With Database Auditing David Eddy

1 INTRODUCTION
Database auditing can be as simple or as complex as one wishes to make it. At its
most elementary, the only measures needed are database-based. However, with
some very minor application changes, a properly-secure auditing system with full user
identity tracking can be used.

This article shows how to set up and maintain a simple database auditing system
using OpenEdge Database Auditing.

2 PLANNING
Progress OpenEdge database auditing requires that certain security functions be
established. There are two ways of doing this:

Establish users in the database; or

Establish a database passphrase which is used to perform certain activities.
Of these, passwords are the more secure and auditiable method of authentication
and authorization for DBA and auditing tasks, so that is the approach that will be
described.

3 SET UP AND CONFIGURATION

3.1 Add Database Storage Areas

Database auditing requires the addition of two Type-II storage areas to your database
to store the audit data, although one of these will be largely unused.

To begin with, set up a file describing the extents to be added to the database. In
this example, a 256MB extent is used for storage of audit data, which assuming
weekly archiving of audit data should be more than sufficient. The file, audit.st, has
the following contents:

d AuditData,32;512 . f 262144
d AuditData,32;512 .
d AuditIndex,8;8 .

Once this file has been prepared, the storage areas are added to the database to have
auditing enabled:

prostrct addonline db audit.st

3.2 Enable Auditing

Once the necessary storage areas exist within a database, auditing is enabled using
the following command:
proutil db C enableauditing area AuditData indexarea AuditIndex deactivateidx

September 2007 Page 4 of 27

Getting Started With Database Auditing David Eddy

Note that the database must be shut down in order to execute this command.

3.3 Configure Database Options and Security

It is necessary to ensure that appropriate database options and certain minimum
database security have been established. The following tasks need to be performed:

Configure the database security/auditing options.

Establish one or more users in the database.
Assign one or more DBAs.
Assign auditing rights.

3.3.1 Database Options

To do this, start the Data Administration tool against the database being configured
and open the Database Options dialog:

Initially, we will select the options which allow the simplest use of the auditing
subsystem. However, as your knowledge and usage of auditing evolves, you may
wish to revisit these options. The following options should be enabled:

Use Application User Id for Auditing

Trust Application Domain Registry

September 2007 Page 5 of 27

Getting Started With Database Auditing David Eddy

The Auditing Options section contains the following fields:

Use Application User Id for Auditing Overrides the database

connections user ID for recording audit user ID and uses the application level
user ID (if set) for auditing.

Enforce Audit Insert Privilege Requires that users of applications that

use Application Auditing and that set Auditing Context have the Application
Audit Event Inserter privilege granted to them.

The Security Options section contains the following fields:

Trust Application Domain Registry Overrides the database connections

domain registry settings and uses the application level domain registry
settings.

Disallow Blank UserId Blocks database connections with blank user IDs.

Record Authenticated Client Sessions Record user login sessions.

Use Runtime Permissions Checking Apply CAN-WRITE and CAN-READ

permissions at runtime, rather than at compile time.

To prepare a typical application for auditing, you configure the client option to record
the user-id of the application which will be synchronized with the ABL session. You
will typically set this option when you do not synchronize the session and database
connection user-ids so that you can use the OpenEdge run-time database security.

The Trust Application Domain Registry options tells OpenEdge to synchronize the
database connection Registries with the session so that you have a single point of
configuration information and control.

For middle-term, you can choose to implement run-time database security or not by
turning on the 4GL client run-time permission checking option. When you do use run-
time permission checking, you would clear the flag that uses the application user-id
for auditing.

September 2007 Page 6 of 27

Getting Started With Database Auditing David Eddy

You want to audit the user-id from the point of table and field access-control
implementation. If your table/field access controls are in your application, audit the
session user-id. If your table and field access controls are handled by OpenEdge
itself, then audit the database connection. Follow these rules and you will always
have the user-id that is permitted access to the database data.

Note: You cannot mix these and audit session user-id for application type events and
the database connection user-id for database operations.

3.3.2 Add Users to the Database

The next step is to add users to the database. If you have done this for another
database, you can import the users by using Admin/Dump/User Table Contents and
Admin/Load/User Table Contents within the Data Administration tool. If you are
doing this for the first time, use Admin/Security/Edit User List

You will need to add user entries for both DBA users and users you wish to have
perform auditing. Also, if you do not add a record for your current user identity, you
will need to log out of the database and reconnect (providing an appropriate userid
and password) before proceeding.

Make sure you add an entry for the user that created the database. You may also
wish to define a user SYSPROGRESS and assign that user a password, depending on
your preference for SQL administration (if you use SQL).

3.3.3 Assign DBA

By default, any user of a Progress database can alter security settings. This needs to
be restricted to one or more users known as DBAs.

Depending on whether you use SQL with the database, your choices here might be
constrained by the fact that by default, the user that created the database is set up
as a DBA user in SQL-92. This may lead to further administrative tasks on the SQL
side of the database which are not discussed here.

Using the Security Administrators screen (Admin/Security/Security Administrators),

establish the list of users permitted to alter security settings. Be sure to include
yourself!

September 2007 Page 7 of 27

Getting Started With Database Auditing David Eddy

3.3.4 Assign Audit Permissions

Finally, it is necessary to associate particular users with the permission to perform
particular auditing tasks. The following rights need to be assigned:

Audit Administrator
Audit Data Archiver
Audit Data Reporter

This is done using the Edit Audit Permissions tool (Admin / Security / Edit Audit
Permissions). Use the Grant button to assign a role to a user. The screenshot
below shows the tool in action:

September 2007 Page 8 of 27

Getting Started With Database Auditing David Eddy

Once again, make sure that you assign at least one user as Audit Administrator. That
user is the one with the rights to set up and change Audit Policy, which defines what
is audited.

3.4 Load Default Policies

To finish the process of initialising auditing, it is necessary to load and activate the
default auditing policies. These policies automatically enable auditing for such
activities as adding a user, altering the database schema, and so forth, but do not
define any auditing for user tables. These will be defined later using the audit policy
tool.

To start the Audit Policy tool, select Tools/Audit Policy Maintenance from the Data
Administration tool. (Note that Audit Policy Maintenance does not appear in the main
desktop Tools menu.)

To load the default policies, choose File/Import Policy and browse to the file %DLC
%\auditing\policies.xml:

September 2007 Page 9 of 27

Getting Started With Database Auditing David Eddy

Then choose File / Commit Changes:

Once this has been done, exit the tool.

3.5 Build an Audit Archive Database

The final set up task is to build a database which will be used as the repository for
auditing information from all audited databases. This database is constructed by
creating an empty database, enabling auditing, and setting its security before using it
as a target for proutil auditload operations.

To build an audit archive database:

1. Create a structure file to describe the new database, for example:

# audarchive.st Structure file for audit archive database
# Standard database components
b E:\Auditing\db
d Schema Area E:\Auditing\db

# Auditing data storage

d AuditData,32;512 E:\Auditing\db f 2097152
d AuditData,32;512 E:\Auditing\db
d AuditIndex,8;8 E:\Auditing\db f 524288
d AuditIndex,8;8 E:\Auditing\db

2. Build the database

cd e:\Auditing\db
procopy %DLC%\empty audarchive

September 2007 Page 10 of 27

Getting Started With Database Auditing David Eddy

3. Enable auditing on the archive database.

proutil audarchive C enableauditing area AuditData indexarea AuditIndex

4. Configure for transaction performance

proutil audarchive C truncate bi bi 32768 biblocksize 16

5. Set the DBA and auditing users as shown above

a. Audit Archiver permission is required to load audit archive data

This database should be configured with 2 APWs and a BIW. Replication can be
enabled if desired.

4 PROCEDURES

4.1 Archiving Audit Data

The following tasks should be performed at least once per week:

1. Archive the collected auditing data in each audit-enabled database (with

Delete option enabled):
cd database_directory
proutil source_db C auditarchive recs 1024 directory -userid archive_user

You will be prompted for the archive users password (or you can specify it
using password).

2. Load the archive into the audit archive database:

proutil E:\auditing\db\audarchive C auditload source_db.adb recs 1024 -userid
archive_user

You will be prompted for the archive users password (or you can specify it
using password).

Note that the audit archive is written with the same base name as the database.
Care should therefore be taken to either remove the archive file after loading, or if
archiving it, renaming it so that duplicate database names will not cause a collision
(and loss of archive files). Long-term storage of archive files is not needed, since
they are present in the archive database after loading and can be regenerated from
the archive database using proutil auditarchive nodelete if needed.

4.2 Maintaining Audit Policy

Audit policy is maintained through the Audit Policy Maintenance Tool (APMT). This
tool is started through the Data Administration tool via menu selection Tools / Audit
Policy Maintenance.

Note that Audit Policy can be updated while other users are using the database;
changes will be propagated to the other users over several minutes after a new policy
is committed.

September 2007 Page 11 of 27

Getting Started With Database Auditing David Eddy

This section shows only a small subset of the capabilities of the APMT. The enabling
of a whole table and the enabling of auditing for only a single column are both shown.
For full information on the APMT, see OpenEdge Getting Started: Core Business
Services.

4.2.1 Enabling Auditing for a Table as a Whole

1. Connect to the database as a user with Audit Administrator privilege.
2. Start the AMPT by entering Data Administration and selecting Tools / Audit
Policy Maintenance.
3. Select the policy to work with. (This can be either your own policy, or you can
use the template PSC-App-Tables policy. In the examples below, I have added
a policy MyApp-Tables and will use that. Note that I have left the policy
inactive for the time being.)

4. Select the Audit Tables tab. Note that there are no tables listed.

September 2007 Page 12 of 27

Getting Started With Database Auditing David Eddy

5. Add the table(s) by pressing the Add button, then selecting the Browse button.
Here I have selected the Customer table.

September 2007 Page 13 of 27

Getting Started With Database Auditing David Eddy

6. Set the auditing levels as shown below. These settings will record all useful
data, while still capturing the very unusual event where a create occurs with
no subsequent update.

7. Press Save ( ).
8. Select the Audit Fields tab.

9. Press Add ( ).
10. Select all fields which make up part of a unique key for this table and press
OK. In this case, I only needed to select the CustNum field.

September 2007 Page 14 of 27

Getting Started With Database Auditing David Eddy

11. Check the Identifying Field box and fill in the ordinal position of that field in
the unique key:

12. Press Save. (If there are multiple fields in the unique key, it might be
necessary to save the multiple selection then individually edit each field to set
its ordinal position.)

September 2007 Page 15 of 27

Getting Started With Database Auditing David Eddy

4.2.2 Enabling Auditing for Specific Fields

1. Enter the APMT, select the policy to work with (MyApp-Tables in this example),
and select the Audit Tables tab.
2. Add the table containing the field to be audited. Set the audit create, update,
and delete levels to Off (which is the default setting). Here, we have decided
to audit a field in the Employee table:

3. Select the Audit Fields tab and add the identifying fields for this table:

September 2007 Page 16 of 27

Getting Started With Database Auditing David Eddy

4. Add the fields to be audited, selecting the appropriate auditing level for each
field. (Note that it may be desirable to audit changes to the identifying field(s)
as well to ensure that changes are properly tracked.)

Note: If you wish to add multiple fields, right-click on the browse and an Add
Multiple Fields popup will appear.

Note: To adjust the audit settings for multiple fields at once, select all the
relevant fields and adjust the audit level settings as desired then press Save.

September 2007 Page 17 of 27

Getting Started With Database Auditing David Eddy

4.2.3 Committing Policy Changes

Changes in APMT must be committed to the database in order to become effective.

1. Verify that the policy is active (the Policy active checkbox in the Policy tab).
If it is not, activate it by ticking the checkbox and pressing Save. Alternately,
select the policy in the Available Audit Policies browse and choose the menu
item Policy / Activate Policies.

September 2007 Page 18 of 27

Getting Started With Database Auditing David Eddy

2. Verify that the changes will have the effect you are expecting by using the
reports:
a. Policy / Report Conflicts ; and
b. Policy / Report Effective Settings
3. Commit the changes to the database using File / Commit Changes.
4. The changed policy will be recorded in the database and the effects will be
seen over the next few minutes as the various connected clients update their
audit policy cache.

5 REPORTING
The audit data is stored in the following tables:

September 2007 Page 19 of 27

Getting Started With Database Auditing David Eddy

_db-detail
_Db-guid
_Db-description
_Db-mac-key
_Db-custom-detail
_client-session
_Client-session-uuid
_Client-name
_User-id (IE1.1)
_Authentication-date-time (IE2.1)
_Server-uuid
_Authentication-domain-type
_Authentication-domain-name
_Db-guid (FK) (IE3.1)
_Session-custom-detail
_Audit-data-security-level
_Data-seal
_aud-event
_Event-id
_Event-type (IE1.1)
_Event-name (IE1.2)
_Event-description (IE2.1)
has consists of
has
_aud-audit-data
_Audit-data-guid consists of
_Database-connection-id (IE1.1)
_Client-session-uuid (FK) (IE1.2)
_User-id (IE2.1) _aud-audit-data-value
_Audit-date-time (IE5.1) _Audit-data-guid (FK)
created _Audit-event-group (FK) (IE3.1) _Field-name (IE1.1)
_Db-guid (FK) (IE3.2) _Continuation-sequence
_Transaction-id (IE3.3)
_Data-type-code
_Transaction-sequence (IE3.4)
_Old-string-value
_Event-id (FK) (IE4.1)
_New-string-value
_Event-context (IE6.1)
_Old-blob-value
_Application-context-id (FK) (IE7.1)
_New-blob-value
_Event-detail
_Old-clob-value
_Audit-custom-detail
_New-clob-value
_Audit-data-security-level
_Audit-data-security-level
_Data-seal
_Data-seal
resulted in supplies context to
is the group for

Altered values are stored in _aud-audit-data._Event-detail with the following format:

_Event-detail == field-change [ field-delim field-change ] ...

field-change = field-name value-delim data-type value-delim [ old-value-phrase ]

value-delim new-value-phrase

old-value-phrase == value-phrase

new-value-phrase == value-phrase

value-phrase == { value | array-value-list }

array-value-list == array-element-value [ array-delim array-element-value ] ...

array-element-value == E[element-number]:value

field-delim == chr(6)

value-delim == chr(7)

array-delim == chr(8)

data-type == { integer | character | logical | int64 | decimal | ... }

Note that field values can be stored one field per record if desired; this is set in the
APMT. In that case, field values are stored in _aud-audit-data-value.

September 2007 Page 20 of 27

Getting Started With Database Auditing David Eddy

Some complications:

The fields are stored in arbitrary order.

Only the identifying fields and the fields changed are stored; unchanged fields
arent mentioned.

Because of this, the best approach to reporting on untransformed audit data is to

populate a ProDataSet, parsing the data during the load. Alternatively, a derived
(parsed) structure can be maintained which simplifies the reporting process.

See OpenEdge Getting Started: Core Business Services chapter 14: Querying and
Reporting on Audit Data for full information on reporting on audit data.

6 APPLICATION CONSIDERATIONS
The main application issues when first implementing Auditing are:

Handling database authentication.

Establishing identity to the Audit system.

In addition to this, in the longer term it might be considered desirable to enrich the
auditing data through:

Establishing application context.

Defining event groups.
Defining and triggering application events.

6.1 Authentication and Audit Login

Normally, when connecting to a database which has _User records present, Progress
automatically runs a program called _prostar.p which calls another programm called
_login.p. There are default versions of these located in the Progress installation
directory, and this is the version normally executed. In this way, the user is
prompted for userid and password no matter what program they are executing.

However, when logging in to an application (which typically does not use the _User
table for authentication), this results in what users see as a double prompt for login,
which is undesirable. Therefore, applications which use _User-secured databases
should provide their own _prostar.p or _login.p.

In addition to the above, the Auditing system gives better results if the new CLIENT-
PRINCIPAL object is used for identification. The results are better in the following
ways:

User logins and logouts are audited when using CLIENT-PRINCIPAL (rather
than simply having their presence be implied by logged database activity); and

September 2007 Page 21 of 27

Getting Started With Database Auditing David Eddy

CLIENT-PRINCIPAL allows user identity to be auditably followed wherever the

flow of program control might go, in particular into AppServers.

In light of the above, then, it is proposed that the application entry be altered as
follows:

Add an application (e.g. freight2020) and a compile user (e.g. developer to

each audited database;
Configure security for the databases so that the application user has no rights
and the compile user has full rights;
Create a new _prostar.p (located in the application PROPATH) which performs
application authentication (using the application authentication database), logs
into the database using the application user, and creates the CLIENT-
PRINCIPAL.
Modify the application so that the startup procedure will look for authentication
information in global shared variables (or SESSION:PRIVATE-DATA) set up in
the _prostar.p instead of running the application login procedure.

Sample code:

/* SAMPLE CODE - _prostar.p */

def new global shared var g_hCP as handle no-undo. /* client-principal */

def var cUser as char /* allow undo */ .

def var lAuthenticated as logical no-undo.
def var lCancelled as logical no-undo.

def var cApplicationUser as char no-undo init freight2020.

def var cApplicationPassword as char no-undo init freight2020.

def var cDefDomName as char no-undo.

def var cDefDomDype as char no-undo.
def var cDefDomDesc as char no-undo.
def var cDomToken as char no-undo.

assign
cDefDomName = OpenEdge
cDefDomType = ABLApplication
cDefDomDesc = Application security management
cDomToken = base64-encode(generate-pbe-key(cDefDomType))
.

/* get the current identity = OS user name */

cUser = getuserid().

assign
lAuthenticated = false
lCancelled = true
.

do on endkey undo, leave

on error undo, leave
on stop undo, leave
on quit undo, quit:

/* prompt for userid and password */

September 2007 Page 22 of 27

Getting Started With Database Auditing David Eddy

/* if they CTRL-C the login dialog the undo block above will cause
login to go ahead with OS userid */

/* this is a stub you need to:

prompt for userid and password
check against application user database
set OK if OK; not OK if fails
handle multiple attempts */
run applicationAuthenticateDialog
(input-output cUser, output lAuthenticated).

/* if we get here, we know the user didnt CTRL-C the dialog */

lCancelled = false.
end. /* login prompt block */

/* at this point, decide what to do with the user. for the moment,
if authentication failed, kick them out, regardless of whether they
cancelled or not */
if lCancelled = true or lAuthenticated = false then
quit.

/* log in to each attached database as the freight2020 user */

/* this code should really be in _login.p */
def var iDB as integer no-undo.
repeat iDB = 1 to num-dbs:
/* does this database have _User records? */
def var hUser as handle no-undo.
create buffer hUser for table ldbname(iDB) + ._User buffer-name bUser.
hUser:find-first().
if hUser:available then
setuserid(cApplicationUser, cApplicationPassword, ldbname(iDB)).
delete object hUser.
end. /* for each connected DB */

/* create the domain registry and lock it */

security-policy:register-domain(OpenEdge, cDomToken).
security-policy:lock-registry().

/* create the client-principal */

create client-principal g_hCP.
assign
g_hCP:user-id = cUser
g_hCP:domain-name = cDefDomName
g_hCP:session-id = substring(base64-encode(generate-uuid), 1, 22)
g_hCP:domain-type = cDefDomType
g_hCP:domain-description = cDefDomDesc
.
g_hCP:seal(cDomToken).
/* note you may wish to alter logic to take advantage of
client-principal:failed-authentication(); this records login failures
in the audit log */

/* finally, set the sessions user id from the client principal */

def var lLoginStatus as logical no-undo.
lLoginStatus = security-policy:set-client(g_hCP) no-error.
if not (lLoginStatus = true) then do:
message Login failed.
delete object g_hCP.
quit.
end.

In addition, application exit code needs to be modified to log out the client-principal
and clean it up:

g_hCP:logout(). /* invalidates client-principal */

security-policy:set-client(?) no-error. /* clear session user-id */
delete object g_hCP. /* reclaim memory */
g_hCP = ?.

September 2007 Page 23 of 27

Getting Started With Database Auditing David Eddy

6.2 Application Context and Event Groups

There are two independent context settings for enriching audit data; these are
Application Context and Event Group data. The are intended to be used in a
hierarchical fashion (see diagram), but in fact they are independent and can even
overlap.

These can be set using the following language elements:

AUDIT-CONTROL:BEGIN-EVENT-GROUP( )

AUDIT-CONTROL:END-EVENT-GROUP( )

AUDIT-CONTROL:SET-APPL-CONTEXT( )

AUDIT-CONTROL:CLEAR-APPL-CONTEXT( )

Executing BEGIN-EVENT-GROUP or SET-APPL-CONTEXT actually creates a new audit

record, which are then referred to by each audit record created subsequently until the
corresponding END-EVENT-GROUP or CLEAR-APPL-CONTEXT is executed. The
reference methodology is as follows:

September 2007 Page 24 of 27

Getting Started With Database Auditing David Eddy

For more information, see:

OpenEdge Development: ABL Reference p. 1010 (AUDIT-CONTROL system

handle)
OpenEdge Getting Started: Core Business Services ch. 11 (Developing an
Audit-enabled OpenEdge Application).

6.3 Application Events

Application events can be very useful, allowing you to define and record any aspect of
application execution that you wish. The events recorded can be anything, but one
common use of application events is to record the display or lookup of sensitive data.

To use application events, one must:

Create application events to be recorded using the APMT;

September 2007 Page 25 of 27

Getting Started With Database Auditing David Eddy

Enable recording of the application events in one or more databases using the
APMT; and
Place AUDIT-CONTROL:LOG-AUDIT-EVENT( ) calls in the application code as
appropriate

September 2007 Page 26 of 27

Getting Started With Database Auditing David Eddy

Corporate Headquarters
Progress Software Corporation, 14 Oak Park Drive, Bedford, MA 01730 USA
Tel: 781 280 4000 Fax: 781 280 4095

Europe/Middle East/Africa Headquarters

Progress Software Europe B.V. Schorpioenstraat 67 3067 GG Rotterdam,
The Netherlands
Tel: 31 10 286 5700 Fax: 31 10 286 5777

Latin American Headquarters

Progress Software Corporation, 2255 Glades Road, One Boca Place, Suite 300 E,
Boca Raton, FL 33431 USA
Tel: 561 998 2244 Fax: 561 998 1573

Asia/Pacific Headquarters
www.progress.com
Progress Software Pty. Ltd., 1911 Malvern Road, Malvern East, 3145, Australia
Tel: 61 39 885 0544 Fax: 61 39 885 9473 Specifications subject to change without notice.
Progress is a registered trademark of Progress Software Corporation. All other 2017 Progress Software Corporation.
trademarks, marked and not marked, are the property of their respective owners. All rights reserved.

September 2007 Page 27 of 27