Netegrity. The leading provider of solutions for securely managing e-business.

Extended NTLM Auth -

Installation and Configuration Guide

Version 1.3
Date: 03-28-2005
 Netegrity Inc., A Division of Computer Associates
201 Jones Road
Waltham, MA 02451
Phone: (781) 890-1700
Fax: (781) 487-0515
http://www.netegrity.com
Copyright © 2005 by Netegrity, Inc.
All Rights Reserved.
Netegrity Customer Service provides technical assistance to customers with current maintenance
agreements at 1-800-325-9870. You can also contact support at support@netegrity.com.
Netegrity also provides, for those customers with current maintenance agreements, free access to
our support website at http://support.netegrity.com.
Every effort was made to ensure the accuracy of this document at the time of this printing.
Additional information or changes made after publication may be included in text files located in
your installation kit.
SiteMinder products and associated documentation are protected by copyright and are distributed
under a licensing agreement. Netegrity Inc. has prepared this document for use by Netegrity Inc.
personnel, licensees, and customers. The information contained herein is protected by copyright.
No part of this document may be reproduced, translated, or transmitted in any form or by any
means, electronic, mechanical, photocopying, optical magnetic, or otherwise, without prior
written permission from Netegrity Inc. Netegrity Inc. reserves the right to, without notice,
modify or revise all or part of this document and/or change product features or specifications.

This product contains encryption technology. Exporting these encryption algorithms to certain
countries may be prohibited or restricted by the laws of the United States.

Some portions of the code are licensed from RSA Data Security, Inc.

SiteMinder products are protected by copyright and are distributed under a licensing agreement.
No part of the SiteMinder product or related documentation may be reproduced without
expressed written permission from Netegrity, Inc.

SiteMinder, Netegrity, and the SiteMinder and Netegrity logos are trademarks of Netegrity, Inc.

All other trademarks or registered trademarks mentioned in this document are the property of
their respective owners.

NETEGRITY INC. SHALL NOT BE LIABLE FOR TECHNICAL OR EDITORIAL ERRORS

OR OMISSIONS CONTAINED HEREIN; NOR FOR INCIDENTAL OR CONSEQUENTIAL
DAMAGES RESULTING FROM THE PERFORMANCE OR USE OF THIS MATERIAL.
 Contents

INTRODUCTION 3
PREREQUISITES 4
SiteMinder 4
Other 4
PRE-INSTALLATION STEPS 5
Checklist 5
Licensing 5
INSTALLING 6
Step 1: Installing Files 6
Step 2: SiteMinder Configuration 6
TROUBLESHOOTING 9
 Introduction
The NT Lan Manager (NTLM) authentication scheme (also known as
Integrated Windows authentication scheme) may be used by
organizations whose users access resources via Internet Explorer Web
browsers and who have at least one IIS Web Server as part of their
site. The NTLM Authentication scheme automatically uses the Windows
NT login name and password of a user in place of a challenge for
credentials. SiteMinder then verifies whether a user is authorized to
access a requested resource.
However when the User Directory is not a WinNT Directory, for
example, a Microsoft Active Directory running in the native mode,
which means that the NT4.0 compatibility mode has been disabled or if
the User Directory is a LDAP or an ODBC database, then an enhanced
version of the SiteMinder NTLM Authentication Scheme will be required
as the out of box NTLM authentication scheme would not work.
For example, if a user in the NTLM 4.0 (Active Directory mixed mode)
authenticates using his user name T-USER into the XYZ domain. NTLM
sees his/her UserID as: XYZ\T-USER and the SiteMinder NTLM
authentication scheme can disambiguate and authenticate the user.
However if the Active directory is running in native mode or if the User
Directory is an LDAP Directory or an ODBC database, the UserID:
XYZ\T-USER could not be found in the User Directory.
So an extended NTLM authentication scheme would be necessary to
disambiguate the Windows login name to the fully qualified DN of the
user as constructed in the User DN Lookup for an Active Directory or
an LDAP Directory.
For an ODBC Database it will disambiguate with respect to the column
name as mentioned in the lookup query.
This solution assumes that the User DN Lookup for Active
Directory/LDAP or the Lookup Query for the ODBC database has been
constructed accordingly by using a unique identifier across the
directory.
For example: For Active Directory the unique identifier may be
SAMACCOUNTNAME or for an iPlanet Directory it may be UID.
Please refer to the Policy Design document (Chapter 7) to learn about
how to construct the User DN Lookup etc. and how SiteMinder
disambiguates a user.
Another feature that the Extended NTLM Auth Scheme supports is
specifying that the user’s Login ID be upper cased or lower cased
before it is disambiguated
The Extended NTLM Auth Scheme is the enhanced version of the
SiteMinder NTLM Authentication Scheme which will do the necessary
disambiguation.

3
SmNTLM Native Auth Installation and Configuration Guide

Prerequisites

SiteMinder
• SiteMinder Policy Server version 5.5 or higher on
Windows or Sun/Solaris

Other
• Internet Explorer 4.x and above

4
 Installation and Configuration

Pre-Installation Steps

Checklist

Please make sure that the following files are included in the kit.

1. smextendedauthntlm.dll
2. SmExtendedAuthNtlm.tar.Z
3. SmExtendedAuthNTLM – Install and Config.pdf

Licensing

This solution supports licensing, including evaluation licenses. Without an installed, valid
license, it will only run within a SiteMinder Policy Service for two hours at a time. After two
hours, it will display a license expired message and return an error to the caller. Restarting the
Policy Service will cause the two hour timer to restart.

The web licensor will send you an email regarding license. To install this license, locate the file
on your SiteMinder/License directory called NPSLicense.txt. If the file does not exist, create a
new one. Cut the lines from the e-mail and paste them into NPSLicense.txt. It does not matter
where in the file (top/bottom) you place these lines, as long as they're together. Please note
that the line containing the encrypted text is a single line. Your mail reader may insert
carriage returns that may need to be removed.

5
SmNTLM Native Auth Installation and Configuration Guide

Installing

Step 1: Installing Files

1. For Windows copy the library SmExtendedAuthNtlm.dll onto the SiteMinder bin directory.

2. For SUN/Solaris: Copy the file SmExtendedAuthNtlm.tar.Z to your SUN policy server machine.
Uncompress and untar the file with the commands:
$uncompress SmExtendedAuthNtlm.tar.Z
$tar –xvf SmExtendedAuthNtlm.tar
Then copy the file libSmExtendedAuthNtlm.so to the siteminder\lib directory.
3. Copy the license onto the SiteMinder license directory.

Step 2: SiteMinder Configuration

A. The Auth Scheme Prerequisites

In order to use this authentication scheme, the following prerequisites must be met:

1. There must be Web Agents on at least one Microsoft IIS Web server (4.0 or later). This IIS
Web server may be part of a farm of IIS web servers that deliver content, or, in a mostly
Apache or Sun One web server environment, it may be inserted into the site just for
authentication purposes.
2. Users must log in using Internet Explorer Web browsers (4.0 or later).
3. Internet Explorer browser options must be setup to allow automatic logon with a user’s
current username and password.
4. The SiteMinder policy server must be running on Windows 2000 or SUN/Solaris.

For Internet Explorer 5.x/6.x Browsers:

From the menu bar in Internet Explorer, select Tools > Internet Options.
The Internet Options dialog box opens.
Click the Security tab to bring it to the front.
Select your Internet zone and click Custom Level.
The Security Settings dialog box appears.
Scroll down to User Authentication > Logon.
Select the Automatic logon with current username and password radio button.
Click OK.

For Internet Explorer 4.x Browsers:

From the menu bar in Internet Explorer, select View > Internet Options.

6
 Installation and Configuration

The Internet Options dialog box opens.

Click the Security tab to bring it to the front.
Select your Internet zone from the drop down list.
In the Internet zone group box, select the Custom radio button and click Settings.
The Security Settings dialog box appears.
Scroll down to User Authentication > Logon.
Select the Automatic logon with current username and password radio button.
Click OK.

Also please refer to the SiteMinder Agent Guide and see how to configure the IIS Web Agent for NT
Challenge/Response Authentication and how to specify files as NTLM Credential Collector.

B. Configuring the Custom Authentication Scheme (Extended NTLM Auth)

Create a New Authentication Scheme.

Choose Custom Template for Authentication Scheme Type.

In the Scheme Type Setup:

Library: smextendedauthntlm
Secret and Confirm Secret should be kept blank.
Parameter:
upperOrlowerCase;domainName;http://servername.domain/siteminderagent/ntlm/creds.ntc

The Parameter represents the case you want applied to the user’s login ID, the domainName
and the URL which points to a .ntc file (NTLM Credential Collector) separated by the delimiter
“;” (semicolon),

• The upperOrlowerCase parameter is optional, and if given

must be one of the values: upper, lower, or none. This
parameter determines if the login ID is to be upper cased,
lower cased, or preserve the original case of the user’s login ID
before it is disambiguated.
• The domainName signifies the WinNT domain name that the
users are logging into.
• The servername.domain signifies the location of the IIS Web
Server where the Web Agent is installed.
• SiteMinder Agents interpret the NTLM Credential Collector in
order to authenticate users based on their current login
usernames and passwords. SiteMinder uses the following value
by default: /siteminderagent/ntlm/creds.ntc

Example:
The Parameter in the Custom Authentication Scheme GUI may look like as follows:
abc_domain;http://xyz.netegrity.com/siteminderagent/ntlm/creds.ntc

7
SmNTLM Native Auth Installation and Configuration Guide

or
lower,abc_domain;http://xyz.netegrity.com/siteminderagent/ntlm/creds.ntc

where “abc_domain” is the WinNT Domain name where the users are logged onto and
“http://xyz.netegrity.com/siteminderagent/ntlm/creds.ntc” is the URL for the NTLM credential
collector.

A typical snapshot of the authentication scheme is as follows:

8
 Installation and Configuration

Troubleshooting
Despite the best efforts in following the installation instructions various
problems can occur. The following hints may be helpful in determining
the cause of the problem.

1. Check the SiteMinder Configuration.

- Check the Authentication Scheme
i. Check the name of the library.
ii. Check the parameter string so that the WinNT domain
name and the NTLM credential collector is present and
they are delimited by a “;” (semicolon).

- Set the TRACE mode on for the debug and check both the
authorization and the authentication log on the policy server.

2. Check the Web Agent Logs.

3. Check the settings in the Internet Explorer Browser.

4. Check the License.