Beruflich Dokumente
Kultur Dokumente
Checklist
Note on IT
Published: June 2004
The Microsoft IT group regularly interacts with enterprise customers in sharing its
experiences and best practices for the deployment and operations of products such
as Microsoft® Exchange Server 2003. In many of these customer contacts, Microsoft
IT personnel have noted a number of configuration issues that repeatedly appear,
especially with running Exchange on clustered servers. This Note is intended to help
enterprise organizations that are running clustered Exchange servers avoid the most
common problems, thereby improving the reliability and efficiency of their Exchange
environments.
Document Definition
A Note on IT is a short, technically deep
drilldown on a specific topic related to
Microsoft IT and is usually associated with
Introduction
an existing IT Showcase document. A Note The Microsoft IT group uses Microsoft Windows Server™ 2003–based clustered servers in
might illustrate how Microsoft IT performs a its Exchange Server 2003 environment. The use of clustered Exchange servers is discussed
specific operational task step by step or
configures a hardware device or software at a high level in the IT Showcase white paper Exchange 2003 Design and Architecture at
application. It might also relate details of a Microsoft, available at
best practice or contain key information http://www.microsoft.com/technet/itsolutions/msit/deploy/ex03atwp.mspx.
about Microsoft IT’s operations that is
regularly requested by customers. This document consists of best-practice configuration settings—in checklist form—that
Intended Audience support the clustered Exchange environment at Microsoft. The configuration settings
Microsoft Exchange Server 2003 described in this document are specific to the Microsoft environment. However, customers
administrators, system architects, and that are using Exchange on clustered servers are likely to find ideas and scenarios in this
messaging support personnel. document that are similar to their own environments and may benefit from adopting some
Products & Technologies form of these best practices.
• Microsoft Windows Server 2003 This document assumes that readers are either Exchange architects or technical
• Clustered servers
implementers and are already familiar with Exchange Server 2003 and Windows Server 2003
• Exchange Server 2003
clusters.
Note: For security reasons, any sample names of forests, domains, internal resources,
organizations, and internally developed applications and files used in this document do not
represent actual names used within Microsoft and are for illustration purposes only. In
addition, the contents of this document describe how Microsoft IT runs its enterprise data
center. The procedures and processes included in this document are not intended to be
prescriptive guidance on how to run a generic data center and may not be supported by
Microsoft Customer Service and Support.
With the exception of one remotely located mailbox server that supports approximately 500
users, Microsoft IT has deployed all of its corporate mailboxes on clustered Exchange
servers by using a multinode configuration that runs Windows Server 2003 and Exchange
Server 2003. The largest of these multinode clusters is based on a seven-node design that
supports 16,000 200-megabyte (MB) mailboxes across four Exchange virtual server
instances. These Exchange virtual instances each run on a designated active (A) node and
share a single dedicated primary passive (P) node as a target for failover. The clusters
maintain additional lower-end servers that act as alternate passive (p) nodes. The seven-
node cluster is abbreviated as AAAAPpp.
The alternate passive nodes support two functions. The primary function is to offload backup
content to tape as part of a two-stage backup process. The secondary function is to serve as
potential, albeit lower-priority, failover targets for the Exchange virtual instances in the
cluster.
Note: For more information about how Microsoft IT backs up its Exchange servers, see the
IT Showcase technical case study “Messaging Backup and Restore at Microsoft,” available
at http://www.microsoft.com/technet/itsolutions/msit/operations/msgbrtcs.mspx and the IT
Showcase technical note on IT “Backup Process Used with Clustered Exchange Server 2003
Servers at Microsoft,” available at http://www.microsoft.com/downloads/details.aspx?
FamilyId=63FA9270-563F-4627-A0A0-8A07E02CF9BF&displaylang=en.
• The xxx part represents the geographic location of the server or Exchange virtual
instance.
• The yyyy part identifies the default role of the server or Exchange virtual instance in the
cluster.
• The n part is the numeric representation of the server or Exchange virtual instance in the
cluster.
The cluster component names in Table 1 represent the server roles and functions referenced
in this document.
Name Description
RED-PASS-0 Primary passive node—primary target for failover of Exchange virtual instance *
RED-PASS-1 Alternate passive node—secondary target for failover of Exchange virtual instance *
RED-PASS-2 Alternate passive node—tertiary target for failover of Exchange virtual instance *
Configuration Processes
The key configurations discussed in this document can be accessed through either a
standard Microsoft Windows® command prompt or a Windows user interface (UI) dialog box.
This document provides commands and relevant command switches that display current
configuration data. When possible, this document provides sample screenshots of dialog
boxes to illustrate a point or clarify readers’ understanding of configuration change
procedures.
Clustering, by default, uses event log replication, which replicates all event occurrences from
one server node in a cluster to all other nodes in the same cluster. Event log replication could
result in excessive alert chatter to any event monitoring system whereby a single replicated
event could generate an alert from every server within the cluster.
To determine whether event log replication is currently enabled, start a command prompt
session on any node in the cluster, and then type the following command:
Cluster.exe /prop
Name Value
--------------------------------------- -----------------------
AdminExtensions {FFFFFFFF-0000-FFFF-0000-FFFFFFFFFFFF}
DefaultNetworkRole 2 (0x2)
Description
Security 01 00 14 80 ... (148 bytes)
Security Descriptor 01 00 14 80 ... (148 bytes)
Note: Extraneous data in the preceding sample list was removed to have the remainder
better fit within the page width.
The EnableEventLogReplication line in the sample list was intentionally made bold and
italic to help readers locate the data.
To disable event log replication in the cluster, start a command prompt session, and then
type the following command:
After changing the setting, retype the Cluster.exe /prop command to confirm that the
setting has been changed. This setting change is dynamic; no restart is required.
Each Exchange virtual instance runs on a preferred node in accordance with the following
sample naming convention:
To display the currently configured list of preferred owners for an Exchange virtual instance,
start a command prompt session, and then type the following command:
In the case of Microsoft IT, the following data appears for resource group RED-MAIL-0:
2. Right-click the relevant cluster resource group, and then click Properties.
4. In the Modify Preferred Owners dialog box, as shown in Figure 1, select a node and
click the Right arrow to move the node into the list of preferred owners to define failover
priority. Click the Up and Down arrows to change the order of preferred owners as
necessary.
To display the list of possible owners, start a command prompt session, and then type the
following command:
In the case of Microsoft IT, the following data appears for resource group RM - System
Attendant:
2. Click to expand the resource group that has the relevant System Attendant resource that
you want to adjust.
5. In the Modify Possible Owners dialog box, as shown in Figure 2, select a node and
click the Right arrow to move the node into the list of possible owners to define possible
failover targets. Click the Up and Down arrows to change the order of preferred owners
as necessary.
The following sample scenario events demonstrate the benefit of cluster group affinity:
1. Exchange virtual instance RED-MAIL-0 has become active on node RED-PASS-0 due to
a fault condition that occurred on its preferred active node, RED-ACTV-0.
3. The cluster detects an affinity conflict and prevents RED-MAIL-1 from attempting to
move to RED-PASS-0.
To list the current affinity settings in the cluster, start a command prompt session, and then
type the following command:
In the case of Microsoft IT, the following data appears for resource group RED-MAIL-0:
Note: Extraneous data in the preceding sample list was removed to have the remainder
better fit within the page width.
The AntiAffinityClassNames line in the sample list has been intentionally made bold and
italic to help readers locate the data.
Cluster.exe net
Network Status
-------------- ---------
Private Up
Private+Public Up
Notes: In the list of current properties, roles are identified by one of three values. Role 1 is a
dedicated private network used by the cluster only for private cluster communications. Role 2
is a public network used by end users only for client access. Role 3 is a mixed network used
for all communications, including node-to-node communications within a cluster as well as
client access.
The lines identifying the network role in the sample list has been intentionally made bold and
italic to help readers locate the data.
2. Right-click the root of the cluster to be edited, and then click Properties.
3. From the cluster priorities dialog box, click the Network Priority tab. Right-click the
network to be edited (run this procedure for each network in the cluster). As an example,
the network properties dialog box for the private network appears, as shown in Figure 3.
4. For the Private network, click Internal cluster communications only (private
network). For the Private+Public network, click All communications (mixed network).
5. Click OK to close.
To assign the highest priority to the Private network communications for private cluster
communications, start a command prompt session, and then type the following command:
Cluster.exe /listnetpri
In the case of Microsoft IT, the following data appears, listing the priorities of all networks:
2. Right-click the root of the cluster to be edited, and then click Properties.
A properties dialog box for the Private network appears, as shown in Figure 4.
Figure 4. Dialog box used for setting network priority of the Private network
4. Click Private network, click Move Up to give it the highest priority, and then click OK.
Caution: Failure to implement this configuration setting may cause problems later when
Exchange is upgraded.
The following steps detail how to check the status of a network connection and, if necessary,
change the position of a network connection:
2. Click the Advanced menu, and then click Advanced Setting. The Adapters and
Bindings tab indicates the priority status of the current network connections.
3. The Private+Public adapter, which supports all communications, should be bound to the
top of the stack, as shown in Figure 5. If it is not, click Private+Public, and then click the
upward arrow to the right of the list to promote it to the top of the stack. When you finish
this step, the Private adapter is bound at a lower priority than Private+Public.
Error 1123:
When encountered, these errors are usually associated with poor network communications
caused by the Private or Private+Public network adapter not negotiating correctly.
Microsoft IT disabled this default advanced behavior to eliminate potential cluster instability
that may occur as a result of different types of resource failures. Figure 6 illustrates how to
configure the properties of a cluster resource to change this behavior.
Another example of instability is when a physical disk resource fails due to spindle failure or
signature mismatch, resulting in all resources being taken offline and failing over to another
node in the cluster. The physical disk resource may fail to come back online after failover,
To mitigate the potential for uncontrolled movement of a resource group between cluster
nodes, Microsoft IT clears the Affect the group option, as seen in Figure 6, on all resources.
This change will prevent any uncontrolled movement of a resource group between nodes as
a result of resource failures. However, it will not prevent the movement of a resource group to
an available node if an active node fails.
Microsoft IT also suggests reducing the number of restart Threshold settings to 1 on the
Exchange System Attendant and the information store resources. Microsoft IT bases this
suggestion on a conclusion that if either service fails to recover after the first attempt, it will
most likely also fail to recover after three attempts.
The implementation of these suggested changes requires the deployment of a reliable server
monitoring solution to ensure detection of individual resource problems, thereby enabling a
quick response to potential failures. Microsoft IT uses MOM 2000 to detect these conditions
and, when necessary, send an alert notification to Exchange administrators regarding
changes in the service state (such as stopped or pending offline) of a resource.
A predefined script, Service Verification Check Service script, is available in the MOM 2000
Exchange 2003 Management Pack under the Server Availability/Verify Exchange Services
processing rule group. This script can be adjusted to detect a status change on any service in
the cluster. The use of this script should be considered a basic requirement before the
implementation of any changes to the resource’s advanced parameters.
Defining the dependency model helps eliminate the potential for database corruption as a
result of the Exchange Data and Log physical disk resources from entering an offline state
before the Exchange Information Store had cleanly shutdown.
The arrows in Figure 7 represent how the various resource dependencies are set against the
Exchange System Attendant resource.
Exchange System
Attendant
Resource
Note: Microsoft IT uses volume mount points for their log drives in the Exchange clusters.
Each mount point is configured as a separate resource that is dependent on its parent
physical disk resource. For more information about how Microsoft IT uses volume mount
points with Exchange Server 2003 clusters, see the IT Showcase technical white paper
Exchange 2003 Design and Architecture, available at
http://www.microsoft.com/technet/itsolutions/msit/deploy/ex03atwp.mspx.
In the case of Microsoft IT, the following data appears, listing all the cluster resource
dependencies on the Exchange System Attendant:
Note: This list is a shortened sample of the data typically provided by this command.
http://www.microsoft.com
http://www.microsoft.com/itshowcase
http://www.microsoft.com/technet/itshowcase
showcase@microsoft.com
The information contained in this document represents the current view of Microsoft Corporation on the issues
discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it
should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the
accuracy of any information presented after the date of publication.
This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Microsoft grants you the right to
reproduce this White Paper, in whole or in part, specifically and solely for the purpose of personal education.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights
covering subject matter in this document. Except as expressly provided in any written license agreement from
Microsoft, the furnishing of this document does not give you any license to these patents, trademarks,
copyrights, or other intellectual property.
Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses,
logos, people, places and events depicted herein are fictitious, and no association with any real company,
organization, product, domain name, email address, logo, person, place or event is intended or should be
inferred.
This document is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR
IMPLIED, IN THIS SUMMARY. Microsoft, Windows, and Windows Server are either registered trademarks or
trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual
companies and products mentioned herein may be the trademarks of their respective owners.