The Exchange 2007 Transport Dumpster (Part 1) Introduction

The transport dumpster feature of Exchange 2007 is specific to the Hub Transport server role. The idea behind the transport dumpster is simple: each Hub Transport server in an Active Directory site that contains a CCR environment maintains a rolling queue of email messages that have recently been delivered to users whose mailboxes are hosted on that CCR environment. Then, during the recovery process that results after a lossy failover of the CCR environment, all Hub Transport servers in the Active Directory site containing the CCR environment automatically redeliver those messages in the transport dumpster queue. It is important to understand that the transport dumpster protects against the loss of some data, not all data. For example, a transient email that is being sent from the Mailbox server to the Hub Transport server at the time of the lossy failover may not be included in the transport dumpster, resulting in data loss. Other examples include calendar appointments and any email messages stored as draft items when Outlook is running in online (non-cached) mode. The transport dumpster feature is optional and therefore can be turned off. However, it is enabled by default and is likely that most organizations will want to make use of this feature. That being said, it is important to note the requirements, such as increased disk I/O, when running the transport dumpster feature. We will be looking at this in part two of this article. Throughout this two-part article I am going to be assuming that Exchange 2007 Service Pack 1 has been deployed as this service pack brought many enhancements to the transport dumpster feature. For example, with Exchange 2007 Service Pack 1, the transport dumpster supports the Local Continuous Replication (LCR) feature. Despite the fact that LCR is supported, Im going to be focusing this article on the transport dumpsters involvement within a Cluster Continuous Replication (CCR) environment.

Transport Dumpster Configuration

The transport dumpster is configured on a per-storage group basis. The two configurable parameters that control the length of time that messages remain in the transport dumpster are:

MaxDumpsterSizePerStorageGroup - As you can probably tell from the name, this is the amount of storage space allocated per storage group for messages within the dumpster. Microsoft makes clear recommendations on the value that you should give MaxDumpsterSizePerStorageGroup. If your organization has a maximum permissible message size configured, simply make the MaxDumpsterSizePerStorageGroup parameter 1.5 x the maximum message size. For example, if your maximum message size is 30MB, the MaxDumpsterSizePerStorageGroup parameter should be set to 45MB. If your organization does not have a maximum permissible message size, you should set the MaxDumpsterSizePerStorageGroup parameter to be 1.5 x the average message size within your organization. If you dont know what your average message size is, you can

1|Transport Dumpster

use the Exchange Profile Analyzer tool which is available for free download from the Microsoft downloads site. Alternatively, other 3rd party tools will be able to provide this information. The default value of MaxDumpsterSizePerStorageGroup is 18MB, so unless your maximum or average message size within your organization is 12MB, you will need to alter this value. MaxDumpsterTime - This parameter states for how many days, hours, minutes and seconds a message can remain in the transport dumpster. By default, this parameter has a value of 7.0:0:0 which translates to 7 days, 0 hours, 0 minutes and 0 seconds. Microsoft recommends that the MaxDumpsterTime is set to 7 days so, generally speaking; you should not have to alter the configuration of this parameter.

To alter these values you can either use the Exchange Management Console or the Exchange Management Shell. Let us take a look at the Exchange Management Shell first. These values are organization-wide transport values and therefore are not tied to a specific Exchange server. Therefore, you would not expect to use the Get-TransportServer or Set-TransportServer cmdlets to view and alter these values. Rather, you need to use the Get-TransportConfig and SetTransportConfig cmdlets. Figure 1 shows you a sample output from the Get-TransportConfig cmdlet where you can clearly see that the MaxDumpsterSizePerStorageGroup and MaxDumpsterTime parameters are both set to their default values, namely 18MB and 7 days respectively.

Figure 1: Results of the Get-TransportConfig Cmdlet If you were required to change the MaxDumpsterSizePerStorageGroup parameter to a value of 45MB, you would simply run the following cmdlet: Set-TransportConfig MaxDumpsterSizePerStorageGroup 45MB Similarly, the following cmdlet can be used to change the MaxDumpsterTime to a sample value of 5 days and 5 hours, although I do not recommend changing this parameter as I have stated earlier in this article. This is based on the fact that Microsoft itself recommends leaving the default value of 7 days in place.
2|Transport Dumpster

Set-TransportConfig MaxDumpsterTime 5.5:0:0 To use the Exchange Management Console to alter these settings, navigate to the Organization Configuration node of the console tree and highlight the Hub Transport object beneath it. In the result pane, select the Global Settings tab and you will see the Transport Settings object. Rightclick the Transport Settings object and choose Properties from the context menu. You will be presented with the screen shown in Figure 2 where you can see where to change the two transport dumpster values.

Figure 2: Transport Dumpster Settings in Exchange Management Console Note: If you wish to disable the transport dumpster for all storage groups across your Exchange 2007 organization, you can set either the MaxDumpsterSizePerStorageGroup or MaxDumpsterTime values to 0. There is therefore no use in setting one parameter, leaving the other set to 0 and expecting the transport dumpster to be operational.

Transport Dumpster Operation

3|Transport Dumpster

Earlier in this article I discussed how the transport dumpster is effectively a rolling queue of email recently delivered to users whose mailboxes reside on either a CCR or LCR environment. Based on what we now know about the MaxDumpsterSizePerStorageGroup and MaxDumpsterTime parameters, let us see how the rolling queue of email looks based upon the configuration of these two parameters. Let us assume that the maximum message size within the organization is 30MB, meaning that the transport dumpster MaxDumpsterSizePerStorageGroup parameter has been set at 45MB. The first two users on the CCR environment are Ann and Bob. Bob requests that Ann send him the 10 PowerPoint presentations that he is after, so Ann sends Bob 10 different email messages, each containing a different 5MB PowerPoint presentation. With the transport dumpster set at 45MB, you can clearly see that the transport dumpster will be able to store 9 email messages before it reaches capacity, since 45MB divided by 5MB is 9. Figure 3 below is a representation of what the transport dumpster would contain once Ann sent the first 9 of the 5MB email messages.

Figure 3: Transport Dumpster Size Queue When Ann sends the 10th message, the 1st message sent will be removed from the transport dumpster since only a maximum of 45MB can be stored at any one time; the transport dumpster operates in a First In First Out (FIFO) manner. Therefore, the transport dumpster queue would then look like the representation shown in Figure 4, where you can see that message number one has been removed and message number 10 has been added to the queue.

Figure 4: Transport Dumpster Later Size Queue A similar process occurs with the MaxDumpsterTime. Let us assume that the MaxDumpsterTime is left at its default setting of 7 days. Using the same example scenario
4|Transport Dumpster

involving Ann and Bob, let us assume, for claritys sake, that Ann chooses to send just one email per day to Bob and that these are the only messages being processed by the system. After 7 days, the transport dumpster would look like the representation shown in Figure 5. Note that I have added a date column to emphasize the point here.

Figure 5: Transport Dumpster Time Queue As you can see from Figure 5, a total of 7 days worth of email messages are stored in the transport dumpster. Since the MaxDumpsterTime is set to 7 days, it follows that on the 8th day the old message from 1st June 2009 will be removed from the queue. As a result, the transport dumpster would then look like the representation shown in Figure 6.

Figure 6: Transport Dumpster Later Time Queue That concludes part one of this two-part article on the Exchange 2007 transport dumpster feature. In part one we have looked at what the transport dumpster is, how it is configured and how it might look when operational. In part two, we will look at sizing and monitoring the transport dumpster.

5|Transport Dumpster

The Exchange 2007 Transport Dumpster (Part 2)

Introduction
This is the second and final part of a two-part article looking at the Exchange 2007 transport dumpster feature. In part one of this article series, we covered what the transport dumpster feature is, how to configure it, and how it looks from an operational perspective. To complete this series, we are going to look at how to re-size the transport dumpster and then finish off by looking at how it can be monitored in your environment.

Sizing The Transport Dumpster

Sizing of the transport dumpster has an impact on the design of your Hub Transport server, since you will have to take into consideration the size of the message queue in the transport dumpster as well as the additional disk I/O that will be seen when the transport dumpster is enabled. Let us look at how you can plan for these two factors. Admittedly, the actual disk size of the transport dumpster is not necessarily that big unless the Hub Transport server is servicing a large number of storage groups. Remember that the transport dumpster sizing is configured on a per-storage group basis. Let us first take an example environment of two Hub Transport servers that service a single CCR environment that itself contains 10 storage groups. If the maximum message size within this organization is 30MB, we have already seen in part one of this article series that the MaxDumpsterSizePerStorageGroup parameter would be set to 45MB. Therefore, the actual amount of disk space that you would need to allow for would be 450MB, derived from 45MB x 10 storage groups. Naturally you would want to allow this amount of disk space on each Hub Transport server, since you would want to ensure full system operation in the event that you lost one of the two Hub Transport servers. Let us consider another example environment consisting of two Hub Transport servers that both service three CCR environments that each contain a total of 15 storage groups with a maximum permissible message size of 50MB. In this case, the MaxDumpsterSizePerStorageGroup parameter would be set at 75MB, since 1.5 x 50MB is 75MB. Since there would be a total of 45 storage groups, you would need to plan for approximately 3.4GB of disk space for the transport dumpster, since 75MB x 45 storage groups is approximately 3.4GB. Do not forget that the disk space requirements of the transport dumpster relate to the disk holding the Hub Transport servers database file mail.que, since the transport dumpster is contained within this database. Since the Hub Transport database is an ESE database, it is normal to see the mail.que file moved to dedicated disk spindles for performance reasons.

6|Transport Dumpster

For the effects on disk I/O caused by the enabling of the transport dumpster feature, its best to turn to the information supplied by Microsoft as a result of its internal testing. In the Transport Server Storage Design information, scroll down until you find the section called Transport Dumpster Sizing Example where you will see that the second table listed shows you the differences in I/O caused when the dumpster is enabled. In short, enabling the transport dumpster does increase the disk I/O so make sure that you plan for this.

Monitoring the Transport Dumpster

In a CCR environment, the Get-StorageGroupCopyStatus cmdlet can be used to obtain information on the overall health of continuous replication. There is an extra parameter, not used by default, that can be used to display information on the transport dumpster statistics. This parameter is DumpsterStatistics. For example, Figure 7 shows the Get-StorageGroupCopyStatus cmdlet run with the output piped to the format-list cmdlet and additionally filtered to show any parameters containing the string dumpster. As you will see from Figure 7, the parameters are repeated twice since there are two storage groups on this particular CCR environment.

Figure 7: Get-StorageGroupCopyStatus Cmdlet What you will note from Figure 7 is that there are no dumpster statistics shown. To actually see these statistics, you will need to include the DumpsterStatistics parameter such as in this example cmdlet: Get-StorageGroupCopyStatus DumpsterStatistics | fl *dumpster* The results of running this cmdlet are shown in Figure 8, where it may become apparent that this cmdlet has been run in a clean test environment. The DumpsterStatistics parameter shows 0 items in the dumpster for the Hub Transport server called SRV1, with these 0 items obviously totaling 0KB in size. Interestingly, you can also see the effects of when a Hub Transport server in the same Active Directory site as the CCR environment is not contactable. In this case the Hub Transport server called SRV2 is shown as being unavailable as it appears in the DumpsterServersNotAvailable list.

7|Transport Dumpster

Figure 8: Get-StorageGroupCopyStatus Cmdlet With DumpsterStatistics Lets look again at the statistics on a slightly busier server. In Figure 9 you can see that the statistics for a single storage group have been retrieved and that the transport dumpster for the Hub Transport server SRV1 now contains 1,030 items at a total size of 819KB. You will also see the date and time of the oldest message in the transport dumpster queue.

Figure 9: Transport Dumpster Statistics To get more detailed information on the transport dumpster statistics, we can use the performance monitor program. On a Hub Transport server, a performance object called MSExchangeTransport Dumpster can be found that contains 5 counters. You can see these counters in Figure 10.

8|Transport Dumpster

Figure 10: Transport Dumpster Performance Counters The counters are:

Dumpster Deletes/sec - This is how many items per second that are deleted from the transport dumpster queue on this Hub Transport server Dumpster Inserts/sec - This is how many items per second that are submitted to the transport dumpster queue on this Hub Transport server Dumpster Item Count - This is the total number of items currently in the transport dumpster queue on the particular Hub Transport server. It is essentially the same information that you can obtain from the DumpsterStatistics parameter of the GetStorageGroupCopyStatus cmdlet that weve already covered Dumpster Size - The Dumpster Size counter is the total size of the items in the transport dumpster on the particular Hub Transport server. This counter is measured in bytes. Again, this counter shows the same information that you can see via the GetStorageGroupCopyStatus cmdlet Redelivery Count - This is simply a count of the number of items that were redelivered after the Exchange Replication Service requested a resubmit of messages.

For example, on a test CCR environment that I created, I forced a lossy failover situation by turning off the active node whilst messages were being processed. Once the Microsoft Exchange Replication Service on the remaining CCR node had requested a resubmit of messages from the Hub Transport server, I then ran the performance monitor tool on that Hub Transport server and measured the Dumpster Item Count, Dumpster Size and Redelivery Count objects as you can see from Figure 11. It is clear from this figure that the entire transport dumpster queue of 1540 items has been redelivered.
9|Transport Dumpster

Figure 11: Measuring Transport Dumpster Performance Counters So what actually happens during a lossy failover event in a CCR environment? Simply put, the previously passive node now becomes the active node and requests from the Hub Transport servers in the same Active Directory site that messages from the transport dumpster are resubmitted. To see this process in action, check the event log on the remaining CCR node for event 2099 that has a source of MSExchangeRepl. An example of this event log entry is shown in Figure 12. You can see from this figure that the CCR environment node called CCRB has requested that the Hub Transport server called SRV1 resubmits messages between the specific time periods. Note that this event will be recorded every 15 minutes if the particular Hub Transport server is unavailable.

10 | T r a n s p o r t D u m p s t e r

Figure 12: Transport Dumpster Resubmission Request

Summary
ad vert i s emen t

That concludes this two-part article series on the Exchange 2007 transport dumpster feature. There is little doubt in my mind that this is an incredibly useful feature of Exchange 2007 as it can help an organization recover from data loss that may be experienced during an unscheduled outage of a CCR environment. As long as you have taken the time to configure the transport dumpster parameters, it should work with little administrator intervention.

11 | T r a n s p o r t D u m p s t e r