Exchange DAG failover procedure with a Real Life Example

From time to time performing maintenance on a DAG member server might be necessary. For example, cumulative updates for Exchange are released quarterly, installing them requires stopping services and rebooting the server. During the process we want the mail flow, database access and client access uninterrupted. In a failover situation (where the active database server disappears) the DAG will take care of the switch-over but in a planned situation we better do the process gracefully, leaving time for the services to accommodate, mail queues to drain on the soon-to-be-rebooted server, etc.

Here are the steps to perform the failover and to reinstate the server after the procedure.

Before you Begin

Check the health of the mailbox databases, make sure the passive copies that are going to be activated are in a healthy state.

PS C:\> Get-MailboxServer | %{Get-MailboxDatabaseCopyStatus -Server $_.name} | ft -AutoSize

Name         Status  CopyQueueLength ReplayQueueLength LastInspectedLogTime ContentIndexState
----         ------  --------------- ----------------- -------------------- -----------------
DB01\MAIL    Mounted 0               0                                      NotApplicable
DB02\MAIL    Mounted 0               0                                      NotApplicable
DB-test\MAIL Mounted 0               0                                      NotApplicable
DB02\MAIL2   Healthy 0               0                 10/05/2020 13:50:41  NotApplicable
DB01\MAIL2   Healthy 0               0                 10/05/2020 13:48:13  NotApplicable

We see that both DB01 and DB02 databases are active on the server called MAIL. We put MAIL in maintenance mode, minimizing user distraction during the process.

Put the server in Maintenance mode

We deactivate all the mentioned services from MAIL.alwayshotcafe.com, leaving MAIL2.alwayshotcafe.com as the active member to serve clients and ensure continuous mail flow.

Before maintenance mode, DB01 is active on MAIL, while DB02 is active on MAIL2 as seen below:

START:

# Redirect SMTP traffic away from the server
PS C:\> Set-ServerComponentState MAIL -Component HubTransport -State Draining -Requester Maintenance
PS C:\> Redirect-Message -Server MAIL -Target MAIL2.alwayshotcafe.com

# Suspend the failover cluster node (to be run on MAIL itself!)
PS C:\> Suspend-ClusterNode -Name MAIL

# Disable the auto-activation of mailboxes on the server and put it in maintenance mode
PS C:\> Set-MailboxServer -Identity MAIL -DatabaseCopyActivationDisabledAndMoveNow:$true
PS C:\> Set-MailboxServer -Identity MAIL -DatabaseCopyAutoActivationPolicy Blocked

PS C:\> Set-ServerComponentState MAIL -Component ServerWideOffline -State Inactive -Requester Maintenance

2. Reinstate MAIL server (remove it from maintenance mode)

When you are done updating, rebooting the server, take it out from maintenance mode:

PS C:\> Set-ServerComponentState MAIL -Component ServerWideOffline -State Active -Requester Maintenance

# Resume the failover cluster node (to be run on MAIL itself!)
PS C:\> Resume-ClusterNode -Name MAIL

# Enable the auto-activation of mailboxes on the server and put it in maintenance mode
PS C:\> Set-MailboxServer -Identity MAIL -DatabaseCopyAutoActivationPolicy Unrestricted
PS C:\> Set-MailboxServer -Identity MAIL -DatabaseCopyActivationDisabledAndMoveNow:$false

# Redirect SMTP traffic away from the server
PS C:\> Set-ServerComponentState MAIL -Component HubTransport -State Active -Requester Maintenance

Monitor Server Status

# Check the database copy status on all servers:
PS C:\> Get-MailboxServer | %{Get-MailboxDatabaseCopyStatus -Server $_.name | ft -AutoSize}

# Check server components if they are active or inactive:
PS C:\ > Get-ServerComponentState -Identity MAIL

Server                 Component                  State
------                 ---------                  -----
MAIL.AlwaysHotCafe.com ServerWideOffline          Active
MAIL.AlwaysHotCafe.com HubTransport               Active
MAIL.AlwaysHotCafe.com FrontendTransport          Active
MAIL.AlwaysHotCafe.com Monitoring                 Active
MAIL.AlwaysHotCafe.com RecoveryActionsEnabled     Active
MAIL.AlwaysHotCafe.com AutoDiscoverProxy          Active
MAIL.AlwaysHotCafe.com ActiveSyncProxy            Active
MAIL.AlwaysHotCafe.com EcpProxy                   Active
MAIL.AlwaysHotCafe.com EwsProxy                   Active
MAIL.AlwaysHotCafe.com ImapProxy                  Active
MAIL.AlwaysHotCafe.com OabProxy                   Active
MAIL.AlwaysHotCafe.com OwaProxy                   Active
MAIL.AlwaysHotCafe.com PopProxy                   Active
MAIL.AlwaysHotCafe.com PushNotificationsProxy     Active
MAIL.AlwaysHotCafe.com RpsProxy                   Active
MAIL.AlwaysHotCafe.com RwsProxy                   Active
MAIL.AlwaysHotCafe.com RpcProxy                   Active
MAIL.AlwaysHotCafe.com XropProxy                  Active
MAIL.AlwaysHotCafe.com HttpProxyAvailabilityGroup Active
MAIL.AlwaysHotCafe.com ForwardSyncDaemon          Inactive
MAIL.AlwaysHotCafe.com ProvisioningRps            Inactive
MAIL.AlwaysHotCafe.com MapiProxy                  Active
MAIL.AlwaysHotCafe.com EdgeTransport              Active
MAIL.AlwaysHotCafe.com HighAvailability           Active
MAIL.AlwaysHotCafe.com SharedCache                Active
MAIL.AlwaysHotCafe.com MailboxDeliveryProxy       Active
MAIL.AlwaysHotCafe.com RoutingUpdates             Active
MAIL.AlwaysHotCafe.com RestProxy                  Active
MAIL.AlwaysHotCafe.com DefaultProxy               Active
MAIL.AlwaysHotCafe.com Lsass                      Active
MAIL.AlwaysHotCafe.com RoutingService             Active
MAIL.AlwaysHotCafe.com E4EProxy                   Active
MAIL.AlwaysHotCafe.com CafeLAMv2                  Active
MAIL.AlwaysHotCafe.com LogExportProvider          Active