Exchange 2013 > Exchange 2007 mail relay issues

Had an interesting issue come to light in our Exchange environment recently. Before I go into it, I’ll give a bit of background on how we’re set up…

Most of our organisation has mailboxes in an Exchange 2007 environment. There are some users and shared mailboxes in Exchange 2013, and this environment is in a hybrid relationship with Exchange Online while we’re migrating to Office 365.

Mail from Exchange 2007 to Exchange 2013 and back is handled by internal transport services. Mail to and from Exchange Online is handled by a set of scoped connectors on the Exchange 2013 side of the organisation. All mail heading into or out of the organisation is relayed through a mail gateway which communicates with the transport services on Exchange 2013.

What started out as an investigation into why our text messaging system was not dealing with emails that should have been sent out as text messages led to the discovery that hundreds of emails had queued on the Exchange 2013 side of the organisation and were not being relayed to Exchange 2007.

The usual tricks of using PowerShell to force-resume the message queues or restarting the transport services did nothing, however we did notice some TLS errors in the event viewer stating that the Exchange 2013 servers could not create a TLS connection with the Exchange 2007 servers. On further investigation, we determined that this was due to a malformed exchange of trusted certificate authorities between the servers. Essentially, because the certificate chains were too long, they were being truncated by the protocol (this behaviour is by design, apparently) and the servers weren’t able to establish an authority to use for mutual trust.

The exact text of the error you’ll see in the log is:

451 4.4.0 Primary target IP address responded with: “421 4.4.2 Connection dropped due to SocketError.” Attempted to failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts.

There are two main ways to fix this error if you encounter it:

  1. Delete some certificates from the computer account’s trusted root CA store. Be careful to only remove CA certificates that have expired or are otherwise invalid as deleting an incorrect certificate can cause serious issues with secure communications.
  2. Create the following registry key on the Exchange 2007 transport servers: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\SendTrustedIssuerList. This should be a DWORD with a value of 0. Once done, restart the Exchange Transport services.

Both options have advantages: Option 1 continues to allow mutual TLS between the servers, however has the risk that you might remove certificates that are needed for other services. Option 2 has the least risk but not exchanging a root CA list has implications for mutual TLS. This Microsoft article describes the issues… Quite why this issue chose to raise its head now when our Exchange co-existence environment has been configured for several years is anyone’s guess…

Share and Enjoy:
  • Digg
  • StumbleUpon
  • Technorati
  • del.icio.us
  • Twitter
  • blogmarks
  • HackerNews
  • Tumblr
  • Posterous
  • email
Bookmark the permalink. Follow any comments here with the RSS feed for this post.
Post a comment or leave a trackback: Trackback URL.

Leave a Reply

Your email address will not be published. Required fields are marked *

This blog is kept spam free by WP-SpamFree.