Route Masters

Most administrators are well acquainted with the inner workings of Exchange, but strangely unfamiliar with how message routing works. So how does the latest incarnation, Exchange 2007, transmit e-mail?

I was running a course on Exchange 2007 recently. In the class of 15, most of the attendees were seasoned Exchange administrators who attended to get an update on the new product. I was surprised at how little they understood about Exchange message routing for their current version (Exchange 2003), so this month I’ll take you through the routing model used by Exchange 2007.

We all know that the Exchange 2000 plus series uses SMTP as the message transport, and although in versions 2000 and 2003 support was retained for X400, it has gone from Exchange 2007.

Back to basics
Exchange 2007 uses SMTP to transmit e-mail, but the connections between servers are authenticated and encrypted. Other connections, including any sort of relays (ie connections from other SMTP servers), are all prohibited by default. No more bouncing messages off an Exchange server to relay your private mail out of Outlook Express. The SMTP engine is no longer borrowed from IIS and many of the features of Exchange 2003 e-mail routing have gone, notably the Routing Groups, the Link State Table, and even the administrative groups. (However, check out the Active Directory hierarchy for Exchange because for upwards compatibility you will still see an Administrative Group and a Routing Group for Exchange 2007 – Exchange 12 rocks).

In a nutshell, the replication topology model used by Active Directory using Sites, Site links and Site-link bridges is now used by Exchange to route e-mail. Bizarre as it sounds, there is no longer a requirement to set up routes for e-mail transmittal as this is done automatically by Exchange by checking out the Active Directory Site topology. So how does this actually happen?

Exchange 2007 has a number of roles, including Mailbox, Client Access, Unified Messaging, Hub Transport and Edge Transport. We are going to focus on just two, the Mailbox and Hub Transport. Think back to Exchange 2003 and remember that when a message was sent it went through the Categorizer on that server prior to being sent to an SMTP queue or sent for local delivery. In Exchange 2007, the role of Categorizer SMTP transport has been separated off into its own role, the Hub Transport. This may be combined with the mailbox to give a broad equivalence to Exchange 2003, or be run on a separate server.

New routes
For Exchange 2007 the routing process has changed significantly. Looking at the Hub Transport routing function, the whole process may be split up into five major stages. At the start is the Microsoft Exchange Transport Service for inbound e-mail. The second element is the infamous Categorizer that processes all e-mail passing through the Hub Transport, regardless of its source. This is followed by the third element, the downstream Message Delivery, which delivers either to the fourth element, the SMTP Send (for remote Site delivery or an external connection) or to the fifth element, the Store Driver for local delivery within the Site. The terminology has changed slightly – the term Local Delivery now means to a mailbox that resides in the same Active Directory Site as the message sender. For Exchange 2003 this term used to mean on the same server.

E-mail enters the Hub Transport in one of four ways. First, on each database there are messages sent by users, sitting in their Outbox. These are picked up by the local store service and dropped onto the submission queue of the local Hub Transport. (This server will be in the local Active Directory Site.) Through the Queue Viewer (in the tools section of the Exchange Management console) the submission queue is easy to see and should be empty.


From the Exchange Management Console Tools you can go to the Queue viewer

Secondly, any Receive connectors run by the Hub Transport may receive messages from other systems, including Exchange 2003 servers (sending messages through a Routing Group Connector through the MS Exchange Transport Service). There are by default two Receive connectors on any standard installation of a Hub Transport, the Default and Client Receive connectors.

Thirdly, there may be other agents that interface with the Hub Transport that forward mail for processing, and finally there is the traditional Pickup folder as used by any SMTP-based engine. Any correctly formatted file will be picked up as an SMTP message and duly processed. The MS Exchange Transport Service runs a whole series of generic processes, including the built-in antispam features, TarPitting and IPConnection Throttling. Any third-party antispam agents will also be run at this stage.

The big Cat
At the centre of everything is the Categorizer. All e-mail goes through it. The Categorizer has many elements, and at the start of the process lies the Submission queue. All messages initially sit in this queue, and from there the Categorizer leaps into action. In principle, this process is not that different from its predecessor insofar as it picks up messages from the Submission queue and performs a series of checks before sending the message onto the appropriate queue. These checks include resolving the recipient list, enumerating distribution lists, ensuring that e-mail does not exceed any global or specific size limits, ensuring that the format is correct and does not require conversion to a foreign format, and processing the destination and deciding the next hop in its transmittal to the final destination.

Built into the Categorizer are a series of agents that have additional roles. All messages are checked by these agent processes and they include an optional antivirus check, followed by the core Transport agent and, if configured, the Journaling agent will fork messages off to an additional destination. The last agent is the RMS pre-licensing agent. This has the job of checking for Rights Management System (RMS) licence restrictions that may apply to a mail message.

Cat key
The Categorizer is still one of the backbone processes of Exchange in that it’s the main engine for routing e-mail. Once the queue has been established and the message added, the Categorizer progresses onto the next message.

The next downstream component is Message Delivery. Messages delivered out of the Categorizer will be delivered to one of three possible queues. Messages destined to the SMTP engine will be queued on a Remote Delivery queue for out of Site transmittal to another Hub Transport. Post-processing of the message may yet take place as the message may be converted from 8-bit to 7-bit (standard SMTP). The connections made by the SMTP service will use Kerberos (authentication) and TLS (encryption) for security. These connections are broadly equivalent to Exchange 2003 connections across a Routing Group Connector.

The second queue in Message Delivery is Foreign Queues and, as the name suggests, relates to e-mail transmittal to foreign systems (using SMTP) typically to the “Drop directories” of a remote smart host.

The third and last queue proper is for internal delivery of e-mail to one of the Mailbox role holders. This may be on the same physical server or, on larger installations, a separate Mailbox server. The message is delivered to the actual Mailbox Store (via the Store Driver) through an encrypted Remote Procedure Call.

These three queues should always be zero in a well-run system. The appearance of queued messages typically indicates a connection failure.

There are two other components of Message Delivery. The Unreachable Queue will hold any messages that have resolved correctly to a valid address but for one reason or another cannot be delivered. This may be caused by a correct and valid DNS resolution for a remote smart host, but a failure in actually connecting to the resolved IP address. The Connection Manager determines if (for instance) Journaling is enabled and has caused the message to be “forked”. The second message is redirected back through the Message Submission queue in the Categorizer for delivery to the journal address.

All in all
The whole process has gained sophistication from the basic message forwarding of old. When you see how e-mail is processed and the complexity of the process, it’s obvious that development is an expensive business and it’s easier to understand why Exchange licences cost so much and why Exchange has pretty much wiped out the competition in the world of e-mail.