KISS and Tell

As an Exchange administrator my guess is that your involvement in the money spent on your hardware is minimal. In other words, someone else holds the purse strings. For most of us, the annual budget review is a huge disappointment and those long-held dreams of new servers, routers and racks are dashed in favour of money spent on the MD’s new Aston Martin. There is a universal rule that we all live within a budget and it is never enough. But what if you could demonstrate that it is possible to truly save money in IT – and I don’t mean by outsourcing the entire IT department. (Why would you ever do that?)

The mantra of Microsoft has always been to save us money. But now I truly believe that the much bandied-about phrase of “lowering the Total Cost of Ownership” has actually come to fruition. You only need to compare the ratio of support staff to desktops in 1998 and 2008. Without question the number of staff required has dropped as the IT department has regained control over the desktop. That brief excursion end users enjoyed towards total desktop independence with the first PCs and the likes of Windows 98 has come to an end. Thank goodness control has been re-established through the wonderful XP and GPOs. Yet the pressure to cut costs never ends and the demand for more resources only ever increases.

You’ll have noticed the appearance and disappearance of what I call IT fashion statements. Some 20 years ago it was the humble PC itself that was “in fashion”, followed by the invasion of PC-based servers into the dark and satanic confines of the mainframe server room. Intel architecture matured and grew, like a cuckoo in the nest pushing out its rival mainframes.

Passing phases
The latest technology soon becomes old. Bright yellow 3/4in coaxial cable with big aluminium boxes sporting a bee sting is replaced by thin grey aerial cable with T-pieces, which is soon eclipsed by nasty grey telephone wire bunched together in huge bundles connected to patch panels and on to hubs, which in its turn is replaced with switches. The terms “flood wiring” and “RSJ45” then percolate the conversation and are later replaced with 100MB and even gigabit networks. You know you have lost the plot if you still sport a 10MB Ethernet. Life is not worth living if your inventory still includes 386s.

The pressure never ends.

Various technologies become fashionable and then fade away. Take the example of disk technology. Some years back if you were anyone at all you derided the bog-standard IDE disks in favour of high-performance SCSI disks. They were three times the price, but the performance improvements were worth the expense. Lately, SCSI (in its various iterations) has been eclipsed by SATA and SATAII disks that challenge SCSI performance at half the cost. The original primary/secondary limitation has been overcome and an impressive series of SATA disks can now emulate the SCSI chains of old.

Just a few years back the ultimate answer to storage appeared from the hardware vendors in the form of a storage area network (SAN). This expensive technology clearly marked its adopters out from the rest of the pack. Vast arrays of “inexpensive” disks may be arranged in any number of ways to be presented to the server processors as logical units of raw space. With the wide adoption of gigabit networks, a similar technology has appeared offering a similar solution: the network area storage device.

Alternative route
In 2008, with intense pressure to keep costs down, Microsoft has appeared to change tack. Within its own internal IT department this drive to reduce costs has manifested itself in the adoption of lower-cost direct attached storage (DAS). This is the first hint of a fashion now going out of fashion. In the same way that lower-cost mini mainframes (VAX, HP and Prime) took down the big, expensive IBM, ICL and HP mainframes, in the same way that PCs took down the mini mainframe, the high cost of a SAN is now being challenged by high-availability software built into Exchange 2007.

The enabling technologies of cluster continuous replication (CCR), local continuous replication (LCR) and standby continuous replication (SCR) are offering an alternative route. Although we shouldn’t forget that SAN technologies are great, and advances mean that SANs may be reliably replicated across the WAN, this comes at a cost. I suspect the intrinsic cost of a SAN (and not the profit margin taken on purchase) will never compete with DAS.

Internally, Microsoft IT reckons on at least a 70 per cent saving on “per gigabyte” storage using DAS over SAN. I believe we must assume that this is on capital cost of new equipment rather than the reuse of existing kit. This is not unreasonable in that the other major cost of re-equipping the Exchange infrastructure is the mandatory 64-bit server hardware.

Demands on storage have taken another quantum leap. Not many years back it was normal to have mailbox sizes in the region of 200 to 300MB with the odd recalcitrant user demanding and getting a gigabyte or two. (In my experience the directors of many companies tend to revel in these extended privileges.) Today, Microsoft IT is looking to offer mail store quotas from 500MB to 2GB as the norm. In a nutshell, the new software technologies have allowed a complete rethink in the approach to hardware.

Looking at some of the other statistics in the world of Microsoft e-mail is quite astonishing. The company receives approximately 30 million messages per day (of which 28 million are immediately blocked). The mailbox count is 147,000. They reside on 34 clustered mailbox servers storing a maximum of 300TB of mailbox data. The majority (approximately 60 per cent) of the users run Outlook Anywhere (an RPC over HTTP connection) that reduces the load on the mailbox server by downloading e-mail to the Outlook client. Go back five years and the mailbox count was half that number at 71,000 and the data stored was only 7TB with a mailbox quota of 100MB.

It is likely that many other large corporations operate in a similar fashion. It is interesting to speculate what the statistics will be in a further five years.

High availability
The DAS-based solution relies on the use of CCR to give full fault tolerance at a low cost. Microsoft’s senior messaging architect Konstantin Ryvkin has been quoted as saying: “With more than 18 months of production use, I can say that CCR on DAS is a great solution for a mailbox server platform running Exchange Server 2007, especially for environments looking for highly available and cost-efficient designs.”

That is some accolade and, bearing in mind the conservative nature of those who run these critical systems, this is a major departure from what would have been seen as the norm, ie SAN.

Sometimes the advantages and disadvantages of a technology change are not always apparent. It has to be said that the old system worked. Exchange 2003 is an excellent product. Within its design parameters it has flourished and my own direct experience of Exchange 2003 failure has been more to do with poor procedures and design than with any intrinsic weakness in the product. Exchange 2007 has significantly moved the goalposts, with the principal advances addressing issues of storage and management.

A major fringe benefit of the implementation of a DAS-based solution is the simplification of the server construction. The old SANs are specialised, so the support is also specialised (and expensive). Using a simple server with basic RAID array technology (albeit using extremely high-capacity disks) means the support skills are that much simpler, and hence may be more widespread across the support staff.

From my experience of SANs, they do go wrong occasionally. When they fail the fallout is horrific. Heads will roll. As one of the Microsoft IT engineers reputedly said, two simple storage solutions (CCR over DAS) are better than one complex one (SAN).

The fault tolerance provided by CCR and the speed of failover are excellent. Over the years I have seen a number of fault-tolerant solutions implemented, but the acid test is the actuality of a failure. A scheduled failover is a confidence booster and maintains the skills of the support team. Using CCR over DAS failover can be easily initiated using the Exchange Powershell command “Move-ClusteredMailboxServer” for scheduled or non-scheduled failovers.

The ultimate KISS (Keep It Simple, Stupid) solution, methinks!