Clever clustering

Clever clustering

2 servers with the same MAC address present a problem for the switch
The following are common solutions to the problem of switch flooding:
‹ With unicast, the cluster members can be attached to a hub hanging off the switch. This way the switch has a one-to-one mapping for MAC to IP address and sends traffic only to the port associated with the hub. When it reaches the hub, the traffic is broadcast, which is default hub behaviour anyway.
‹ Use multicast and create static ARP entries/mappings so that the flooding happens only on the ports associated with the cluster (which is, after all, the intended behaviour).
‹ Use IGMP on the routers (and network cards).

These additional methods have been suggested, although success may depend on the behaviour of the switch hardware:
‹ Use port mirroring (Switched Port Analyzer in Cisco terms). This is a function of a switch whereby traffic destined for a port is also sent to a second port, usually for analysis such as packet sniffing. It requires some tampering of the MAC mappings to avoid switch confusion and therefore switch flooding.
‹ Configure all the cluster traffic to travel over a VLAN and not the entire switch(es).

Affinity affairs
Another important aspect of an NLB cluster is the network affinity. This is the method by which the traffic is directed within the cluster. Once a client session has been established, you want all further communication to and from that client to be with the same cluster node. If a client establishes an SSL connection, for example, with one of the nodes and further communication is picked up by another node, the connection will fail since the second node has no idea that the SSL connection has been agreed in the first place. Single affinity creates an affinity between a single client IP address and an NLB node. When the first request comes in from the client, it is serviced by a node as determined by the NLB service. Further traffic between the cluster and this client address is serviced thereafter by the same host.

Class-C affinity is similar to single affinity but it maps the cluster node with the class C address range from which the traffic comes. If a client site has multiple proxy servers, traffic traversing the ISA server may come from different source IP addresses, which could break IP affinity. Assuming all of these proxy servers are sitting on the same class C subnet, clients will keep the same session even if the source address changes. However, this situation won't often occur.

Client/server applications can keep connection state information within cookies, which means clients can connect to any node and keep the session alive without any renegotiation. However, this is not within the scope of Windows NLB.

Two-card trick
One of the areas of confusion when configuring NLB clusters is how to set up the second network card. In a simple MSCS cluster, one NIC is used for client/server traffic and the other is a dedicated NIC – usually on a separate subnet – for inter-host communication. Often, when installing an NLB cluster, people will take the same approach and, for example, will join two NICs on the main subnet (192.168.7.x) to form the cluster address (192.168.7.155) and add a second subnet (10.0.0.x) for the heartbeat connection. The idea is that, just as in a server cluster, the inter-node traffic runs across the dedicated heartbeat NIC.

Dual-NIC NLB cluster
The problem with this is that NLB clusters don't work the same way as MSCS clusters from this perspective. Whereas MSCS nodes communicate between them using the dedicated NIC, NLB clusters have no setting to configure such behaviour, and in fact all cluster (heartbeat) communication is broadcast from the NLB-enabled NIC. In practice, if you implement the system above, you've just wasted two network cards since the cluster won't use the "heartbeat" NICs at all. There is no way of controlling which traffic goes across which NIC except the binding order of the NICs, which we will configure later.

Generally, NLB clusters should always be configured using two network cards in each host. NICs are cheap and doing things this way could save you some effort. Servers want to talk to each other, and setting up a single NIC cluster where the nodes can't communicate just brings you hassle and pain. Now let's look at the traffic on each NIC.

We'll call the first card "front end", the second "back end". The front end hosts the NLB services, so the front-end network is the NLB-enabled network across which flows the incoming traffic towards the cluster IP address. The back-end network handles traffic to and from other back-end servers.
Let's assume you are starting from the point of two individual, single NIC servers with which you want to create the NLB cluster. These NICs have the following addresses:
NODE1 192.168.7.151/24
NODE2 192.168.7.152/24

At present, these cards host all the domain traffic between the servers and other hosts on the network. They are also registered in DNS so another host looking for "NODE1" finds 192.168.7.151. It's easier to keep things this way, so these cards will be the ones that handle the main LAN traffic.
The second cards will be on the "front-end" network and will host the cluster IP address, not the existing cards. The front-end network will host traffic coming into the cluster and cluster heartbeat traffic.

An optimized network load balancing cluster
The front-end network cards are assigned a dedicated IP address (DIP) and the cluster IP address. (The front-end NICs on both nodes also host the same cluster IP address, but this apparent clash is taken care of by the NLB service.) The DIPs are used for nothing more than hosting the cluster address. The frontend network cards’ configuration should look like the following figure.

The frontend NIC has a DIP address configured only
The IP address and subnet mask are the only settings configured on this network. There are no default gateway, DNS or WINS settings. NLB is enabled, however, as this interface is hosting the cluster address.

The back-end network is where most of the action happens. This NIC is configured exactly as it was before you even considered putting an NLB cluster on the box. The IP address is registered in DNS (and perhaps WINS), and the gateway and DNS server settings populated. Also, this NIC is sitting at the top of the network bindings list, so outgoing traffic will go through this network.

[Image Missing]

Further configuration
‹ Configure the binding order with the back-end network at the top. This will ensure all outgoing network traffic goes through the back-end NIC.
‹ Configure NLB on the front-end network card so that the dedicated IP address is 192.168.7.156 (the primary address of this NIC). The cluster IP address (192.168.7.155) must be the second IP address listed on the NIC.
‹ Once the cluster is configured, you can configure port rules so it listens for specific traffic types.
‹ Register the cluster in DNS manually. Create a DNS host record on one of your domain DNS servers corresponding to the NLB DNS name and IP address (in this case, "nlbcluster.lochwallace.com" and 192.168.7.155).
‹ Check that traffic is coming in at the front and out at the back. A simple way to so this is to use "netstat –an" on the back-end cluster. Log on to any other server on the LAN and establish a connection to the cluster address. On the active cluster node, run "netstat –an" and check which servers it has TCP/IP connections with. You'll notice that the traffic is coming from the back-end addresses and not the cluster address.

There is a welcome by-product of this configuration in which incoming traffic goes to the cluster IP and outgoing traffic leaves the back-end single network cards: one of the methods to avoid switch flooding on the NLB network is to plug these NICs into a hub and the hub into a switch. It's not so common, though, to see gigabit hubs, so this could pose a speed problem in some cases. However, the request traffic is typically a lot lower in volume than the response traffic, so you could plug the front-end NICs into a low-speed hub.

Essential summary

• NLB clusters are often misconfigured owing to a lack of information on the best way to set them up.
• In unicast mode, NLB nodes share a virtual MAC address. This can cause traffic flooding, in which the hardware is overwhelmed by the amount of information being transmitted. Multicast mode minimises this
• problem by sending traffic to specific nodes.
• NLB clusters should always be configured using two network cards in each host: a front-end card that hosts the NLB services, and a back-end card handling traffic to and from other back-end servers.

Business viewpoint

• NLBs provide a means to have clustered computers on a Windows network so ensuring high availability of corporate IT resources.
• Load balancing is used to spread work between two or more computers, network links, or other resources to make maximum use of available hardware.
• It's important to configure NLBs correctly to gain maximum benefit and avoid problems on the company network.
  
  
  Further reading
  
  Windows Product Help
  Contains lots of info on NLB clusters. Go to the Windows Help and Support Center in Windows Server 2003, open “Multiple Network Adapters” and “Multiple network adapters in unicast mode”.
  Windows Server 2003 Technical Reference site