https://blogs.technet.microsoft.

com/josebda/2010/09/03/using-the-multiple-nics-of-your-file-
server-running-windows-server-2008-and-2008-r2/

Using the multiple NICs of your File

Server running Windows Server 2008
(and 2008 R2)
★★★★★

★★★★

★★★

★★

JoseBarreto

JoseBarreto
Microsoft Corporation

MSFT

95,992 Points 12 4 4
Recent Achievements

New Gallery Storyteller Gallery Contributor II New Gallery Contributor

View Profile

September 3, 20109


 0
 2

IMPORTANT NOTE: This blog post was created before the release of Windows Server 2012,
which introduced SMB 3.0 and the new SMB Multichannel feature and significantly
improved SMB's ability to use multiple network interfaces. You can read more about SMB
Multichannel at http://blogs.technet.com/b/josebda/archive/2012/06/28/the-basics-of-
smb-multichannel-a-feature-of-windows-server-2012-and-smb-3-0.aspx

——————————————————
1 – Overview

When you set up a File Server, there are advantages to configuring multiple Network
Interface Cards (NICs). However, there are many options to consider depending on how
your network and services are laid out. Since networking (along with storage) is one of the
most common bottlenecks in a file server deployment, this is a topic worth investigating.

Throughout this blog post, we will look into different configurations for Windows Server
2008 (and 2008 R2) where a file server uses multiple NICs. Next, we’ll describe how the
behavior of the SMB client can help distribute the load for a file server with multiple NICs.
We will also discuss SMB2 Durability and how it can recover from certain network failure in
configuration where multiple network paths between clients and servers are available.
Finally, we will look closely into the configuration of a Clustered File Server with multiple
client-facing NICs.

2 – Configurations

We'll start by examining 8 distinct configurations where a file server has multiple NICs.
These are by no means the only possible configurations, but each one has a unique
characteristic that is used to introduce a concept on this subject.

2.1 – Standalone File Server, 2 NICs on server, one disabled

This first configuration shows the sad state of many File Servers out there. There are
multiple network interfaces available, but only one is actively being used. The other is not
connected and possibly disabled. Most server hardware these days does include at least
two 1GbE interfaces, but sometimes the deployment planning did not include the
additional cabling and configuration to use it. Ironically, a single 1GbE (which provides
roughly 100Mbytes per second of throughput) is a common bottleneck for your file server,
especially when reading data from cache or from many disk spindles (physical disk
throughput is the other most common bottleneck).
Having a single NIC has an additional performance downside if that NIC does not support
Receive-side Scaling (RSS). When RSS is not available, a single CPU services all the
interrupts from a network adapter. For instance, If you have an 8-core file server using a
single non-RSS NIC, that NIC will affinitize to one of the 8 cores, making it even more likely
to become a bottleneck. To learn more about deploying RSS, check
http://www.microsoft.com/whdc/device/network/NDIS_RSS.mspx.

2.2 – Standalone File Server, 2 NICs on server, teamed

One simple and effective solution for enabling the multiple NICs on a File Server is NIC
Teaming, also known as “Link Aggregation” or “Load Balancing and Failover (LBFO)”. These
solutions, providers by vendors like Intel, Broadcom and HP, effectively combine multiple
physical NICs into one logical or virtual NIC. The details vary based on the specific solution,
but most will provide an increase of throughput and also tolerance to the failure of a NIC or
to a network cable being accidentally unplugged.

The NIC team typically behaves as a single NIC, requiring only a single IP address. Once you
configure the team itself, the Windows Server and File Server configuration proceeds as if
you had only one NIC. However, NIC teaming is not something included with Windows
Server 2008 or Windows Server 2008 R2. Support for these solutions (the hardware, the
drivers and the configuration tools) is provided by the hardware manufacturer.

You can find Microsoft’s support policy for these types of solutions at
http://support.microsoft.com/kb/254101 and http://support.microsoft.com/kb/968703.
2.3 – Standalone File Server, 2 NICs on server, single subnet

If you don’t have a NIC teaming solution available but you are configuring a File Server,
there are still ways to make it work. You can simply enable and configure both NICs, which
will each need their own IP address. If everything is configured properly, both IP addresses
will be published to DNS under the name of File Server. The SMB client will then be able to
query DNS for the file server name, find that it has multiple IP addresses and choose one of
them. Due to DNS round robin, chances are the clients will be spread across the NICs on
the file server.

There are several Windows Server components contributing to make this work. First, there’s
the fact the File Server will listen on all configured network interfaces. Second, there’s the
dynamic DNS that automatically registers all the IP addresses under the server’s name (if
configured properly). Third, there’s the fact that DNS will naturally round robin through the
different addresses registered under the same name. Last but not least, there is the File
Client that will use one of the available IP addresses, giving a priority to the first address on
the list (to honor the DNS round robin) but using one of the others if the first one does not
respond quickly. The SMB client will use only one of the IP addresses at once. More on this
later.

What’s more, due to a feature called SMB2 durability, it’s possible that the SMB client will
recover from the failure of a NIC or network path even if it’s right in the middle of reading
or writing a file. More on this later as well.
It's important to note that applications other the file server and file client might not behave
properly with this configuration (they might not listen on all interfaces, for instance). You
might also run into issues updating routing tables, especially in the case of a failure of a NIC
or removal of a cable. These issues are documented in KB article 175767. For these reasons,
many will not recommend this specific setup with a single subnet.

2.4 – Standalone File Server, 2 NICs on server, multiple subnets

Another possible configuration is for each of the File Server NICs to connect to a different
set of clients. This is useful to give you additional overall throughput, since you get traffic
coming into both NICs. However, in this case, you are using different subnets. A typical case
would be a small company where you have the first floor clients using one NIC and the
second floor using the other.

While both of the IP addresses get published to DNS (assuming everything is configured
correctly) and each of the SMB clients will learn of both, only one of them will be routable
from a specific client. From the SMB client’s perspective, that is fine. If one of them works,
you will get connected. However, keep in mind that this configuration won’t give your
clients a dual path to the File Server. Each set of clients has only one way to get to the
server. If a File Server NIC goes bad or if someone unplugs one of the cables from the File
Server, some of your clients will lose access while others will continue to work fine.
2.5 – Standalone File Server, 2 NICs on server, multiple subnets, router

In larger networks, you will likely end up with various networks/subnets for both clients and
servers, connected via a router. At this point you probably did a whole lot of planning, your
server subnets can easily be distinguished from your client subnets and there’s a fair
amount of redundancy, especially on the server side. A typical configuration on the server
side would include dual top-of-rack switches on the server side, aggregated to a central
switching/routing infrastructure.

If everything is configured properly, the File Servers will have two IP addresses each, both
published to the dynamic DNS. From a client perspective, you have a File Server name with
multiple IP addresses. The clients here see something similar to what clients see in
configuration 2.3, except for the fact that the IP addresses for the client and servers are on
different subnets.

It is worth noting that, in this configuration and all the following ones, you could choose to
leverage NIC teaming, as described in configuration 2.2, if that is an option available from
your hardware vendor. This configuration might bring additional requirements, since each
of the NICs in the team are going into a different switch. The configuration of Windows
Server itself would be simplified due to the single IP address, although additional teaming
configuration with the vendor tool will be added.
2.6 – Standalone File Server, 2 NICs on “clients” and servers, multiple subnets

This last standalone File Server configuration shows both clients and servers with 2 NICs
each, using 2 distinct subnets. While this configuration is unusual for regular Windows
clients, servers are commonly configured this way for added network fault tolerance. Here
are a few examples of such server workloads:

 IIS servers that store their HTM and JPG files on file share
 SQL Servers that regularly send their database backups to a UNC path
 Virtualization Servers that use the file server as a library server for ISO files
 Remote Desktop Servers (Terminal Servers) where many users use the file server to
store their home folders
 SharePoint servers configured to crawl file shares and index them

You can imagine the computers below as part of the configuration 2.5 above, only with
more servers to the right of the router this time.
2.7 – Clustered File Servers, 2 NICs on servers, multiple subnets, router

If you are introducing dual network interfaces for fault tolerance, you are also likely
interested in clustering. This configuration takes config 2.6 and adds an extra file server to
create a failover cluster for your file services. If you are familiar with failover clustering you
know that, in addition to the IP addresses required for the cluster nodes themselves, you
would need IP addresses for each cluster service (like File Server A and File Service B). More
on this later.

Although we’re talking about Clustered File Services with Cluster IP addresses, the SMB
clients will essentially see a File Server name with multiple IP addresses for each clustered
file service. In fact, the clients here see something similar to configurations 2.3, 2.5 and 2.6.
It’s worth noting that, if File Server 1 fails, Failover Clustering will move File Service A to File
Server 2, keeping the same IP addresses.
2.8 – Clustered File Server, 2 NICs on “clients” and servers, multiple subnets

This last configuration focuses on file clients that are servers themselves, as described in
configuration 2.6. This time, however, the File Servers are clustered. If you are interested in
high availability for the file servers, it’s likely you would be also clustering the other server, if
the workload allows for it.

3 – Standalone File Server

3.1 – SMB Server and DNS

A Windows Server file server with multiple NICs enabled and configured for dynamic DNS
will publish multiple IP addresses to DNS for its name. In the example below, for instance, a
server with 3 NICs each in a different subnet is shown. You can see how this got published
on DNS. For instance, FS1 shows with 3 DNS A records for 192.168.1.2, 192.168.2.2 and
192.168.3.2. It’s important to know also that the SMB server will listen on all interfaces by
default.

3.2 – SMB Client and DNS

From an SMB client perspective, the computers will query DNS for the name of the File
Server. In the case of the example above, you will get an ordered list of IP addresses. You
can query this information using the NSLOOKUP tool, as shown in the screenshot below.

The SMB client will attempt to connect to all routable IP addresses offered. If more than
one routable IP address is available, the SMB client will connect to the first IP address for
which a response is received. The first IP address in the list is given a time advantage, to
favor the DNS round robin sequence.

To show this in action, I created a configuration where the file server has 3 IP addresses
published to DNS. I then ran a script to copy a file to a share in the file server, then flush the
DNS cache, wait a few seconds and start over. Below are the sample script and screenshot,
showing how the SMB client cycles through the 3 different network interfaces as an effect
of DNS round robin.

@ECHO OFF
:LOOP
IPCONFIG /FLUSHDNS
PING FS1 –N 1 | FIND “Reply”
NET USE F:=\\FS1\TEST >NUL
COPY C:\WINDOWS\SYSTEM32\MRT.EXE F:\
NET USE F: /DELETE >NUL
CHOICE /T 30 /C NY /D Y /M "Waiting a few seconds… Continue"
IF ERRORLEVEL 2 GOTO LOOP
:END
3.3 – SMB2 Durability

When the SMB2 client detects a network disconnect or failure, it will try to reconnect. If
multiple network paths are available and the original path is now down, the reconnection
will use a different one. Durable handles allow the application using SMB2 to continue to
operate without seeing any errors. The SMB2 server will keep the handles for a while, so
that the client will be able to reconnect to them.

Durable handles are opportunistic in nature and offer no guarantee of reconnections. For
durability to occur, the following conditions must be met:

 Clients and servers must support SMB2

 Durable handles must be used (this is the default for SMB2)
 Opportunistic locks (oplocks) must be used (this happens when files are opened)
(this is the default for SMB2)

For Windows operating systems, SMB2 is found on Windows Vista, Windows 7 (for client
OSes), Windows Server 2008 and Windows Server 2008 R2 (for server OSes). Older versions
of Windows will have only SMB1, which does not have the concept of durability.

To showcase SMB2 durability, I used the same configuration shown in the previous
screenshot and copied a large number of files to a share. While the copy was going, I
disabled Network1, the client network interface that was being used by the SMB client.
SMB2 durability kicked in and the SMB client moved to use Network3. I then disabled
Network3 and the client started using Network2. You can see in the screenshot below that,
with only one of the three interfaces available, the copy is still going.
4 – Clustered File Server

4.1 – Cluster Networks, Cluster Names and Cluster IP Addresses

In a cluster, in addition to the regular node name and IP addresses, you get additional
names for the cluster itself and every service (Cluster Group) you create. Each name can
have one or more IP addresses. You can add an IP address per public (client-facing) Cluster
Network for every Name resource. This includes the Cluster name itself, as shown below:

For each File Server resource, you have a name resource, at least one IP address resource, at
least one disk resource and the file server resource itself. See the example below for the File
Service FSA, which uses 3 different IP addresses (192.168.1.101, 192.168.2.101 and
192.168.3.101).
Below is a screenshot of the name resource properties, showing the tab where you can add
the IP addresses:
4.2 – How Cluster IP addresses are published to DNS

Note that, in the cluster shown in the screenshots, we have 5 distinct names, each of them
using 3 IP addreses, since we are using 3 distinct public Cluster Networks. You have the
names of the two nodes (FS1 and FS2), the name of the cluster (FS) and the names of the
two Cluster File services (FSA and FSB). Here’s how this shows in DNS, after everything is
properly configured:
4.3 – Cluster Name and Cluster IP Address dependencies

When your clients are using a File Server resource with multiple routable IP addresses, you
should make sure the IP addresses are defined as OR resources, not AND resources, in your
dependency definitions. This means that, even if you lose all but one one IP address, the file
service is still operational in that node. The default is AND and this will cause the file service
to fail over upon the failure of any of the IP addresses, which is typically not desired.
Below you can see the File Service resource with only one of the three IP addresses failed.
There is an alert, but the file service is still online and will not failover unless all IP addresses
fail.
5. Conclusion

Network planning and configuration plays a major role in your File Server deployment. I
hope this blog post has allowed you to consider increasing the throughput and the
availability of your File Server by enabling and configuring multiple NICs. I encourage you
to experiment with these configurations and features in a lab environment.

Tags SMB Storage Windows Server 2008 Windows Server 2008 R2 Windows Storage Server
 Comments (9)

Name *

Email *

Website

1. Anonymous

December 14, 2016 at 5:14 pm

Ned here. Our friend and colleague Jose Barreto from the File Server development team has
posted a very

Reply

2. Anonymous

December 14, 2016 at 5:14 pm

New blog post covers Multi-NIC File Server configurations: When you set up a File Server,
there are advantages

Reply

3. JoseBarreto

December 14, 2016 at 5:14 pm

 There are multiple solutions for NIC teaming provided and supported by different hardware
manufacturer (not by Microsoft). I suggest that you ask the question about switchport
flooding to the vendor providing the solution. You could also

search the web for "Link Agreggation Control Protocol" to read more about how it can be
done.

Reply

4. JoseBarreto

December 14, 2016 at 5:14 pm

Bart. That’s due to NetBIOS over TCP advertising both names on the same subnet. Not sure
if you are using NetBT/WINS for name resolution or if you’re using just DNS. Many people
these days disable NetBT, but some older clients and applications might still
require it. See
technet.microsoft.com/…/cc782733(WS.10).aspx and
support.microsoft.com/…/299977

Reply

5. JoseBarreto

December 14, 2016 at 5:14 pm

Andrei, section 3.3 is about durability, which is part of the

SMB2 protocol. You can get more details about the SMB2 changes in Windows 7 from this

presentation on SMB 2.1

Reply

6. Andrei Ungureanu

September 6, 2010 at 5:31 pm

I'm curious about the section 3.3. I've noticed some changes in the Windows 7 DNS Round
Robin behaviour that are NOT documented and I was wondering if the failover is built in
the SMBv2 code or it has something to do with the TCP/IP stack (changing the order of the
IP address in the list).

And at 3.2 you should detail the client version as the DNS round robin behaves different
based on the client, destination address selection, subnet prioritization setting, etc
 Reply

7. Jason

September 17, 2010 at 11:28 pm

When using NIC teaming. Is it ok to have both adapters connected to the same switch? will
there be any switchport flooding similar to what I have seen with NLB.

Thanks.

Reply

8. Bart

September 28, 2010 at 12:54 pm

I'm using the configuration as described in 2.3 on a DC with DNS installed. i'm receiving
multiple NetBT Event ID 4319 Errors stating "A duplicate name has been detected on the
TCP network. The IP address of the computer that sent the message is in the data." I did
not see an IP address in the data. The best I can tell is that the conflict is with itself when I
use nbtstat -n I see only the name of the DC I'm logged into. There is the one server name
for each IP addresses. If this is the case is this a problem? Thanks!

Reply

9. sergio a. bonanni

October 5, 2016 at 2:54 am

I’ve a simple problem for my PC (client): sometimes my lan card 2 (by Locus, 1Gbps) loses
the driver (!) and I become disconnected.
But I’ve another lan card 1 (onboard ASUS, only 100Mbps).
The Locus card doesn’t have a driver for Windows 10: I think this is the cause for my
connection problem.
I would know if there is a program that allow me to switch (automatically or manually) from
the first to the second card.
Thanks to everyone may help me

Reply
https://www.youtube.com/watch?v=vjxCrh-L_HM

https://community.hpe.com/t5/ProLiant-Servers-ML-DL-SL/Nic-teaming-in-Server-2008-R2-
X64/td-p/6341445

https://blogs.technet.microsoft.com/josebda/2010/09/03/using-the-multiple-nics-of-your-file-
server-running-windows-server-2008-and-2008-r2/

http://h20565.www2.hp.com/hpsc/swd/public/detail?swItemId=MTX_b8d6223891ca4cbaaba
3d949b4&sp4ts.oid=5177954