Networking (Standard and Distributed)

This section will cover both standard networking and distributed networking (vCenter server required for distributed networking), rst I will start with standard
networking.
Standard Networking
The VMkernel can create virtual switches called vSwitches, the virtual machines virtual NIC's are plugged into vSwitches these are then mapped to the physical NIC's
on the ESXi server. This means many virtual machines can use the physical NIC's on the ESXI server. The other clever thing VMware can do is that if two virtual
machines communicate with each other on the same vSwitch no physical network trafc is generated. The VMkernel moves the data into memory seamlessly from one
virtual machine to another without ever involving the physical network interface. Knowing this knowledge you can design your VM's that communicate with each
other to use the same vSwitches thus increasing performance.
vSwitches are VLAN aware and can control outbound trafc using a VMware technology called trafc shaping, you can also impose security settings. A vSwitch can
contain 0,1 or many physical NIC's assigned to it, a vSwitch without any NIC's attached to it is called a internal vSwitch, as it only allows communication with the
host. The internal vSwitch could be used as staging area before moving it into Production, one note to remember is that vSwitches do not communicate with one
another.
The only drawback of internal vSwitches is that you cannot carry out vMotion events without rst disconnecting users from the virtual machine. Since they are internal
to the ESXi server, we cannot guarantee that users would have a continuous connection to the virtual machine during the vMotion event.
A vSwitch with one physical NIC gives you basic connectivity to the outside world. This might be suitable for vMotion or one that does not require fault tolerance
(FT), ideally this would be for testing, development environments, etc. A vSwitch with two physical NIC's gives you fault tolerance (FT) and load balancing, this
would be idea for vMotion and could be used in Production environments.
You can have up to 20 physical NIC's in an ESXi server of any link speed, there are now 56 ports by default on a vSwitch and this is congurable for up to 4,088 ports.
Port Groups
vSwitches can be divided into smaller units called port groups, there are three types of port groups
Virtual Machine
Service Console
VMKernel (for vMotion, VMware FT logging and IP storage)
You could create one big fat vSwitch and connect all the NIC's to this but you should really separate the network trafc on separate NIC's. Try and name your port
groups with a meaningful name so that if support up the service they can see what the port group is used for, idea names would be vlan25, vlan26. An important note
to remember is that if you rename a port group virtual machines become orphaned from the switch because the name of the virtual switch to which the virtual machine
is attached is held in the virtual machines conguration le (.vmx), so bear that in mind if you rename a port group that many have many virtual machines attached,
that's a lot of work to resolve the problem and many even need a script to resolve it. An example of this that I change my iSCSI network port group to iSCSI networks
(additional s in network) and as you can see in the image on the left had 6 virtual machines attached, a simple name change now has orphaned all 6 virtual machines,
see the image on the right (the images were taken from the summary screen of the ESXi server)
Here is a guide on how to create the various standard vSwitch
Internal standard vSwitch
To create a internal vSwitch follow the notes below, remember to name your vSwitch appropriately
In vSphere Client, select the ESXi host 1.
Select the configuration tab 2.
In the hardware panel, select Networking 3.
Click the Add Network link 4.
Choose Virtual Machine and click next 5.
Make sure no network adapters are selected and then click Next 6.
In the Port Groups Properties dialog box, type a descriptive and unique name as internal0-vmware1 7.
Click Finish 8.
You should end up with something like below
Teamed Standard Switch
To create a teamed vSwitch it is the same as above, apart from selecting two or more physical adapters, you should end up with
something like below
Standard vSwitch with
VLAN support
ESXi supports two main methods of enabling access to VLAN's
External switch tagging (EST) - simply plug in the relevant NIC's to the relevant VLAN's and set the virtual machines IP settings
for that network. This will consume a lot of NIC's for each VLAN you need one NIC.
Virtual switch tagging (VST) - the network interfaces are plugged into truck ports on the physical switch, truck ports allow
many VLAN packets to traverse them, even with just one or two NIC's, ESXi can allow multiple virtual machines to access many
VLAN's
To create a VLAN vSwitch follow below
In vSphere Client, select the ESXi host 1.
VMware - Networking
1 of 9
Select the configuration tab 2.
In the hardware panel, select Networking 3.
Click the Add Network link 4.
Choose Virtual Machine and click next 5.
Select the desired NIC's 6.
put a friendly name and the VLAN ID 7.
click Next and then click finish 8.
to add more VLAN ID's simply select properties 9.
select the add button and repeat the process as above 10.
Hopefully you should have something like below
VMKernel standard
vSwitch for vMotion
When you create a vMotion port group you will be asked for the following details
IP Address
Subnet Mask
Default gateway (this is optional)
You can use vMotion across routers using the gateway but this is not a preferred way, normally ESXi servers are attached to the same
network to increase performance and other problems.
To create a vMotion vSwitch follow below
In vSphere Client, select the ESXi host 1.
Select the configuration tab 2.
In the hardware panel, select Networking 3.
Click the Add Network link 4.
Choose VMKernel and click next 5.
select the desired NIC's 6.
In the port groups dialog box type a friendly name, in my case I type "vMotion" 7.
The select the "use this port group for vMotion" 8.
Set the IP address and subnet mask and gateway if required 9.
Hopefully you should get something like below, you may get a license warning come up, it still will create the port group.
Conguration and Problems of vSwitches and Port Groups
To check to see if a NIC has a problem connecting to the network, go to the Networking screen and check if there is a red-cross against the NIC, see the below image,
use standard networking fault nding to correct the problem.
There are a number of conguration settings on vSwitches and Port Groups
Increase the number of
ports on a vSwitch
You can now change the number of ports up to 4088 with ESXi 4, if you have a large number of VM's you may have to increase this value
To increase the number of ports on a vSwitch
In vSphere Client, select the ESXi host 1.
Select the configuration tab 2.
In the hardware panel, select Networking 3.
Click the properties of the vSwitch that you wish to change 4.
make sure the vSwitch option is highlighted 5.
select the edit button 6.
then using the down arrow, select number ports that you desire 7.
click OK to implement 8.
VMware - Networking
2 of 9

Setting speed and duplex
on physical NIC's
You can change the speed and duplex of any on the physical NIC's in your ESXi server, generally they will auto-negotiate
In vSphere Client, select the ESXi host 1.
Select the configuration tab 2.
In the hardware panel, select Networking 3.
Click the properties of the vSwitch that you wish to change 4.
select the "Network Adapters" tab at the top 5.
select the physical NIC and click the edit button 6.
choose the desired speed and duplex 7.
click OK to implement 8.

Setting vSwitch and Port
Group policies
I am going to cover these briefly, you have three additional areas that can be changed
Security
promiscuous mode - allows a NIC to collect all network packets including ones
that are not intended for it
MAC address change - allow for the MAC address to change, this may be a
requirement for things like Microsoft cluster and loading balancing
Forged Transmits - allows a VM to send traffic under a MAC address that is
different from that of the VM, Microsoft Network Loading balancing may break
if you reject this
Most systems the default option will be fine
Traffic Shaping
Traffic shaping is ESXi method of controlling outbound traffic generated by VM's you
can control the bandwidth, traffic shaping is not dynamic once set those values are
set in stone. Not sure why you would use this feature as i like to give my servers as
much bandwidth as possible but you can make use of it.
NIC Teaming
NIC teaming (bonding in the Linux world) is a fault tolerance, load balancing feature,
you can configure the load balancing, what to detect to cause a failover, failback
when the problem has been solved and select particular NIC's to standby or active
NIC's
This is pretty much the same as all NIC teaming in other O/S
VMware - Networking
3 of 9
Service Console Network
ESXi 4 the networking architecture treats the Service Console as if it were just another VM connected to a vSwitch, this default vSwitch is called vSwitch0, the service
console ports have a special name vswif which stands for virtual switch interface. You should try to have a backup service console port or try to protect the existing by
making use it is fault tolerant to network failures. If you were to lose the service console to have to go to the command-line to either re-install it of repair it. Here are
some commands that will help in recovering a failed service console
Service Console troubleshooting
Correcting your NIC
selection
# Displaying the vSwitch configurations
esxcfg-vswitch -l
# Display all the NIC's
esxcfg-nics -l
# Link another NIC to the switch
esxcfg-vswitch -L vmnic1 vSwitch0
# Unlink a NIC from a Switch
esxcfg-vswitch -U vmnic0 vSwitch0
Correcting IP settings
# Display current IP address settings
esxcfg-vmknic -l
# Change the Service console network
esxcfg-vmknic "Management Network" -i 192.168.0.190 -n 255.255.255.0
# Restart the network services
service network restart
Configuring the Cisco
Discovery Protocol
# Display current CDP settings, it should return "listen"
esxcfg-vswitch -b vSwitch0
# Make the vSwitch bidirectional
esxcfg-vswitch -B both vSwitch0
# Display CDP data
esxcfg-info | more +/CDP\ Summary
Change the MTU value
# Change the MTU value
esxcfg-vswitch -m 1500 vSwitch0
Distributed Switches
Distributed virtual networking (DVN) is completely new to version 4, it enhances the network layer of the VMkernel, in uses features such as below, however you
must be using vCenter in order to create DvSwitches
private VLAN's
Network vMotion
API support for 3rd parties
VMDirectPath I/O
new NIC drivers within guest o/s using vmxnet3.sys
At a basic level a DvSwitch is a global switch, rather than congure vSwitches for each ESXi server, you create a DvSwitch that can be applied to every ESXi server,
they also implement private VLAN's (PVLAN) within VMware. DvSwitches can do all the things a normal vSwitch can do, they also offer more ports than a standard
vSwitch.
DvSwitches use distributed virtual uplink (DvUplink) ports, it is merely a container for holding a reference to a physical NIC. When you create a DvSwitch the system
attempts to set the correct number of DvUplink ports for you, it does this by looking at all the ESXi servers and searching for the ESXI server with the most uplinks.
DvUplinks can be renamed this allows you given more meaningful names.
VMware - Networking
4 of 9
I am going to combine a number of items to explain how to create and congure a DvSwitch, I am going to create a DvSwitch, add a port group and then add another
port group that uses vMotion. This should give you a fairly good understanding of DvSwitches, I will nish off looking at the advanced features (actually they are
pretty much the same as a standard vSwitch).
First let me explain what I had already congure, I had a vSwitch congured with two port groups one for the private LAN and one port group for vMotion (don't
worry too much about vMotion for the moment I will be covering this in greater detail in a later section).
Current setup and freeing
up some NIC's
This what I had setup before, a vSwitch with two port groups. Each ESXi server I have (vmware1 and vmware2) has this vSwitch setup.
The first thing I do is free up some NIC's by removing the Physical adapters from both ESXi servers, you should end up with something like below, you can keep the groups in
case you want to restore back to what you had.
Creating a DvSwitch
To create a DvSwitch first go to the home page in the vCenter and select networking. You can either select the "New vNetwork Distributed switch" icon or select the "Add a
vNetwork Distributed Switch" in the main screen, you then get the the below screen
Select vNetwork Distributed Switch Version 4.1.0 (unless you need version 4.0) and click Next
I changed the name to "Private DvSwitch" but you can call it whatever you want, because I have only one NIC to offer from each of my ESXi servers (vmware1 and vmware2),
I change the "Number of dvUplink ports" to one. In the real world you probably will have many NIC's to add so this can be increased, click next when finished.
Expand the ESXi server which then should display the available NIC's (if you remember these are the ones we free 'ed up earlier). Select all NICs and then click next to
continue
You now get a summary screen, notice the "automatically create a default port group" is ticked, you can leave this as we will make use of this group. Click next to create
the DvSwitch. After renaming the dvSwitch to "dvswitch-NIC2" and renaming the port group to "Private" (you can do this by right-clicking each item and selecting edit
settings, then change the name at the top", you should have something like below
VMware - Networking
5 of 9
If you select the dvSwitch-NIC2 icon you get a number of tabs appear, to view the ports select ports, remember we selected one dvUplink port for each ESXi server which
gives us two in total, you can also see that "Connectee" comes from each ESXi server
You can confirm the ESXi servers by then select the "Hosts" tab
So what you have now is a DvSwitch with two NIC's attached (one from vmware1 and one from vmware2), you can makes changes to the NIC's and it will be
implemented on both ESXi servers, this can be very handy if you have lots of ESXi servers to manage as most congurations only need slight tuning. The private port
group will be empty as we have no VM's using it, I will be discussing how a VM can make use of this in my virtual machine section.
Next we will create a vMotion port group within our DvSwitch, follow below
Create a vMotion port
group
Right click on the DvSwitch icon and select "New port group"
I have change the name to "vmotion", but again you can choose whatever you want
Once the port group is created you need to add the vMotion information for each ESXi server, you will need to provide an IP address and the subnet mask for each ESXi
server using the vMotion.
************** YOU WILL NEED TO REPEAT THE BELOW STEPS FOR EACH ESXI SERVER ************
So from the Home page select "Hosts and Clusters", from there select the first ESXi server (in my case it is vmware1) and select the configuration tab. Select Networking
from the hardware then select the "vNetwork Distributed Switch and you then should have a screen similar to below. Select the "Manage Virtual Adapters" ,then select add
at the top
You should then have the screen below, select "New Virtual adapter"
then select "VMKernel" (this was the only choice for me)
Now select the vmotion port group from the first drop down list and also tick the "Use this virtual adapter for vMotion"
VMware - Networking
6 of 9
Now enter the IP address and subnet mask, when you repeat this step for the other ESXi server remember you use a different IP address but keep it on the same subnet
Next comes the summary screen, as you can see the vMotion port group now has a new VMKernel port with the IP address that you choose
************* NEW REPEAT THIS STEP OF OTHER ESXI SERVERS THAT YOU WANT TO USE VMOTION *********************
Once you have configured all the ESXi servers, if you select from the "home page" -> networking, then select the DvSwitch group icon, then select the configuration tab you
will see the screen below. I have two physical NIC's from the ESXi servers and two IP address for the vMotion (one each for each ESXi server). If you click on one of the IP
addresses you can see what NIC it goes to (follow the orange line), as you can see IP address 192.168.2.192 goes to vmnic2 vmware1, in this case. Also from this screen if
you look in the top right hand corner you can manage the DvSwitch by removing it, adding hosts, managing hosts (use this option to add additional NIC's), creating a new
port group and evening editing the existing DvSwitch.
You may have noticed that I have a VM using the private port group, I will cover this in greater detail in my virtual machine section.
Create a port group for the
Service console
This pretty much the same when you get to the "add Virtual Adapter" screen, I choose the service_console port group (this was a new port group that I added) and then
ticked the last option "Use this virtual adapter for management traffic, you will then be ask to enter an IP address and subnet for the service console
Once you have finished the screen should look like below see my new service console port group.
As you see a DvSwitch is very similar to a normal standard vSwitch, have a look at each of them side by side, NIC's are represented on the right-hand side and the port
groups are on the left-hand side, there is very little difference in them.
VMware - Networking
7 of 9
Standard vSwitch DvSwitch
I am not going to discuss how to remove a DvSwitch as it is in reserve order as creating one, rst remove from the VMKernel ports from each ESXi server rst
(otherwise when you try to remove the DvSwitch it will say it's in use), then remove each port group and lastly remove the DvSwitch. Most of the time you remove the
item by selecting it's icon and right-clicking, then selecting remove.....
The last topic to cover is the advanced sections of a DvSwitch, again there is very little difference than a normal vSwitch
Add/Remove NIC
You can add or remove additional NIC's form the DvSwitch view. From the home page select networking, then select the DvSwitch icon then select configuration from
the tabs, you should see a screen like below
Select manage hosts from the top right-hand corner, then select the hosts from which you wish to add NIC's to then select next, you should see the hosts and available
NIC's
Select the additional NIC's and select next, if the NIC's already belong to an existing port group it will ask you if you wish to migrate them, as you can see the one i
select already belong to an existing group hence the warning message. Continue on and the NIC will be added to your DvSwitch.
To remove a NIC just deselect at the "Select Physical Adapters page" and continue through the rest of the screens.

Advanced settings for a
DvSwitch
You can edit the setting of a DvSwitch by right-clicking it's icon and then select "edit settings", the first tab called properties you can change the name,
increase/decrease the number of dvUplinks and even put notes for anyone.
VMware - Networking
8 of 9
The next tab "Network Adapters" just displays the available ESXi servers and the available dvUplinks
The last tab "private VLAN" you can add primary or secondary VLAN ID's
Advanced settings for the
DvSwitch adapters
I am not going into much detail on this as it is pretty much the same as the standard vSwitch
Security - covers the promiscuous mode, MAC address change and the forged transits
Traffic shaping - covers Ingress traffic (traffic that comes from outside a ESXi server - inbound traffic management) and Egress activity that comes into the ESXi
server from either virtual machines or the ESXI server itself
VLAN - select the VLAN type and VLAN ID's
Teaming and Failover - the same as a standard vSwitch but the NIC references are related to dvUplinks port values
Miscellaneous - allows you to enable port blocking
Advanced - allow override port polices (override the settings on the dvUplink group) and Configure reset at disconnect
Final Comments
DvSwitches are idea for large corporations, it can be a headache to manage lots of standard vSwitches, but bear in mind that you do require vCenter to use DvSwitches
which comes with a price and can be off putting to smaller corporations.
VMware - Networking
9 of 9