vSphere Networking Basics-Part1
In this two part series, I discuss some of vSphere’s networking aspects, something you probably have come across when using VMware software. In this first part, I’ll talk mainly about the vSphere Standard Switch (vSS) in context of setting one up and configuring it on a standalone ESXi host (not managed by vCenter Server). I’ll also go over VMkernel adapters, port groups and some of the standard switch settings.
In the second part, I’ll introduce another type of virtual switch called the vSphere Distributed Switch which is only available one vCenter Server has been installed.
The vSphere Standard Switch
A vSphere Virtual Switch, allows a number of virtual machines connected to it to communicate with one another, pretty much like their physical counterparts would when connected to a physical switch. In addition, the vSS bridges virtual networks to physical ones using the ESXi host’s network cards, on which the vSS has been created, as uplink ports to a physical switch.
By default, a standard switch is automatically created when you install ESXi. This is labelled vSwitch0 as per Figure 1.
Note: An uplink port is simply a designated port on a networking device such a switch or a router that connects or cascades one piece of networking equipment to another. Figure 1 shows how the two physical adapters – labelled 3 – on the ESXi host set up for this post have been designated as uplink ports.
Looking closely at Figure 1, you should also notice the headings Virtual Machine Port Group and VMkernel Port denoted by 1 and 2 respectively. I’ll explain what these mean in the upcoming sections.
Port Groups
A port group, as the name implies, is a grouping of switch ports. By applying a network policy to a port group, one can enforce security and traffic shaping rules. Additionally, if you have VLANs set up on your physical switches, you can assign a VLAN ID to a port group such that any vm on it will, for all intents and purposes, reside on that specific VLAN.
When you create a network enabled virtual machine, you are in fact connecting its virtual adapters to one or more port groups. Figure 2 illustrates this aspect. The vm properties, including the selected port group, are shown on the left-side of Figure 2 while the switch settings – where port groups are defined – are shown on the right. In this instance, I created two vSphere standard switches each having a distinct port group. Notice how each switch has its own dedicated uplink (vmnic0 and vmnic1).
TIP: If there’s a need to isolate one or more virtual machines from the rest of the network, whether physical or virtual, you can simply create a standard switch with no specified physical adapters (uplinks). Any virtual machine placed on the switch’s associated port group will still be able to talk to any other vm on the same port group but will be completely cut off from other networks. Such a setup comes in handy whenever you need to replicate, say, a live environment in keeping with the same layer 3 addressing (IPv4/6) but without impacting any of the production systems.
The VMkernel
The VMkernel network interface, adapter or port is basically a service provider used by the ESXi host to communicate with the outside world and the rest of the VMware based infrastructure. VMkernel adapters are created according to the type of services required be it vMotion, Fault Tolerance, Management or perhaps vSAN. A list of services necessitating of a VMkernel are depicted under Services in Figures 3 and 4. A VMkernel is assigned an IP address using either DHCP or one which is assigned manually, the latter option being, in my opinion, the most sensible one.
By default, vmk0 is the first VMkernel adapter created which is enabled for management traffic. Interestingly enough, you’ll still be able to connect to ESXi if Management is ticked off.
After creating a VMkernel you’ll come across something that looks like a port group but not quite. This is in fact a port. Depending on the tool used to create it, things can get somewhat confusing. For instance, when using the C# vSphere client, you’d be forgiven for thinking that the VMkernel is in fact a port group since the same icon is used for both (Figure 5).
vSphere Networking Maximums
Before I move on, I’m going to list a few vSphere 6.0 maximums related to standard switches. You will find the full list here.
- 512- Port groups per standard switch.
- 1016 – Maximum active ports per host.
- 4096 – Total virtual network switch ports per host.
- 4088 – Virtual network switch creation ports per standard switch .
Creating a virtual standard switch
There are a number of ways to create a standard switch. These include PowerCLI, ESXCLI, the standard and Web vSphere clients and the recently introduced VMware Host Client assuming you’ve made the transition to ESXi 6.0 update 2. Here’s a quick rundown.
ESXCLI
Use something like putty to SSH to the ESXi host and run the following command to create a 24 port switch called myVSS.
PowerCLI
Connect to the ESXi host using the Connect-VIServer cmdlet and then issue the following to create the same switch as per the ESXCLI command.
Note: Starting with ESX 5.5, switches are created elastic meaning there is no need to specify the number of ports since these are added dynamically as required. I’ve included the parameter just for completeness sake. In fact in the case of PowerCLI the ports parameter is completely ignored.
Thick vSphere Client
When using the vSphere clients, simply highlight the ESXi host on which you want to create the switch, change over to the Configuration tab and click on Add Networking as shown in Figure 6-9. Next, choose Virtual Machine (if this is not misleading, I don’t know what is) binding it to a physical nic as well as creating a new port group for it.
VMware Host Client
If you have ESX 6.0 U2 (or have installed the fling) you can also use the VMware host Client to create a standard switch. Once you log in, select Networking from Navigator and switch to the Virtual Switches tab. Click on Add standard virtual switch and fill in the details. It’s as simple as that.
Configuring a standard switch
For this section I’ll be using the C# vSphere client and an ESXi 6.0 U2 setup. Just as a reminder, vSwitch0 is created automatically as part of the ESXi installation process.
To view the switch’s configuration, highlight the host’s IP address or hostname, select the Configuration tab and select Networking under Hardware. Click on Properties as shown in Figure 6.
On the next screen you’ll find 2 tabs, one called Ports (probably for lack of a better name) and another called Network Adapters. I’ll start with the latter since I’ll refer to it later on. This is basically where you will bind the physical ESXi host’s network cards to the virtual switch. Binding two or more nics will provide for link aggregation and failover options.
Under the Ports tab – as far as vSwitch0 is concerned – you’ll find three items created automatically when ESXi was first installed. These are the vSwitch configuration, a default port group and a VMkernel. You’ll be able to add other port groups and VMkernels using the Add button at the bottom of the vSwitch properties window.
Once again, things can get a little bit confusing depending on the client or method used to create and manage networking. As shown in Figure 15, you’ll need to select Virtual Machine as the connection type if you want to create a new port group.
To view the virtual switch settings, just click on the Edit button. Here you’ll find 4 tabs related to the switch’s network policy comprising various options and settings that govern both the switch and how it handles traffic. Again, depending on the tool used, some options may or may not be available. For instance, NIC Teaming is unavailable when using the VMware Host Client (at least, I wasn’t able to find it listed anywhere).
Standard Switch Settings and Policies
The switch’s configuration is accessed by clicking on the Edit button while the vSwitch configuration item is highlighted. This brings up a window with the previously mentioned tabs. Keep in mind that the settings and options set at the vSwitch level will percolate down to every other item in the configuration list. That said, an override option is available for each and every item.
Let’s go over the settings / option found under each tab.
General
Here you’ll set the number of switch ports required (max. 4088) and the MTU (Maximum Transmission Unit) size. The latter is generally changed if jumbo frames are enabled on your networks.
Security
Under the Security tab, you’ll find 3 settings defining the security policy for the switch.
The settings available are as follows;
- Promiscuous Mode – If set to accept, any vm on the switch having a virtual adapter in promiscuous mode will be able to eavesdrop on all the traffic passing through the switch.
- MAC Address Changes – Determines what happens when the MAC address of a network adapter on a vm is changed. If set to Reject, all traffic emanating from the vm is dropped when the MAC address supplied does not match that found in the vm’s vmx configuration file.
- Forged Transmits – If set to reject, frames with spoofed source MAC addresses will be dropped.
Traffic Shaping
The traffic shaping policy, if enabled, regulates bandwidth and burst size on the switch. This may come in handy when a number of port groups are created on the switch and you wish to apply a traffic utilization baseline across all port groups. Yet again, such a policy can be overridden if required for each and every port group.
NIC Teaming, Failover and Load Balancing
If you have 2 or more physical ESXi nics bound to a virtual switch, you will be able to increase bandwidth availability using either Ether Channel or Link Aggregation Control Protocol (LACP). This may require familiarity with some advanced networking concepts (depending on your background) which I’m not going to discuss here. However, this is a great article which goes into great depth on how to configure ESXi to leverage LACP and EtherChannel as implemented on Cisco and HP switches.
Under the NIC Teaming tab you’ll find a number of failover options. In the example below, I have two physical nics set up as Active Adapters. This ensures that the uplink is maintained if any one adapter goes offline. I could also move one adapter to the Standby Adapters group. When you do this, the standby adapter will only become active when all the nics within the Active Adapters group fail.
For a complete listing of the available NIC Teaming, Failover and Load Balancing options please visit here.
Configuring a port group
Most of the settings just covered equally apply when configuring a port group, the main difference being those found under the General tab.
The Network Label setting is self-explanatory. This is the name you assign to the port group and is what you see listed when you connect a vm’s virtual adapter to a network. The VLAN ID setting is on the other hand slightly more interesting albeit optional. If you have VLANs set up on your physical switches, ESXi operates in 3 different ways to take VLANs into account, these being;
- External Switch Tagging (EST) – This is required when VLAN tagging of packets is performed on the physical switch. The ESXi host’s network adapters are in this case connected to access ports on the physical switch. The VLAN ID must be set to 0.
- Virtual Switch Tagging (VST) – Contrary to EST, all VLAN tagging is carried out by the virtual switch before leaving the host. Host network adapters must be connected to trunk ports on the physical switch. Port groups connected to the virtual switch are assigned a VLAN ID between 1 and 4094.
- Virtual Guest Tagging (VGT) – With VGT, all VLAN tagging is done by the virtual machine. VLAN tags are preserved between the virtual machine networking stack and external switch when frames pass to and from virtual switches. Host network adapters must be connected to trunk ports on the physical switch. For a standard switch the VLAN ID of port groups with VGT must be set to 4095.
Configuring a VMkernel
There are 2 main aspects to configuring a VMkernel, its IP address and the services it provides. Under the General tab you can specify a name for the VMkernel, the services provided as well as an MTU value if you’re using anything other than the default value of 1500. Under the IP Settings tab you’ll find settings for the IP address and network mask. The remaining tabs are again identical to the ones previously covered with all the settings being inherited but which can nevertheless be overridden.
Conclusion
This concludes the first part of the series. I think I covered the salient aspects of vSphere standard switches and even though there’s probably more that can be said, I need to draw a line somewhere lest this ends up being one ginormous post. Further details are to be found under the Networking section on the vSphere 6.0 documentation site.
vNIC
This Virtual Network Adapter is used by Virtual Machines to communicate with other network devices in Virtual or Physical environment.
VMKNIC
Its a specialized Vmware Paravirtualized adapter used by Vmkernel TCP/IP stack. This Specialized adapter has its own MAC address and used to connect the Vmkernel to services that it provides.
Vmkernel NIC provides mainly 5 services :
- Management Connectivity
- IP Storage
- NFS connectivity
- vMotion
- Fault Tolerance
VSWIF
Its again a specialized Paravirtualized adapter similar to vmxnet that was used only by the ESX service console in ESX Product. Since Service Console has been removed in ESXi so vswif doesn’t exist anymore.
VPORT
vPorts on a virtual switch provide logical connection points among virtual devices and between virtual and physical devices. You can think of them as virtual RJ-45 connectors.
Each virtual switch can have upto 1016 active ports with a max limit of 4096 ports per switch.
Each virtual switch can have upto 1016 active ports with a max limit of 4096 ports per switch.
PORTGROUPS
Portgroups are templates used to configure persistent network policies for virtual ethernet adapters. Group of Virtual Machine share common network configuration can be mapped same Portgroup.
UPLINKS
Physical Ethernet Adapter referred as Uplink in Virtual Networking. Uplink serve as bridges between Virtual and Physical Network. Virtual Ports of vSwitch connected to Uplinks are called Uplink Ports.
A Single ESX host can have maximum of 32 uplinks.
vSPHERE STANDARD SWITCH
vSphere Standard vSwitch is layer2 software switch that resides in Vmkernel and provides traffic management for Vms and Vmkernel traffic. Control plane and IO Plane sits in ESXi Host only so user needs to manage standard vSwitches individually. In Short, there is no centralized management for vSphere Standard Switches.
VSPHERE DISTRIBUTED SWITCH
Vsphere Distributed Switch is centralized Layer2 switch provides traffic management for Vms and Vmkernel traffic. Distributed switches are centralized switch spans across multiple ESXi hosts. This Distributed switch can only be managed by vCenter Server.
