This article is on how to setup a 10G fabric for the purpose of iSCSI/vMotion/Backups. In this article I’m using a Netgear M4300 8X8F switch. I’ve moved away from Cisco this past year and into the Netgear space since they work very well, their quality has improved greatly, and their performance is finally at par with Cisco. The CLI is very similar to that of Cisco and it’s greatly appreciated.
A few things to note with Netgear M series switches:
Before you can even make the vlan’s in a M series you must setup the vlan database much like in a Cisco switch for each vlan and then enable routing for each vlan. To do this you must type the following after entering the elevated privileged level:
vlan routing 100
Then after this is done you need to setup the vlans much like any other switch, for posterity I’ll show the configuration:
configure (not config t)
interface vlan 100
description ‘iSCSI vLAN’
ip address 172.16.10.15 255.255.255.224
Now in order for one to make this interface work with VMware, you have to look at two things; will this port be used for a single vlan or multiple vlans? – if you are going to use one vlan the configuration below is as follows:
switchport mode access
switchport access vlan 100
vlan participation include vlan 100
**NOTE** : (only specific “vlan participation” if the vlan in question is not the native vlan)
Now let take this a step further and make this support two vlan’s and more, for this example I’ll just use two vLAN’s: iSCSI and vMotion. But 1st we need to make another vLAN, vLAN 200. Return to the elevated privileged level if you are still in config level.
vlan routing 200
interface vlan 200
description ‘vMotion vLAN’
ip address 172.16.20.15 255.255.255.224
Now that the vMotion vLAN has been created we can now assign this to an interface, let use interface 10.
switchport mode trunk
switchport trunk allowed vlan 100, 200
vlan participation include vlan 100, 200
One thing I forgot to mention but honestly didn’t mention is that in order for you to get the most bang for your packets, you always want to use Jumbo Frames. As you can see above the mtu is set to 9000. What I like about the Netgear M series switches is you can make this change on a per port basis.
However one thing to note about Jumbo Frames, you need to enable it throughout the entire path the data will traverse. Meaning, if you have a iSCSI appliance, you must enable Jumbo on the network adapter that connects to the M series switch set to a MTU of 9000, then the ESXi host’s vSwitch that is associated with iSCSI needs to be set to 9000 and then the Port Group/VMkernel associated with the iSCSI network also needs to be set to 9000. Lastly, don’t forget to bind the VMNIC to the iSCSI adapter. If one link in the chain is set to the default 1500 and the rest is 9000, the chain is broken and the data won’t flow.
Always make sure the path is set to 9000 thru and thru.
Next up I’ll make a posting on how to setup a LAG and a MLAG for use with VMware from 5.5 and above and for use with NSX.