I’m still working on the way to setup LAG groups with VMware NSX cause LACP is not supported with NSX only two teaming policies: Route Based on Originating Port or Route Based on Source Mach Hash.
Not to get too off track, but the number of active uplinks configured on the vDS must all be configured with VTEP’s. it’s not possible to isolate uplinks to specific port groups, as NSX will provision a VTEP vmkernel interface for every active uplink. To isolate uplinks for non-VXLAN traffic with a multi-VTEP configuration, a separate vDS should be provisioned and the non-VXLAN traffic moved to that vDS, or a single VTEP teaming policy can be used, and the correct uplinks set to active for the desired port groups…
What I’ve learned thus far is that if you use VXLAN VTEP’s on a vDS you must use the above teaming policies. If for example you want you use the teaming policy IP hash for iSCSI on a vDS, this iSCSI teaming policy will need to be on a seperate vDS altogether with it’s own uplinks from the physical network.
I found our that my favorite IP Hash policy is not supported in a multi-VTEP in a way with NSX and if you must use LACP with NSX, with IP Hash, you can only use it will one VTEP per vDS. I wonder if that means you can use NSX on a single vDS for iSCSI and for iSCSI to span L3 between nodes; I’m going to have to ask a VMware Engineer about this one…
I think it’s worth mentioning if you use LACP with NSX you might want to use a “SRCDEST” load balancing method on the switch forming the bond, because LACP will always be looking for the VXLAN-encapulation, over UDP, it will want to know in advance the source and destination ports as-well as the vLAN. So this to me means that you will want to choose the “Source and Destination IP Address, TCP/UDP and VLAN” load balancing mode.
I still need to test this out so i can whittle down the use-cases of this in production, but this is my rant on LACP in NSX.