Deep Dive of the Day NSX tunnel endpoints in VCF 9.x: MTU, shared VLANs and the choices you can't undo
Tunnel endpoints (TEPs) are the VMkernel and Edge interfaces that carry NSX's Geneve overlay traffic between hosts and Edges. They're easy to set up and easy to forget, right up until a tunnel shows down or large packets quietly disappear. In VCF 9.x, a few of the decisions you make about them at deployment time are effectively permanent, so they're worth getting right first time, in production and in the lab.
Start with MTU. Broadcom's network fabric design requires the host and Edge TEP VLAN, and its first-hop gateway, to be set to 1700 or greater, and recommends 9000. Geneve will run at 1600, but Broadcom calls that out as not the recommended minimum, because it leaves too little room for future header growth. The overlay segments themselves have to be 200 bytes smaller than the TEP MTU, so a 9000-byte TEP network gives you 8800-byte segments at most. Geneve packets carry the don't-fragment flag, so anything in the path with a smaller MTU drops them rather than fragmenting. That's why jumbo frames have to be enabled end to end, including on intermediate switches and gateways.
The VCF 9 Edge workflow in vCenter adds its own twist. KB 422523 explains that it always uses the default "nsx-edge-multiple-vteps-uplink-profile", whose MTU you can't edit, because the workflow takes the Edge TEP MTU from NSX's Global Fabric Settings. Change that value after deployment and the Edges you've already built keep their old MTU. So set the Edge tunnel endpoint MTU under System > Fabric > Settings > Global Fabric Settings before you start, and check the value the wizard shows you as it runs.
VLANs deserve the same care. KB 456147 describes Faulty TEP alarms on VCF 9.0 and 9.0.1 where host-to-host and Edge-to-Edge tunnels are fine but host-to-Edge tunnels are down. The cause is an Edge TEP that kept its original VLAN after a new uplink profile was attached. The Edge TEP VLAN is set at deployment, and the only supported fix on those versions is to redeploy the Edge cluster. If you suspect it, SSH to the Edge, use get gateways to find the tunnel VRF, then vrf <ID> and get interfaces to see the VLAN it's really using.
Sharing one TEP VLAN between hosts and Edges is common in small sites and labs, and it's supported, with a condition. KB 411992 says that on VCF 9.x the Edge tunnel won't come up on a shared VLAN unless "NSX on DVPG" is enabled for the cluster hosting the Edges. You can enable it if NSX isn't a single-node deployment, or if the Edges sit on a cluster that doesn't host vCenter. A single-node NSX lab with vCenter on the same cluster has to scale NSX out to three nodes first, or simply use separate TEP VLANs for hosts and Edges. Daniel Krieger's evoila write-up is a useful picture of how the shared-VLAN design hangs together, written for NSX 4.1 rather than VCF 9: give host uplink profiles a standby uplink so a link failure can't black-hole Edge overlay traffic, remember that Edge uplink profiles don't support standby links, and keep the Edge fast-path interfaces on trunk segments.
For nested labs, the outer environment is usually what bites. Your physical or outer virtual switch has to pass frames large enough for the inner TEP MTU, so if it can't carry 9000, make sure every hop of the TEP path, gateway included, carries at least the 1700 minimum. In production, settle the TEP MTU and VLAN plan with your network team before bring-up, write the global fabric MTU into your build checklist, and treat a host-to-Edge-only tunnel failure as a VLAN or MTU question before you go looking anywhere else.
Based on Network Fabric Detailed Design (Broadcom TechDocs) Official KB 422523: Unable to change MTU on Edge fp-eth interfaces in VCF 9 Official KB 456147: Faulty TEP alarm and BFD tunnels down in VCF 9.0 Official KB 411992: Sharing the same VLAN TEP for ESXi host and Edge VM Official evoila: Sharing the overlay transport VLAN between ESXi TEPs and Edge TEPs (Daniel Krieger)
