Three datacenters, one IP plan: identical isolated pods with NSX VPCs

Here are three hosts, all answering to vmk0 = 172.30.0.40:

ssh root@192.168.144.30  ->  [root@esx01-a:~]  vmk0  172.30.0.40
ssh root@192.168.144.32  ->  [root@esx01-b:~]  vmk0  172.30.0.40
ssh root@192.168.144.34  ->  [root@esx01-c:~]  vmk0  172.30.0.40

Same IP. Same VLAN. Same gateway. Same MAC address, as it turns out. And none of them can reach any of the others. This is the post where NSX VPCs stop being a workaround for nested labs and become genuinely better than the physical alternative.

Why identical addressing matters

If you’ve ever built training pods, cert-study labs, or per-team reproduction environments, you know the pain: every copy needs a unique address plan, so every runbook, every screenshot, every “type this exact command” has to be parameterised per pod. Students in seat 7 see different numbers from the slides. Reproductions drift from the original.

The fix is obvious and normally impossible: give every pod the same addresses. On a physical fabric that means VRFs, per-pod NAT, and a network team that stops answering your emails. In an NSX VPC it’s the default behaviour.

The mechanism: overlapping privateIPs

Each pod gets its own VPC, and every VPC declares the same private range:

apiVersion: vpc.nsx.vmware.com/v1alpha1
kind: VPC
metadata: {name: nested-vpc-a}      # then -b, then -c
spec:
  privateIPs: ["172.30.0.0/16"]      # identical in all three

A Private subnet is never advertised beyond its VPC, so NSX has no objection to three VPCs carving up the same /16. The pods aren’t “firewalled from each other” — there is simply no route between them. Isolation by construction, not by policy.

NSX’s subnet view filtered to sn-mgmt: four rows, four VPCs, one CIDR.

The trick: deterministic realization

Identical ranges aren’t enough — I want identical subnets, so the management gateway is .33 and the hosts are .40/.41 in every pod. NSX allocates subnets from privateIPs in creation order, and a fresh VPC allocates deterministically. So the topology is applied in a fixed order — trunk, mgmt, vMotion, vSAN — and every pod realizes the same map:

In the catalog blueprint that order is enforced with dependsOn between the subnet resources — the one place a declarative tool needs to be told about sequence. Skip it and two pods can come out with mgmt and vMotion swapped, which works perfectly and confuses everyone.

The door: one VIP per host

Each pod is unreachable from outside by design, so each host gets a VirtualMachineService of type LoadBalancer publishing SSH and HTTPS. The VIPs come from the org’s external block, and they’re the only addresses that differ between pods:

Which is how the opening transcript works: three VIPs, three hosts, one inside address.

Proving the isolation

Claims are cheap. The test matrix, from a VM in a fourth VPC (the org default):

ping 172.30.0.140 (own VPC)....... REACHABLE
ping 172.30.0.40  (pod space)..... unreachable
curl http://172.31.0.2/ (shared).. shared-svc repo01

Its own VPC’s 172.30.0.140: reachable. 172.30.0.40 — an address that exists in three other VPCs simultaneously: unreachable, because from here there is no such route. (The third line is the shared-services VPC, which is the next post.)

Inside each pod, east-west is normal: esx01 → esx02 vmkping passes on all three VLANs, in all three pods. And the detail I didn’t expect: the nested-ESXi appliance derives vmk0’s MAC deterministically from its config, so the three hosts share a MAC as well as an IP. Harmless — each VPC is its own L2 domain — but a nice demonstration of how complete the separation is.

What this replaces

Why this matters outside the lab

“Identical environments” sounds like a lab nicety. It’s actually one of the most requested things in enterprise IT, usually asked for in other words:

• Training at scale — every seat in the room sees the same addresses as the slides, so material is written once and never parameterised per pod.
• Per-engineer or per-team replicas of a reference environment, for development and testing that behaves exactly like the original.
• Regulatory or business-unit separation on shared infrastructure without VRF sprawl or a bespoke firewall estate — isolation is a property of the topology, which is the easiest kind to evidence to an auditor.
• Blue/green copies of an environment for change rehearsal, then cut-over or discard.

On a physical network each of these is a project. On VCF with NSX VPCs it’s a template.

Rules learned

• Overlapping privateIPs across VPCs is supported and intentional. Identical pods are a feature, not a hack.
• Fresh VPCs realize subnets deterministically in creation order — fix the order (dependsOn in a blueprint) and every pod gets the same map.
• Pods are unreachable from outside by construction; publish exactly what you mean to via LoadBalancer VIPs from the external block.
• Prove isolation from a different VPC, with a positive control (own VPC reachable) beside the negative.
• Expect duplicate MACs across pods from appliance images. It’s fine.

Previously: the LB that must exist first. Next: one WSUS for pods that can’t see each other.

Lab environment; opinions my own. Output captured live, trimmed for length, never edited for outcome.