Introduction: Nested Virtualization Grows Up

In the early days of virtualization, running a "nested" environment — an entire enterprise infrastructure stack simulating production inside a single physical host — was a niche experiment. Today it's the standard path for VMware Certified Professional (VCP) candidates, infrastructure architects, and anyone who wants to explore VMware Cloud Foundation (VCF) without a $50,000 hardware bill.

VMware Cloud Foundation 9.1, shipped in mid-2026, builds on the VCF 9.0 architectural reset that followed Broadcom's acquisition. It ships alongside ESXi 9.1, keeps the unified VCF Installer appliance, and expands the VCF Fleet management model with clearer fleet-level versus instance-level responsibilities. If you last played with VCF 5.x, you're in for a surprise. Verify current VCF 9.1 component names and requirements against Broadcom TechDocs before deployment.

Building a fully functional, nested VCF 9.1 environment on a single host — verified against William Lam's official lab resources, Broadcom's TechDocs, and real community deployments — is entirely feasible with the right preparation.

Quick note on licensing: As of mid-2026, VMUG Advantage members with VCP-VCF certification can access VCF lab tokens directly from the VMUG Advantage portal. For VCF 9.1, verify the current entitlement and token workflow in the portal before starting your build. Annual VMUG Advantage membership runs ~$200/year.
You can build this as VVF instead of full VCF. The same nested ESXi/foundation-host prep works for a VMware vSphere Foundation (VVF) 9.x lab. In the VCF Installer, choose the VVF deployment workflow instead of the full VCF workflow. That gives you the vSphere/vSAN foundation without the full VCF private-cloud stack. Use VVF if your goal is vSphere, vSAN, lifecycle basics, or a smaller home lab. Use full VCF if you specifically need NSX-backed private cloud networking, workload domains, or the complete VCF operations model.

Part I: Know Before You Build

The New VCF 9.1 Architecture

VCF 9.x drops the old "Bring Up" + SDDC Manager model. In 9.1, Broadcom tightened management-plane consolidation: fleet-level services coordinate inventory, lifecycle, licensing, and depot content across VCF instances, while instance-level operations remain tied to each deployed environment. Broadcom renamed several components in 9.1, and this article uses the most current names available as of July 2026; verify current VCF 9.1 component names and requirements against Broadcom TechDocs before deployment. The new structure is:

This matters for your lab: if you're following older VCF 5.x or early 9.0 guides, the UI flows, JSON schemas, component names, and management responsibilities have changed significantly. Expect more tasks to land in Fleet/Fleet Services rather than a classic SDDC Manager-only workflow. Treat all VCF 9.1-specific naming in this article as deployment guidance, not a substitute for the current Broadcom TechDocs terminology for your exact build.

Hardware Requirements — The Real Numbers

Full VCF 9.1 Simple deployment has materially higher lab requirements than many older VCF 5.x guides imply. A full VCF 9.1 Simple deployment requires the approximate appliance sizing below. Verify current VCF 9.1 component names and requirements against Broadcom TechDocs before deployment.

ComponentvCPUMemoryStorage (provisioned)Actual Used
vCenter Server421 GB642 GB~30 GB
NSX Manager624 GB300 GB~14 GB
SDDC Manager / VCF Operations Fleet Mgr416 GB931 GB~77 GB
VCF Operations416 GB290 GB~13 GB
VCF Operations Fleet Manager412 GB206 GB~117 GB
VCF Operations Collector416 GB280 GB~11 GB
VCF Automation (VCFA)2496 GB626 GB~72 GB
TOTAL (with VCFA)~48~194 GB~3.2 TB~330 GB
TOTAL (without VCFA)~24~98 GB~2.6 TB~260 GB

(Baseline source: William Lam — Minimal Resources for Deploying VCF 9.0 in a Lab. Verify current VCF 9.1 appliance sizing before deployment.)

VCF Automation is optional for lab purposes. VCFA (formerly vRealize Automation) is not required to learn VCF core operations, NSX, vSAN, or lifecycle management. If your goal is cert prep, feature exploration, or vSAN/NSX testing, skip VCFA entirely — it adds 24 vCPU and 96 GB RAM to your resource footprint for functionality most labs never use. The rest of this article works with or without VCFA deployed. Only deploy it if you specifically need to test automation/orchestration workflows.

If you do deploy VCF Automation (VCFA): it requires a 24 vCPU VM. Your physical host must have at least 12 cores / 24 threads. Many popular home lab mini-PCs (GMKtec K11, NUC 12s with 8-core CPUs) will hit this wall.

If you can't meet the 24 vCPU minimum, Broadcom's Tomas Fojta has confirmed you can manually edit the VCFA VM down to 16 vCPU (or even 12) via vSphere Client — YMMV on performance. Memory cannot be reduced without breaking VCFA entirely.

Practical minimums for a nested lab:

Community hardware picks: The Minisforum MS-A2 (16C/32T AMD Ryzen 9) runs full VCF 9.x including VCFA. Two units needed for a 2-node vSAN cluster with comfortable headroom. The ASUS NUC 15 Pro is another solid option. See William Lam's community survey for 50+ tested configurations.

CPU Compatibility Warning

ESXi 9.1 drops support for several older Intel CPU generations that ESXi 8.x still handled:

If you have one of these, ESXi 9.1 will block installation by default. You can override with the kernel boot option allowLegacyCPU=true (SHIFT+O during boot) — for lab use only, not Broadcom-supported. Note: XSAVE CPU instruction support is also mandatory and cannot be bypassed.

Part II: Planning the Nested Architecture

Topology: Single-Node vs. Multi-Node

For most lab use cases — cert prep, feature exploration, PoC testing — a Single-Node deployment is the right call. VCF 9.1 keeps the VCF 9 simplifications vs. VCF 5.x:

VCF 9.1 still requires a minimum of 3 ESXi hosts for vSAN (OSA or ESA), or 2 ESXi hosts for external storage (FC/NFS). This is an improvement over VCF 5.x's 4-host vSAN minimum. For a single physical host lab, you'll use the single-host override (covered in the deployment steps).

Two Nested Lab Flavors

  1. 1. vSphere Foundation (VVF) Lab — ESXi + vCenter + basic networking and storage. No NSX required. Good for VCP-DV prep with lighter hardware demands (~24 vCPU, ~98 GB RAM without VCFA).
  2. 2. Full VCF 9.1 Lab — Complete stack: ESXi 9.1, NSX, VCF Operations, VCF Fleet Services, vSphere Kubernetes Service, with VCF Automation optional. Most core VCF workflows are accessible in single-node mode, but HA behavior, scale testing, and some production topology features are not represented. Best for VCF-related certifications.

Critical: What Does NOT Work in Nested VCF 9.1

Before diving into deployment, know these hard limits:

FeatureNested Compatible?Notes
vSAN OSA✅ YesThin provisioned against host datastore
vSAN ESA✅ Yes (with workaround)Requires vSAN ESA Hardware Mock VIB
NVMe Tiering❌ NoIncompatible with nested virtualization entirely
IDE Controller on nested ESXi VMs❌ NoRemoved in ESXi 9.1; use SATA or SCSI
NSX Edge on AMD Ryzen (physical host)⚠️ WorkaroundRequires PowerCLI remediation script

Network Planning

Network misconfiguration is the #1 reason nested labs fail. Plan three segments before touching an installer:

VMkernel / SegmentPurposeRecommended VLAN
ManagementvCenter / VCF Operations APIs, SSHVLAN 10
vSANStorage traffic between nested hostsVLAN 20
NSX TransportNSX control plane and data planeVLAN 30

On the foundation host, set port groups for NSX transport to Promiscuous Mode = Accept. Without this, nested nodes cannot communicate through NSX overlays. For nested ESXi host uplink port groups (management, vSAN, and NSX), enable Promiscuous Mode, MAC Address Changes, and Forged Transmits:


# Verify promiscuous mode via ESXCLI (use same port group name as creation)
esxcli network vswitch standard portgroup policy security get \
  --portgroup-name "NSX-Transport"

MTU requirements: The NSX Tunnel Endpoint (TEP) requires minimum 1600 MTU for Geneve overlay traffic — standard 1500 will fail the VCF Installer network validation. Best practice is 9000 (jumbo frames) end-to-end across all lab segments. If your physical switch doesn't support jumbo frames, see the TEP MTU bypass in the troubleshooting section.

Part III: Step-by-Step Deployment

Step 1 — Install ESXi 9.1 on the Foundation Host

Download ESXi 9.1 from the Broadcom Support Portal (VMUG Advantage members: use the VMUG portal instead). VMware's FAQ confirms free ESXi availability through 8.x; for 9.x, verify access and entitlement in the Broadcom/VMUG portal before planning a lab build. Boot from USB or ISO.

During install, dedicate one NIC for management and leave others available for nested VM bridging. After booting, set a static IP via ESXCLI:


# Configure management IP (adjust to your network)
esxcli network ip interface ipv4 set \
  -i vmk0 \
  -I 192.168.1.10 \
  -N 255.255.255.0 \
  -t static

esxcli network ip route ipv4 add \
  --network default \
  --gateway 192.168.1.1


# Verify connectivity
vmkping -I vmk0 192.168.1.1

Step 2 — Critical Foundation Host Tuning

Before deploying any VCF components, tune the foundation host for nested workloads. This step is consistently missing from beginner guides.

Enable Huge Pages — nested ESXi instances rely heavily on huge pages for memory management. Without this, you'll hit "Insufficient Memory" errors before VCF even finishes initializing:


# Enable Huge Pages (requires host reboot to take effect)
esxcli system settings advanced set \
  -o /VirtualMem/HugePages/Enabled \
  -i 1

# Verify setting
esxcli system settings advanced list -o /VirtualMem/HugePages/Enabled

Set Promiscuous Mode on NSX port groups:


# Via PowerCLI (run from management workstation)
Get-VirtualPortGroup -Name "NSX-Transport" | `
  Set-SecurityPolicy -AllowPromiscuous $true -ForgedTransmits $true -MacChanges $true

Configure Virtual Switch for Nested Traffic:

LAB HOST ONLY. Do not run on a production ESXi host.


# ESXCLI: Create a vSwitch for nested lab traffic (9000 MTU for NSX TEP)
esxcli network vswitch standard add -v vSwitch-Lab
esxcli network vswitch standard set -v vSwitch-Lab -m 9000

# Add port groups with VLAN IDs (use same names everywhere)
esxcli network vswitch standard portgroup add -p "MGMT-Nested" -v vSwitch-Lab
esxcli network vswitch standard portgroup set -p "MGMT-Nested" -v 10

esxcli network vswitch standard portgroup add -p "vSAN-Nested" -v vSwitch-Lab
esxcli network vswitch standard portgroup set -p "vSAN-Nested" -v 20

esxcli network vswitch standard portgroup add -p "NSX-Transport" -v vSwitch-Lab
esxcli network vswitch standard portgroup set -p "NSX-Transport" -v 30
VLAN model: These VLAN IDs are used by the outer (foundation) host's physical switch. If your physical switch handles VLAN tagging at the trunk port, match these to your actual switch config. If you're not using VLANs (simple lab switch), omit the -v flag — but VLAN separation is recommended for production-like NSX behavior.

Step 3 — Deploy the VCF 9.1 Installer Appliance

In VCF 9.x, the old Cloud Builder is gone. It's replaced by the VCF Installer (OVA filename changes by release and build; verify the current VCF 9.1 OVA/build number in the Broadcom download portal before scripting deployment). This appliance handles the initial bringup and can optionally transform itself into the VCF Operations Fleet Manager post-deployment.

Deploy via OVFTool or the vSphere Client OVA deploy wizard. Inspect the OVA properties first — property keys are case-sensitive and vary by appliance:


# List OVF properties before deploying (verify exact keys)
ovftool VCF-Installer-9.1.x.x-.ova | grep -i "Property"

# Deploy via OVFTool (use property keys from output above)
ovftool \
  --name="VCF-Installer" \
  --diskMode=thin \
  --datastore="datastore1" \
  --network="MGMT-Nested" \
  --prop:guestinfo.hostname="vcf-installer.lab.local" \
  --prop:guestinfo.ip0="192.168.10.10" \
  --prop:guestinfo.netmask0="255.255.255.0" \
  --prop:guestinfo.gateway="192.168.10.1" \
  --prop:guestinfo.DNS="192.168.10.53" \
  VCF-Installer-9.1.x.x-.ova \
  "vi://root@192.168.1.10/"
OVA filename note: The installer OVA may be named "VCF-Installer" or "VCF-SDDC-Manager-Appliance" depending on the Broadcom download portal build. They are the same appliance — the VCF Installer transforms into the SDDC Manager / VCF Operations Fleet Manager post-deployment. Check the download portal for the current filename.

Once powered on, access the VCF Installer UI at https://<installer-ip>. During the deployment workflow, choose VCF for the full Cloud Foundation stack or VVF for the smaller vSphere Foundation path. The host prep, nested ESXi networking, DNS/NTP, and datastore work in this guide still apply either way; NSX and workload-domain sections are only relevant to full VCF.

Step 4 — Single-Host Override (Lab Only)

LAB HOST ONLY. Do not run on any production VCF Installer or management appliance. By default, VCF 9.1 requires a minimum of 3 ESXi hosts for vSAN or 2 for external storage. For a single physical machine lab, apply this unsupported-but-community-validated override. Use only ONE flag based on your build — check William Lam's resources for your specific VCF 9.x release:


# SSH into the VCF Installer appliance as 'vcf' user
# BACKUP FIRST:
cp /home/vcf/feature.properties /home/vcf/feature.properties.bak.$(date +%F-%H%M)

# For VCF 9.0.1+ (including 9.1) — use ONLY this flag:
echo "feature.vcf.vgl-29121.single.host.domain=true" >> /home/vcf/feature.properties

# For VCF 9.0.0 (original GA only) — use this flag INSTEAD:
# echo "feature.vcf.internal.single.host.domain=true" >> /home/vcf/feature.properties

# Restart VCF Installer services
echo 'y' | /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh
Disclaimer: This override is not officially supported by Broadcom. Use exclusively for lab/learning purposes. Do not apply to production environments. The wrong flag for your build will either be silently ignored or cause validation errors — verify before relying on it.

After restarting services, use the JSON deployment method instead of the interactive wizard — the UI still enforces the 3-host minimum at the front-end level. Upload your JSON with a single host reference and proceed directly to validation.

Step 5 — Deploy the VCF Fleet

With the installer configured, proceed through the bringup:

  1. 1. Specify ESXi hosts — enter your nested ESXi host(s) with their credentials
  2. 2. Choose storage: vSAN (OSA or ESA) or external. For first-time nested bringup, vSAN OSA is simpler — fewer validation gotchas. Use vSAN ESA only if you specifically want to learn ESA storage policies, and apply the vSAN ESA Hardware Mock VIB if your disks aren't on the HCL.
  3. 3. Network topology — assign your pre-created port groups to the management, vSAN, and NSX transport roles
  4. 4. Licenses — enter your VMUG Advantage VCF 9.1 token; VCF Operations License Server handling changed in 9.1, so confirm the Fleet/Operations license server workflow during bringup
  5. 5. Depot — VCF 9.1 adds HTTP Offline Depot support, useful for isolated labs without direct Broadcom portal access
  6. 6. Deploy — the installer provisions vCenter, NSX Manager, VCF Operations/Fleet management components, and VCF Fleet Services in sequence. Budget 90–120 minutes.

# Monitor deployment progress (SSH to installer appliance)
tail -f /var/log/vmware/vcf/vcf-bringup/vcf-bringup.log | grep -E "(INFO|WARN|ERROR)"

Step 6 — Verify the VCF Fleet

Once bringup completes, log in to the VCF management UI exposed by your deployed management appliance (the VCF Operations Fleet Manager, which consolidates lifecycle and fleet management functions):


https://<vcf-operations-fleet-manager-ip>/ui

You should see:

Step 7 — Add vSphere Kubernetes Service (Optional)

VCF 9.1 ships with vSphere Kubernetes Service (VKS) — the renamed successor to Tanzu Kubernetes Grid (TKG). Enabling it requires the Supervisor to be activated on your vSphere cluster.

From vCenter, navigate to Workload Management > Enable:

  1. 1. Select your VCF cluster
  2. 2. Choose the NSX networking option (required for full VKS functionality)
  3. 3. Configure storage policies for Persistent Volumes
  4. 4. Deploy a Supervisor — this creates the management Kubernetes control plane embedded in ESXi

Once the Supervisor is healthy, you can deploy Kubernetes workload clusters via kubectl or the vSphere Client:


# Login to Supervisor with kubectl (vSphere Plugin required)
kubectl vsphere login \
  --server=https://<supervisor-ip> \
  --vsphere-username=administrator@vsphere.local \
  --insecure-skip-tls-verify

# Create a vSphere Namespace
kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: lab-workloads
EOF

# Deploy a Kubernetes cluster in that namespace (VKS)
kubectl apply -f vks-cluster.yaml
Resource caveat: Enabling VKS adds significant memory overhead. Only attempt this if your foundation host has 256 GB+ RAM available.

Part IV: Common Issues & Verified Fixes

Issue 1: "Insufficient Memory" During Nested Deployment

Symptom: VCF bringup fails when spinning up nested ESXi VMs; VCFA fails to power on.

Root causes:

Fix:


# Confirm Huge Pages enabled and active
esxcli system settings advanced list -o /VirtualMem/HugePages/Enabled
# Should return "Int Value = 1"

# Check memory pressure
esxcli system memory list
# Look for "Consumed" vs "Total" — >90% = trouble

If memory is genuinely tight, reduce VCFA vCPU to 16 via vSphere Client (Settings → Edit Settings → CPU). Memory reduction is not viable — the 96 GB is a hard requirement.

Issue 2: Nested ESXi Boot Fails — "Failed to Locate Kickstart on CD-ROM"

Symptom: Nested ESXi 9.1 VMs fail to boot with a kickstart error referencing the CD-ROM.

Cause: ESXi 9.1 dropped support for the IDE Controller. If your nested ESXi VM templates still have an IDE CD-ROM device, they will fail to locate the installation media.

Fix: Remove the IDE controller from your nested ESXi VM config. Use SATA or SCSI for any virtual disks, and attach the ESXi ISO via a SATA CD-ROM device:


# Check VM's controller configuration via PowerCLI
Get-VM "Nested-ESXi-01" | Get-HardDisk | Select Name, StorageFormat, DiskType
# Remove any IDE-attached devices and re-attach as SATA/SCSI

Issue 3: Network Connectivity Lost Between Nested Nodes

Symptom: Nested hosts can't ping each other; vMotion fails; NSX shows transport nodes as Disconnected.

Fix checklist:


# 1. Confirm Promiscuous Mode = Accept on NSX port groups (name must match creation)
esxcli network vswitch standard portgroup policy security get \
  --portgroup-name "NSX-Transport"
# macChanges, forgedTransmits, allowPromiscuous should all = true

# 2. Check MTU consistency — must be 9000 (or minimum 1600) end-to-end
esxcli network vswitch standard list
# All vSwitches should show same MTU (9000 recommended)

# 3. Check VLAN tagging
esxcli network vswitch standard portgroup list
# Ensure correct VLAN IDs match your physical switch config

Issue 3b: VCF Installer Fails MTU Validation (TEP 1600 Check)

Symptom: VCF Installer blocks deployment with MTU validation error on the NSX Tunnel Endpoint.

Cause: VCF requires minimum 1600 MTU for NSX TEP traffic. If your physical switch only supports 1500, the validation fails.

Fix (lab only): Bypass the MTU validation check on the VCF Installer (source: William Lam):


# SSH to VCF Installer as 'vcf' user
sudo echo "validation.disable.network.connectivity.check=true" >> /etc/vmware/vcf/domainmanager/application.properties
sudo echo "nsxt.mtu.validation.skip=true" >> /etc/vmware/vcf/domainmanager/application.properties

# Restart services
echo 'y' | /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh
Warning: This bypasses a real validation. NSX overlay traffic will be fragmented at 1500 MTU, which can cause performance issues and packet drops under load. Use only for lab testing where jumbo frames are not available. Fix the underlying MTU if at all possible.

Issue 4: vSAN ESA Shows "No Certified Disks Found"

Symptom: VCF Installer fails vSAN ESA validation with disk HCL errors.

Fix: Deploy William Lam's vSAN ESA Hardware Mock VIB — compatible with both VCF 5.x and 9.0, works on physical and nested ESXi hosts:


# Install the VIB on each ESXi host (SSH in first)
esxcli software vib install -v \
  /tmp/VMware_bootbank_vsan-esa-mock_x.x.x.x-xxxxxxxx.vib

# Verify installation
esxcli software vib list | grep vsan-esa-mock

(Download from William Lam's blog: vSAN ESA Disk HCL Workaround for VCF 9.1)

Issue 5: NSX Transport Node Failures on AMD Ryzen Hosts

Symptom: NSX Edge services fail to deploy on a physical AMD Ryzen host.

Fix: Apply Broadcom's officially documented PowerCLI remediation script for AMD Ryzen NSX Edge compatibility:


# PowerCLI remediation for AMD Ryzen NSX Edge
# Full script: https://williamlam.com/2025/06/powercli-remediation-script-for-running-nsx-edge-on-amd-ryzen-for-vcf-9-0.html

Connect-VIServer -Server "vcenter.lab.local" -User "admin" -Password "VMware1!"
# [Apply per the blog post's script — sets required CPU feature masks on Edge VMs]

Conclusion: What You've Built

If you've followed this guide to completion, you're running:

This is a genuine, full-fidelity VCF 9.1 environment for:

Next Steps

1. Automate with PowerCLI:


# Script your VCF bringup JSON for reproducible lab rebuilds
$vcfBringupSpec = @{
  sddcManagerSpec = @{
    hostname = "vcf-mgr.lab.local"
    ipAddress = "192.168.10.10"
  }
  # ... full spec
} | ConvertTo-Json -Depth 10

2. Practice Lifecycle Management: Use VCF lifecycle workflows to practice an 8.x → VVF 9.1 → VCF 9.1 upgrade path and future 9.1 patching. Understanding the update cadence is itself a certification exam topic.

3. Explore NVMe Tiering on Physical: If you have a second physical host, NVMe Tiering (incompatible with nested) becomes available and dramatically changes vSAN performance characteristics. Worth a standalone lab post.

Resources

Last updated: July 2026 for VCF 9.1 context. Always check Broadcom's release notes and download portal for the latest build numbers, OVA filenames, license-server behavior, and compatibility information.