All Apps in one paragraph

VCF Automation 9.1 has two provisioning models side by side. VM Apps is the classic Aria Automation path — cloud templates through an IaaS engine that drives vCenter. All Apps is the supervisor-native path: the blueprint composes Kubernetes objects (a Supervisor Namespace, VM Service VMs, NSX subnets, VKS clusters) and the vSphere Supervisor’s controllers reconcile them. A blueprint is formatVersion: 2; its resources are CCI.Supervisor.Namespace and CCI.Supervisor.Resource — the latter is literally “here’s a manifest, apply it in that namespace.”

That makes the blueprint a composition of the manifests from the earlier posts, with two additions: inputs, and dependsOn.

The blueprint, section by section

Inputs — the form

inputs:
  podName:  {type: string, default: nested-pod, pattern: '^[a-z0-9]([-a-z0-9]*[a-z0-9])?$'}
  vpcName:  {type: string, description: Must exist and be Realized before deploying.}
  esxOva:   {type: string, default: vmi-61bb062ddfc506b79}   # Nested ESXi 9.1 appliance
  isoImage: {type: string, default: vmi-39f562e2ae9e9c501}   # the ISO to attach
  vmClass:  {type: string, default: best-effort-large, enum: [best-effort-large, best-effort-xlarge, best-effort-2xlarge]}

The namespace — with libraries attached

namespace:
  type: CCI.Supervisor.Namespace
  properties:
    generateName: ${input.podName}-        # NOT name — new namespaces get a suffix
    className: large
    regionName: f06
    vpcName: ${input.vpcName}              # pins the namespace to the pod's VPC
    storageClasses: [{name: vSAN Default Storage Policy, limit: 400000Mi}]
    zones: [{name: domain-c9, cpuLimit: 40000M, memoryLimit: 64000Mi, ...}]
    contentSources:
      - {name: ISO, type: ContentLibrary}
      - {name: f06-vks-lib01, type: ContentLibrary}

contentSources is the line that closes the gap a lot of first attempts hit: a VCFA-created namespace has no content library, so there are no VirtualMachineImages and nothing can be deployed. Declaring the libraries here attaches them at creation.

The topology — ordered on purpose

snTrunk:   {type: CCI.Supervisor.Resource, properties: {context: ${resource.namespace.id}, manifest: <Subnet sn-trunk>}}
snMgmt:    {dependsOn: [snTrunk], ...  manifest: <Subnet sn-mgmt>}
snVmotion: {dependsOn: [snMgmt],  ...  manifest: <Subnet sn-vmotion>}
bmMgmt:    {... manifest: <SubnetConnectionBindingMap sn-mgmt -> sn-trunk, vlan 1610>}
bmVmotion: {... manifest: <SubnetConnectionBindingMap sn-vmotion -> sn-trunk, vlan 1611>}

The dependsOn chain is the whole reason every pod has identical CIDRs: fresh VPCs realize subnets in creation order, and the blueprint fixes that order.

The hosts — dual-NIC, ISO attached, bootstrapped by OVF

esx01:
  type: CCI.Supervisor.Resource
  dependsOn: [bmMgmt]                    # no point booting before VLAN 1610 exists
  properties:
    manifest:
      kind: VirtualMachine                # vmoperator.vmware.com/v1alpha5
      spec:
        hardware:
          cdrom: [ ... the ISO, declared, connected ... ]
        network:
          interfaces: [ eth0 -> sn-trunk, eth1 -> sn-trunk ]
        bootstrap:
          vAppConfig: [ guestinfo.hostname / ipaddress / vlan / ... ]
  wait:
    fields: [{path: status.powerState, value: PoweredOn}]

(Abridged — the full resource carries the image references, VM class, guest ID and the complete guestinfo set.)

Two vNICs, both on the trunk — the nested equivalent of a VCF host’s two pNICs. The ISO rides along as a declarative CD-ROM. And the wait block makes the deployment’s completion mean something: the request doesn’t finish until the host is powered on.

The doors — one VIP per host

A VirtualMachineService of type LoadBalancer per host, selecting it by label and publishing 22 and 443, and a blueprint output that reads the VIP back out of the service’s status:

outputs:
  esx01Ssh: {value: "ssh root@${resource.esx01Access.object.status.loadBalancer.ingress[0].ip}"}

The outputs surface in the deployment view — the requester gets the SSH command, not a scavenger hunt.

The request flow, end to end.

What the blueprint cannot express (yet)

Three cluster-scoped objects have no blueprint resource type, and they must exist before the request — in this order:

1. VPC — privateIPs: 172.30.0.0/16, same in every pod
2. VPCAttachment — connectivity profile with the service gateway; the LB creation fails loudly without it
3. LoadBalancer — silently, permanently required before the namespace

Today that’s a short script or a runbook step per pod. The honest framing: the blueprint is the pod; the VPC is the tenancy, and tenancy is still created one layer up. I’d expect that layer to become blueprintable; until then, keep the three calls next to the blueprint in version control.

Publishing: one version at a time

Blueprint → BlueprintVersion → release. Validation happens at version time, not create time, and the result lives in status.validationMessages rather than the HTTP code — a 200 with ContentValid: False is a thing. And only one version can be published: unrelease 1.0.0 before releasing 1.1.0, or you get a 409. (The full list of sharp edges is its own post.)

Why this matters outside the lab

This is where platform engineering turns into a service. The difference between “we can build you an environment” and “request one from the catalog” is the difference between days and minutes — and between a bespoke build and one that is consistent, quota-controlled and recorded every time. For an organisation that means:

• Time-to-environment measured in minutes, requested by the people who need it, without a queue.
• Consistency by construction — every environment comes from the same definition, so support, training material and runbooks all match.
• Governance built in — quotas, ownership, history and clean teardown are properties of the deployment record, not a spreadsheet.

The nested-ESXi pod is one catalog item. The same approach delivers any environment shape: application stacks for developers, sandboxes for a proof of concept, demo kits for a sales team, isolated builds for a partner.

Rules learned

• All Apps blueprints are compositions of manifests: CCI.Supervisor.Namespace plus CCI.Supervisor.Resource per object. If it works with kubectl, it works in a blueprint.
• generateName, not name, for the namespace; contentSources to attach libraries at creation; zones/storageClasses flat, not wrapped.
• dependsOn is how you get deterministic CIDRs — order the subnets.
• wait.fields turns “request complete” into “host is powered on”.
• VPC / VPCAttachment / LoadBalancer are prerequisites outside the blueprint, in that order, before every request.
• One published version per blueprint; validation in status, not the HTTP response.

Previously: shared services for isolated tenants. This closes the Pod Papers’ core arc — the companion posts on dual-NIC, no-DHCP bootstrap and blueprint gotchas fill in the details.

Lab environment; opinions my own. Blueprint nested-esxi-pod 1.1.0 is live in the lab catalog; YAML above trimmed for length.

1. ${input.x} is illegal inside a flow mapping

This looks like valid YAML and valid blueprint syntax:

- {key: guestinfo.hostname, value: {value: "esx01.${input.podName}.res.lab"}}

It fails content validation. The expression parser doesn’t reach into flow-style ({...}) mappings. Block style is fine:

- key: guestinfo.hostname
  value:
    value: esx01.${input.podName}.res.lab

Mixed style in the same list is fine too — only the entries that carry an expression need to be block-style. (This is why the blueprint’s vAppConfig list looks inconsistent; it’s deliberate.)

2. name vs generateName for a new namespace

A CCI.Supervisor.Namespace you’re creating must use generateName. metadata.name is rejected by the CCI API — the platform appends a random suffix, so pod-a- becomes pod-a-dgf5p. Everything downstream should reference ${resource.namespace.id}, never a literal name.

3. Zones and storage classes are flat

Early attempts wrapped them the way the raw CCI API does:

initialClassConfigOverrides:
  zones: [...]

In a blueprint they’re top-level properties of the namespace resource:

zones:
  - name: domain-c9
    cpuLimit: 40000M
    memoryLimit: 64000Mi
storageClasses:
  - name: vSAN Default Storage Policy
    limit: 400000Mi

And zones are required — omit them and the API says “Zone should be specified”, which at least is a clear message.

4. A new namespace has no content library

Deploy the namespace, deploy a VM, and get: no VirtualMachineImage found. A VCFA-created namespace attaches no content libraries by default. The fix is one block:

contentSources:
  - {name: ISO, type: ContentLibrary}
  - {name: f06-vks-lib01, type: ContentLibrary}

Without it you’re in the vSphere Client attaching libraries to a namespace by hand, which rather defeats the catalog.

5. Images sync after attach — wait for status.disks

Even with libraries attached at creation, the first VM create in a fresh namespace can be rejected:

no disks found in image ... status.disks

The image objects appear immediately; their disk metadata syncs over the next 1–3 minutes. The quota webhook checks status.disks and refuses until it’s populated. In a blueprint, put the hosts dependsOn something that takes a couple of minutes (the binding maps did the job here), or add an explicit wait. In a script, poll:

kubectl get virtualmachineimage -n <ns> <vmi> -o jsonpath='{.status.disks}'

6. Validation lives in status, not the HTTP code

Creating a BlueprintVersion returns 200 whether or not the content is valid. Read the object back:

status:
  contentValid: false
  validationMessages:
    - "... unexpected token ..."

If your pipeline checks the response code, it will happily publish a broken blueprint. Check status.contentValid and print the messages.

7. Only one published version — 409 on the second

Release 1.1.0 while 1.0.0 is released and you get a 409. It isn’t a transient conflict; it’s the rule. Unrelease the current version, then release the new one. Practically that means a publish step is unrelease old → release new, and there’s a short window where the catalog item has no released version. Do it when nobody’s requesting.

Bonus: the things that aren’t blueprint problems

Three prerequisites have no blueprint resource type and have to exist before the request — VPC, VPCAttachment, LoadBalancer, in that order. The blueprint’s vpcName input says “must exist and be Realized”, and it means it. Nothing in the blueprint fails if they’re missing; the deployment just never gets a VIP.

Why this matters outside the lab

VCF Automation’s All Apps model is new, and new platforms have edges. None of these seven are documented; all of them stall a first project by days if you meet them cold. The value of a delivery partner who has already built on the platform isn’t the YAML — it’s that a customer’s first blueprint publishes on day one instead of week two, and that the sharp edges are encoded into templates and provisioning scripts where users never meet them.

Rules learned

• Expressions need block-style YAML; flow mappings don’t get parsed.
• generateName, and reference the namespace by ${resource.x.id}.
• zones and storageClasses are flat and zones are required.
• contentSources on the namespace, or nothing can be deployed.
• Wait for image status.disks before the first VM (1–3 min).
• Check status.contentValid — the HTTP code lies by omission.
• One released version per blueprint: unrelease, then release.

Companion to a datacenter in a catalog tile.

Lab environment; opinions my own. Error text captured live.