<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Architecture on The Nested Lab</title>
    <link>https://thenestedlab.com/tags/architecture/</link>
    <description>Recent content in Architecture on The Nested Lab</description>
    <generator>Hugo</generator>
    <language>en-gb</language>
    <lastBuildDate>Wed, 16 Sep 2026 07:00:00 +0100</lastBuildDate>
    <atom:link href="https://thenestedlab.com/tags/architecture/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>VM Apps vs All Apps: a field comparison of VCF Automation&#39;s two provisioning models</title>
      <link>https://thenestedlab.com/posts/vm-apps-vs-all-apps/</link>
      <pubDate>Wed, 16 Sep 2026 07:00:00 +0100</pubDate>
      <guid>https://thenestedlab.com/posts/vm-apps-vs-all-apps/</guid>
      <description>VCF Automation 9.1 ships two provisioning architectures in one org. I ran the same use cases through both — Linux and Windows VMs, ISO attach, multi-NIC, isolated pods, nested ESXi, shared services, a catalog item. The honest scorecard, and how each use case is actually achieved.</description>
      <content:encoded><![CDATA[<p>VCF Automation 9.1 has two ways to build things, side by side, in the same
organisation. If you&rsquo;ve come from Aria Automation you&rsquo;ll recognise one of
them immediately. The other looks like Kubernetes because it <em>is</em>
Kubernetes. Choosing between them isn&rsquo;t a matter of taste — they have
genuinely different shapes, and some use cases are natural in one and
awkward in the other.</p>
<p>I&rsquo;ve now pushed the same list of requirements through both on a live VCF
9.1 lab. This is the comparison I wish I&rsquo;d had at the start.</p>
<h2 id="the-two-shapes">The two shapes</h2>
<p><strong>VM Apps</strong> — the classic Aria Automation model:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">Org ─ Project ─ Cloud Zone(s)
</span></span><span class="line"><span class="cl">                   │
</span></span><span class="line"><span class="cl">        Cloud Account (vCenter / NSX)
</span></span><span class="line"><span class="cl">                   │
</span></span><span class="line"><span class="cl">   flavor · image · network · storage profiles
</span></span><span class="line"><span class="cl">                   │
</span></span><span class="line"><span class="cl">   Cloud Template (formatVersion 1) ──▶ IaaS engine ──▶ vCenter API
</span></span><span class="line"><span class="cl">        Cloud.vSphere.Machine, Cloud.NSX.Network, customization spec
</span></span></code></pre></div><p>Provisioning is <em>imperative through a broker</em>. The IaaS engine holds the
vCenter session; the tenant references abstractions (flavors, image
mappings) that resolve at request time. State lives in the IaaS database.</p>
<p><strong>All Apps</strong> — supervisor-native, via the Cloud Consumption Interface:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl">Org ─ CCI Project ─ Region
</span></span><span class="line"><span class="cl">          │
</span></span><span class="line"><span class="cl">   VPC (tenant-created; overlapping privateIPs allowed)
</span></span><span class="line"><span class="cl">          │   + VPCAttachment   + LoadBalancer (before the namespace!)
</span></span><span class="line"><span class="cl">   Supervisor Namespace (spec.vpcName pins it)
</span></span><span class="line"><span class="cl">          │
</span></span><span class="line"><span class="cl">   Kubernetes objects reconciled by the Supervisor:
</span></span><span class="line"><span class="cl">     VirtualMachine / VirtualMachineService / Subnet / BindingMap / VKS Cluster
</span></span><span class="line"><span class="cl">          │
</span></span><span class="line"><span class="cl">   Blueprint (formatVersion 2, CCI.Supervisor.*) → BlueprintVersion → Catalog
</span></span></code></pre></div><p>Provisioning is <em>declarative</em>. The blueprint states desired objects; the
supervisor&rsquo;s controllers converge reality onto them and keep it there.
State lives in etcd. Networking is NSX VPC-native.</p>
<h2 id="scorecard-by-use-case">Scorecard by use case</h2>
<p>Every row below was actually built, not read about.</p>
<table>
	<thead>
			<tr>
					<th>Use case</th>
					<th>VM Apps</th>
					<th>All Apps</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Linux VM + software</td>
					<td><code>Cloud.vSphere.Machine</code> + cloudConfig</td>
					<td>VM Service VM + cloud-init (typed field)</td>
			</tr>
			<tr>
					<td>Windows VM</td>
					<td>customization spec + software components</td>
					<td><code>bootstrap.sysprep</code>; ISO attachable declaratively</td>
			</tr>
			<tr>
					<td>ISO / CD-ROM attach</td>
					<td>day-2 edit in vCenter</td>
					<td><code>hardware.cdrom</code> in the manifest</td>
			</tr>
			<tr>
					<td>Multi-NIC</td>
					<td>NICs across network profiles</td>
					<td>multiple <code>interfaces</code> on VPC subnets; failover verified</td>
			</tr>
			<tr>
					<td>Self-service catalog</td>
					<td>template released to catalog</td>
					<td>blueprint → version → publish (one live version)</td>
			</tr>
			<tr>
					<td>Load-balanced access</td>
					<td>NSX LB via network profile</td>
					<td><code>VirtualMachineService</code> → VPC LB VIP</td>
			</tr>
			<tr>
					<td>Kubernetes clusters</td>
					<td>separate TKG integration</td>
					<td>native <code>Cluster</code> object in the namespace</td>
			</tr>
			<tr>
					<td><strong>Identical isolated environments</strong></td>
					<td>hard: on-demand NAT nets, unique addressing usually forced</td>
					<td><strong>natural</strong>: VPC per pod, overlapping CIDRs, zero route between</td>
			</tr>
			<tr>
					<td><strong>Nested ESXi / VLAN networks</strong></td>
					<td>trunk portgroups on the physical vDS — a fabric change</td>
					<td><strong>trunk subnet + binding maps</strong> — no fabric change</td>
			</tr>
			<tr>
					<td>Shared infra (WSUS, repos)</td>
					<td>shared segment routed everywhere</td>
					<td>one <code>PrivateTGW</code> subnet, one-way reachability</td>
			</tr>
			<tr>
					<td>Extensibility</td>
					<td>vRO, ABX, event broker (rich)</td>
					<td>controllers, GitOps, kubectl (thinner day-2 today)</td>
			</tr>
			<tr>
					<td>Deep per-device vSphere tuning</td>
					<td>anything vCenter can do</td>
					<td>what the VM Service API models</td>
			</tr>
	</tbody>
</table>
<p>The two bold rows are the ones that decided it for me. Both are
<a href="/posts/three-datacenters-one-ip-plan/">documented</a> <a href="/posts/nested-esxi-nsx-vpc/">in this
series</a>; both are genuinely hard in VM Apps
and simply the default in All Apps.</p>
<h2 id="where-vm-apps-still-wins">Where VM Apps still wins</h2>
<p>Be fair to the incumbent:</p>
<ul>
<li><strong>Years of content.</strong> vRO workflows, ABX actions, template libraries and
a mature day-2 action framework carry over unchanged. If you have an
estate of them, that&rsquo;s real value you&rsquo;d be throwing away.</li>
<li><strong>Deep vSphere reach.</strong> Anything vCenter can do to a VM — RDMs, per-device
tuning, exotic customization — the IaaS engine can do, because it drives
vCenter directly.</li>
<li><strong>Familiar network model.</strong> Segments, portgroups, on-demand routed/NAT
networks from a network profile. No new mental model.</li>
<li><strong>Multi-cloud lineage.</strong> The same template idiom stretched to other
endpoints.</li>
</ul>
<p>The <a href="/series/the-windows-build-pipeline/">Windows Server 2025 pipeline</a>
elsewhere on this blog is a VM Apps build, and it&rsquo;s a good one: Event Broker
hooks for hostname allocation and placement metadata, cloudbase-init
staging, a reboot-safe state machine in the guest. Nothing about it needs
rewriting.</p>
<h2 id="where-all-apps-wins-and-why-its-structural">Where All Apps wins, and why it&rsquo;s structural</h2>
<ul>
<li><strong>Declarative and self-healing.</strong> Desired state in etcd, controllers
reconcile. There&rsquo;s no second database to drift from vCenter.</li>
<li><strong>Structural multi-tenancy.</strong> A VPC per tenant is a hard NSX boundary,
not an administrative one. Overlapping CIDRs are <em>allowed</em>, so
cookie-cutter environments deploy side by side.</li>
<li><strong>VMs and Kubernetes are one model.</strong> The same blueprint composes a VKS
cluster, VMs, secrets and networking. GitOps-able with ordinary tools.</li>
<li><strong>VPC networking is first-class.</strong> Trunk subnets, binding maps,
<code>PrivateTGW</code>, per-VPC gateway firewall — none of it has a VM Apps
equivalent.</li>
<li><strong>Modern bootstrap.</strong> cloud-init and sysprep as typed API fields.</li>
</ul>
<h2 id="where-all-apps-hurts-today-all-observed-live">Where All Apps hurts today (all observed live)</h2>
<ul>
<li><strong>Ordering matters and the errors are opaque.</strong> <a href="/posts/the-lb-that-must-exist-first/">The LB must exist
before the namespace</a>; new
namespaces reject VMs until images sync; blueprint validation has
<a href="/posts/cci-blueprint-gotchas/">seven sharp edges</a>.</li>
<li><strong>The tenancy layer isn&rsquo;t blueprintable.</strong> VPC, VPCAttachment and
LoadBalancer are cluster-scoped CCI objects with no blueprint resource
type — scripted, not catalogued.</li>
<li><strong>Two endpoints.</strong> The CCI proxy serves tenancy objects; workload
manifests go to the supervisor. You&rsquo;ll hold two kubeconfigs.</li>
<li><strong>Day-2 is thinner.</strong> No event broker; extensibility means controllers
and GitOps, which is fine if that&rsquo;s your team and a gap if it isn&rsquo;t.</li>
</ul>
<h2 id="recommendation">Recommendation</h2>
<p>Default to <strong>All Apps</strong> for new build-outs. Every use case on the list —
including the two that are genuinely hard in VM Apps — is natural in the
VPC model, and the whole estate is version-controlled YAML. Keep <strong>VM
Apps</strong> as the compatibility surface for existing vRA content and for the
rare thing that needs direct vCenter device manipulation. They coexist per
org, so migration is incremental and nobody has to rewrite a working
pipeline on a deadline.</p>
<p>And whichever you pick: <strong>codify the ordering rules</strong> into the scripts
that provision tenancy, so the sharp edges stay encapsulated and the people
requesting catalog items never meet them.</p>
<h2 id="why-this-matters-outside-the-lab">Why this matters outside the lab</h2>
<p>Most customers arriving at VCF 9 have an estate of Aria Automation content
and a question: rewrite, coexist, or migrate? The answer above is the one
we take into design workshops. In practice it plays out as an assessment —
which existing templates and workflows still earn their keep, which use
cases are genuinely better served by the VPC model, and where the
boundaries sit — followed by a coexistence plan that moves workloads across
opportunistically instead of on a deadline. The two architectures sharing
one organisation is what makes that low-risk.</p>
<h2 id="rules-learned">Rules learned</h2>
<ul>
<li>Two shapes, not two skins: imperative-through-a-broker vs
declarative-reconciled. Pick per use case, not per org.</li>
<li>All Apps is structurally better at <strong>isolation with identical
addressing</strong> and <strong>nested/VLAN networks without fabric changes</strong>.</li>
<li>VM Apps is still the home for <strong>existing vRO/ABX content</strong> and
<strong>deep vCenter device work</strong>.</li>
<li>All Apps&rsquo; pain is <em>ordering</em>: VPC → attachment → LB → namespace →
subnets → image sync → workloads. Script it once.</li>
<li>They coexist; migrate opportunistically.</li>
</ul>
<hr>
<p><em>Lab environment; opinions my own. Grounded in a VCF 9.1 / vSphere
Supervisor with NSX VPC networking build-out; every row was deployed.</em></p>
]]></content:encoded>
    </item>
  </channel>
</rss>
