kubedoom       192.168.144.20:5900   (VNC — yes, it kills pods)
kubeinvaders   192.168.144.21
kube-ops-view  192.168.144.22
pacman         192.168.144.23        (+ MongoDB on a PVC — persistent high scores)
podinfo        192.168.144.24
goldpinger     192.168.144.25        (DaemonSet incl. control plane)
prometheus     (in-cluster: KSM + node-exporter, 11/11 targets up)

Captured from the VIPs the platform handed out. KubeDoom is VNC-only and podinfo is an API, so they sit this one out; the Ops topology tile shows all seven by name.

The path that matters: tenanted, not shortcut

The first version of this stack was deployed against the supervisor — an admin kubeconfig, a vSphere namespace, kubectl apply. It worked and it was wrong, for the reason the previous post spells out: nothing about it was provided to anyone. So it was torn down (seven apps, cluster, VPC and VIPs gone in about seven minutes) and rebuilt the proper way:

dev-01 org → default-project → SupervisorNamespace (class large, region f06, VPC default-f06)
          → VKS cluster vks-demo01 → seven apps

All created through VCF Automation — the CCI API for the namespace and cluster, then the apps via the cluster’s own kubeconfig. Two auth facts worth writing down, because they cost a cycle each:

• A provider service account reaches the cloud API only. CCI (/cci/kubernetes/apis/...) rejects provider tokens with 401 — it needs an org-scoped service account.
• Device-flow login uses the service account’s own UUID as client_id (not its software ID), against the tenant endpoint /oauth/tenant/<org>/device_authorization.

What the platform does for free

Each app is an ordinary Deployment + Service of type LoadBalancer. The supervisor turns each service into an NSX VPC LB virtual server and hands back a VIP from the org’s external block. No ingress controller, no MetalLB, no port-forwarding — seven services, seven VIPs, done. Pac-Man’s MongoDB asks for a PVC and gets a vSAN-backed volume through the CSI the supervisor already installed. Goldpinger runs as a DaemonSet across every node including the control plane and draws the node-to-node mesh live.

That’s three platform services (LB, storage, networking) exercised by apps that know nothing about VCF.

The traps

Supervisor refuses cluster-scoped RBAC — even to admin. Demo apps that need ClusterRoles (kube-ops-view, Goldpinger, KubeDoom) must live on a guest cluster. You cannot run them as vSphere Pods on the supervisor.

PodSecurity restricted is the VKS 1.35 default. Half the demo set runs as root. Symptom: Deployment shows 0 UP-TO-DATE, ReplicaSet exists, zero pods, events say FailedCreate. Fix: label the namespace pod-security.kubernetes.io/enforce=privileged — and mention in the demo that you did, because it’s a teaching moment.

node-exporter without hostNetwork. The VPC fabric blocks pod → node IP scrapes, so the stock DaemonSet’s hostNetwork: true doesn’t help; run it as a normal pod and let Prometheus scrape it in-cluster.

Docker Hub is flaky from behind a proxy. TLS handshake timeouts put pods into kubelet’s image-pull backoff. Deleting the stuck pods bypasses the backoff; a Harbor proxy-cache project fixes it properly.

Pod CIDR shadowing. The stock 192.168.0.0/16 pod range hid the org’s 192.168.144.0/21 external block from inside the cluster — apps couldn’t reach their neighbours’ VIPs. Pod CIDR is now 172.16.0.0/16.

Automation notes

The whole stack is a checkbox on the lab’s catalog item that deploys a supervisor: installDemoApps creates the cluster (newest compatible Kubernetes release, newest built-in ClusterClass, auto-detected), applies the seven apps, and reports their URLs in the deployment summary. Run integrated on a fresh environment: 16.7 minutes to CREATE_SUCCESSFUL. An immediate re-run: 3.3 minutes, all steps idempotent — which is the number I actually care about, because it means a broken demo is a re-run, not a rebuild.

Why this matters outside the lab

A demo stack sounds like a toy. It’s actually the fastest way to make a platform legible to people who don’t read YAML: a customer watches a request become a cluster, watches seven services get their own addresses, opens one and plays it. Everything underneath — self-service Kubernetes, load balancing, persistent storage, isolation — is being exercised in a way a non-technical stakeholder can see working. The same stack is what we put in front of a new team on day one, and the same idempotent deploy is what makes it safe to demonstrate live: if it breaks on stage, it re-runs in three minutes.

Rules learned

• Demo apps that need cluster-scoped RBAC must run on a guest cluster; the supervisor won’t grant it, even to admin.
• Build the stack through the tenanted path (org SA → CCI → namespace → cluster → apps). Same apps, but now they’re provided, quota’d and visible in Ops.
• VKS 1.35: restricted PodSecurity by default. Label the namespace and say why.
• LoadBalancer per app = NSX VIP per app. No ingress needed for a demo.
• Pick a pod CIDR that doesn’t overlap the VPC external block.
• Make the deploy idempotent; a demo that re-runs in 3 minutes is one you can afford to break on stage.

Previously: one VKS cluster, two ways. The Pac-Man instance here is the one from What’s a VPC?.

Lab environment; opinions my own.

All Apps in one paragraph

VCF Automation 9.1 has two provisioning models side by side. VM Apps is the classic Aria Automation path — cloud templates through an IaaS engine that drives vCenter. All Apps is the supervisor-native path: the blueprint composes Kubernetes objects (a Supervisor Namespace, VM Service VMs, NSX subnets, VKS clusters) and the vSphere Supervisor’s controllers reconcile them. A blueprint is formatVersion: 2; its resources are CCI.Supervisor.Namespace and CCI.Supervisor.Resource — the latter is literally “here’s a manifest, apply it in that namespace.”

That makes the blueprint a composition of the manifests from the earlier posts, with two additions: inputs, and dependsOn.

The blueprint, section by section

Inputs — the form

inputs:
  podName:  {type: string, default: nested-pod, pattern: '^[a-z0-9]([-a-z0-9]*[a-z0-9])?$'}
  vpcName:  {type: string, description: Must exist and be Realized before deploying.}
  esxOva:   {type: string, default: vmi-61bb062ddfc506b79}   # Nested ESXi 9.1 appliance
  isoImage: {type: string, default: vmi-39f562e2ae9e9c501}   # the ISO to attach
  vmClass:  {type: string, default: best-effort-large, enum: [best-effort-large, best-effort-xlarge, best-effort-2xlarge]}

The namespace — with libraries attached

namespace:
  type: CCI.Supervisor.Namespace
  properties:
    generateName: ${input.podName}-        # NOT name — new namespaces get a suffix
    className: large
    regionName: f06
    vpcName: ${input.vpcName}              # pins the namespace to the pod's VPC
    storageClasses: [{name: vSAN Default Storage Policy, limit: 400000Mi}]
    zones: [{name: domain-c9, cpuLimit: 40000M, memoryLimit: 64000Mi, ...}]
    contentSources:
      - {name: ISO, type: ContentLibrary}
      - {name: f06-vks-lib01, type: ContentLibrary}

contentSources is the line that closes the gap a lot of first attempts hit: a VCFA-created namespace has no content library, so there are no VirtualMachineImages and nothing can be deployed. Declaring the libraries here attaches them at creation.

The topology — ordered on purpose

snTrunk:   {type: CCI.Supervisor.Resource, properties: {context: ${resource.namespace.id}, manifest: <Subnet sn-trunk>}}
snMgmt:    {dependsOn: [snTrunk], ...  manifest: <Subnet sn-mgmt>}
snVmotion: {dependsOn: [snMgmt],  ...  manifest: <Subnet sn-vmotion>}
bmMgmt:    {... manifest: <SubnetConnectionBindingMap sn-mgmt -> sn-trunk, vlan 1610>}
bmVmotion: {... manifest: <SubnetConnectionBindingMap sn-vmotion -> sn-trunk, vlan 1611>}

The dependsOn chain is the whole reason every pod has identical CIDRs: fresh VPCs realize subnets in creation order, and the blueprint fixes that order.

The hosts — dual-NIC, ISO attached, bootstrapped by OVF

esx01:
  type: CCI.Supervisor.Resource
  dependsOn: [bmMgmt]                    # no point booting before VLAN 1610 exists
  properties:
    manifest:
      kind: VirtualMachine                # vmoperator.vmware.com/v1alpha5
      spec:
        hardware:
          cdrom: [ ... the ISO, declared, connected ... ]
        network:
          interfaces: [ eth0 -> sn-trunk, eth1 -> sn-trunk ]
        bootstrap:
          vAppConfig: [ guestinfo.hostname / ipaddress / vlan / ... ]
  wait:
    fields: [{path: status.powerState, value: PoweredOn}]

(Abridged — the full resource carries the image references, VM class, guest ID and the complete guestinfo set.)

Two vNICs, both on the trunk — the nested equivalent of a VCF host’s two pNICs. The ISO rides along as a declarative CD-ROM. And the wait block makes the deployment’s completion mean something: the request doesn’t finish until the host is powered on.

The doors — one VIP per host

A VirtualMachineService of type LoadBalancer per host, selecting it by label and publishing 22 and 443, and a blueprint output that reads the VIP back out of the service’s status:

outputs:
  esx01Ssh: {value: "ssh root@${resource.esx01Access.object.status.loadBalancer.ingress[0].ip}"}

The outputs surface in the deployment view — the requester gets the SSH command, not a scavenger hunt.

The request flow, end to end.

What the blueprint cannot express (yet)

Three cluster-scoped objects have no blueprint resource type, and they must exist before the request — in this order:

1. VPC — privateIPs: 172.30.0.0/16, same in every pod
2. VPCAttachment — connectivity profile with the service gateway; the LB creation fails loudly without it
3. LoadBalancer — silently, permanently required before the namespace

Today that’s a short script or a runbook step per pod. The honest framing: the blueprint is the pod; the VPC is the tenancy, and tenancy is still created one layer up. I’d expect that layer to become blueprintable; until then, keep the three calls next to the blueprint in version control.

Publishing: one version at a time

Blueprint → BlueprintVersion → release. Validation happens at version time, not create time, and the result lives in status.validationMessages rather than the HTTP code — a 200 with ContentValid: False is a thing. And only one version can be published: unrelease 1.0.0 before releasing 1.1.0, or you get a 409. (The full list of sharp edges is its own post.)

Why this matters outside the lab

This is where platform engineering turns into a service. The difference between “we can build you an environment” and “request one from the catalog” is the difference between days and minutes — and between a bespoke build and one that is consistent, quota-controlled and recorded every time. For an organisation that means:

• Time-to-environment measured in minutes, requested by the people who need it, without a queue.
• Consistency by construction — every environment comes from the same definition, so support, training material and runbooks all match.
• Governance built in — quotas, ownership, history and clean teardown are properties of the deployment record, not a spreadsheet.

The nested-ESXi pod is one catalog item. The same approach delivers any environment shape: application stacks for developers, sandboxes for a proof of concept, demo kits for a sales team, isolated builds for a partner.

Rules learned

• All Apps blueprints are compositions of manifests: CCI.Supervisor.Namespace plus CCI.Supervisor.Resource per object. If it works with kubectl, it works in a blueprint.
• generateName, not name, for the namespace; contentSources to attach libraries at creation; zones/storageClasses flat, not wrapped.
• dependsOn is how you get deterministic CIDRs — order the subnets.
• wait.fields turns “request complete” into “host is powered on”.
• VPC / VPCAttachment / LoadBalancer are prerequisites outside the blueprint, in that order, before every request.
• One published version per blueprint; validation in status, not the HTTP response.

Previously: shared services for isolated tenants. This closes the Pod Papers’ core arc — the companion posts on dual-NIC, no-DHCP bootstrap and blueprint gotchas fill in the details.

Lab environment; opinions my own. Blueprint nested-esxi-pod 1.1.0 is live in the lab catalog; YAML above trimmed for length.

1. ${input.x} is illegal inside a flow mapping

This looks like valid YAML and valid blueprint syntax:

- {key: guestinfo.hostname, value: {value: "esx01.${input.podName}.res.lab"}}

It fails content validation. The expression parser doesn’t reach into flow-style ({...}) mappings. Block style is fine:

- key: guestinfo.hostname
  value:
    value: esx01.${input.podName}.res.lab

Mixed style in the same list is fine too — only the entries that carry an expression need to be block-style. (This is why the blueprint’s vAppConfig list looks inconsistent; it’s deliberate.)

2. name vs generateName for a new namespace

A CCI.Supervisor.Namespace you’re creating must use generateName. metadata.name is rejected by the CCI API — the platform appends a random suffix, so pod-a- becomes pod-a-dgf5p. Everything downstream should reference ${resource.namespace.id}, never a literal name.

3. Zones and storage classes are flat

Early attempts wrapped them the way the raw CCI API does:

initialClassConfigOverrides:
  zones: [...]

In a blueprint they’re top-level properties of the namespace resource:

zones:
  - name: domain-c9
    cpuLimit: 40000M
    memoryLimit: 64000Mi
storageClasses:
  - name: vSAN Default Storage Policy
    limit: 400000Mi

And zones are required — omit them and the API says “Zone should be specified”, which at least is a clear message.

4. A new namespace has no content library

Deploy the namespace, deploy a VM, and get: no VirtualMachineImage found. A VCFA-created namespace attaches no content libraries by default. The fix is one block:

contentSources:
  - {name: ISO, type: ContentLibrary}
  - {name: f06-vks-lib01, type: ContentLibrary}

Without it you’re in the vSphere Client attaching libraries to a namespace by hand, which rather defeats the catalog.

5. Images sync after attach — wait for status.disks

Even with libraries attached at creation, the first VM create in a fresh namespace can be rejected:

no disks found in image ... status.disks

The image objects appear immediately; their disk metadata syncs over the next 1–3 minutes. The quota webhook checks status.disks and refuses until it’s populated. In a blueprint, put the hosts dependsOn something that takes a couple of minutes (the binding maps did the job here), or add an explicit wait. In a script, poll:

kubectl get virtualmachineimage -n <ns> <vmi> -o jsonpath='{.status.disks}'

6. Validation lives in status, not the HTTP code

Creating a BlueprintVersion returns 200 whether or not the content is valid. Read the object back:

status:
  contentValid: false
  validationMessages:
    - "... unexpected token ..."

If your pipeline checks the response code, it will happily publish a broken blueprint. Check status.contentValid and print the messages.

7. Only one published version — 409 on the second

Release 1.1.0 while 1.0.0 is released and you get a 409. It isn’t a transient conflict; it’s the rule. Unrelease the current version, then release the new one. Practically that means a publish step is unrelease old → release new, and there’s a short window where the catalog item has no released version. Do it when nobody’s requesting.

Bonus: the things that aren’t blueprint problems

Three prerequisites have no blueprint resource type and have to exist before the request — VPC, VPCAttachment, LoadBalancer, in that order. The blueprint’s vpcName input says “must exist and be Realized”, and it means it. Nothing in the blueprint fails if they’re missing; the deployment just never gets a VIP.

Why this matters outside the lab

VCF Automation’s All Apps model is new, and new platforms have edges. None of these seven are documented; all of them stall a first project by days if you meet them cold. The value of a delivery partner who has already built on the platform isn’t the YAML — it’s that a customer’s first blueprint publishes on day one instead of week two, and that the sharp edges are encoded into templates and provisioning scripts where users never meet them.

Rules learned

• Expressions need block-style YAML; flow mappings don’t get parsed.
• generateName, and reference the namespace by ${resource.x.id}.
• zones and storageClasses are flat and zones are required.
• contentSources on the namespace, or nothing can be deployed.
• Wait for image status.disks before the first VM (1–3 min).
• Check status.contentValid — the HTTP code lies by omission.
• One released version per blueprint: unrelease, then release.

Companion to a datacenter in a catalog tile.

Lab environment; opinions my own. Error text captured live.

I’ve now pushed the same list of requirements through both on a live VCF 9.1 lab. This is the comparison I wish I’d had at the start.

The two shapes

VM Apps — the classic Aria Automation model:

Org ─ Project ─ Cloud Zone(s)
                   │
        Cloud Account (vCenter / NSX)
                   │
   flavor · image · network · storage profiles
                   │
   Cloud Template (formatVersion 1) ──▶ IaaS engine ──▶ vCenter API
        Cloud.vSphere.Machine, Cloud.NSX.Network, customization spec

Provisioning is imperative through a broker. The IaaS engine holds the vCenter session; the tenant references abstractions (flavors, image mappings) that resolve at request time. State lives in the IaaS database.

All Apps — supervisor-native, via the Cloud Consumption Interface:

Org ─ CCI Project ─ Region
          │
   VPC (tenant-created; overlapping privateIPs allowed)
          │   + VPCAttachment   + LoadBalancer (before the namespace!)
   Supervisor Namespace (spec.vpcName pins it)
          │
   Kubernetes objects reconciled by the Supervisor:
     VirtualMachine / VirtualMachineService / Subnet / BindingMap / VKS Cluster
          │
   Blueprint (formatVersion 2, CCI.Supervisor.*) → BlueprintVersion → Catalog

Provisioning is declarative. The blueprint states desired objects; the supervisor’s controllers converge reality onto them and keep it there. State lives in etcd. Networking is NSX VPC-native.

Scorecard by use case

Every row below was actually built, not read about.

The two bold rows are the ones that decided it for me. Both are documented in this series; both are genuinely hard in VM Apps and simply the default in All Apps.

Where VM Apps still wins

Be fair to the incumbent:

• Years of content. vRO workflows, ABX actions, template libraries and a mature day-2 action framework carry over unchanged. If you have an estate of them, that’s real value you’d be throwing away.
• Deep vSphere reach. Anything vCenter can do to a VM — RDMs, per-device tuning, exotic customization — the IaaS engine can do, because it drives vCenter directly.
• Familiar network model. Segments, portgroups, on-demand routed/NAT networks from a network profile. No new mental model.
• Multi-cloud lineage. The same template idiom stretched to other endpoints.

The Windows Server 2025 pipeline elsewhere on this blog is a VM Apps build, and it’s a good one: Event Broker hooks for hostname allocation and placement metadata, cloudbase-init staging, a reboot-safe state machine in the guest. Nothing about it needs rewriting.

Where All Apps wins, and why it’s structural

• Declarative and self-healing. Desired state in etcd, controllers reconcile. There’s no second database to drift from vCenter.
• Structural multi-tenancy. A VPC per tenant is a hard NSX boundary, not an administrative one. Overlapping CIDRs are allowed, so cookie-cutter environments deploy side by side.
• VMs and Kubernetes are one model. The same blueprint composes a VKS cluster, VMs, secrets and networking. GitOps-able with ordinary tools.
• VPC networking is first-class. Trunk subnets, binding maps, PrivateTGW, per-VPC gateway firewall — none of it has a VM Apps equivalent.
• Modern bootstrap. cloud-init and sysprep as typed API fields.

Where All Apps hurts today (all observed live)

• Ordering matters and the errors are opaque. The LB must exist before the namespace; new namespaces reject VMs until images sync; blueprint validation has seven sharp edges.
• The tenancy layer isn’t blueprintable. VPC, VPCAttachment and LoadBalancer are cluster-scoped CCI objects with no blueprint resource type — scripted, not catalogued.
• Two endpoints. The CCI proxy serves tenancy objects; workload manifests go to the supervisor. You’ll hold two kubeconfigs.
• Day-2 is thinner. No event broker; extensibility means controllers and GitOps, which is fine if that’s your team and a gap if it isn’t.

Recommendation

Default to All Apps for new build-outs. Every use case on the list — including the two that are genuinely hard in VM Apps — is natural in the VPC model, and the whole estate is version-controlled YAML. Keep VM Apps as the compatibility surface for existing vRA content and for the rare thing that needs direct vCenter device manipulation. They coexist per org, so migration is incremental and nobody has to rewrite a working pipeline on a deadline.

And whichever you pick: codify the ordering rules into the scripts that provision tenancy, so the sharp edges stay encapsulated and the people requesting catalog items never meet them.

Why this matters outside the lab

Most customers arriving at VCF 9 have an estate of Aria Automation content and a question: rewrite, coexist, or migrate? The answer above is the one we take into design workshops. In practice it plays out as an assessment — which existing templates and workflows still earn their keep, which use cases are genuinely better served by the VPC model, and where the boundaries sit — followed by a coexistence plan that moves workloads across opportunistically instead of on a deadline. The two architectures sharing one organisation is what makes that low-risk.

Rules learned

• Two shapes, not two skins: imperative-through-a-broker vs declarative-reconciled. Pick per use case, not per org.
• All Apps is structurally better at isolation with identical addressing and nested/VLAN networks without fabric changes.
• VM Apps is still the home for existing vRO/ABX content and deep vCenter device work.
• All Apps’ pain is ordering: VPC → attachment → LB → namespace → subnets → image sync → workloads. Script it once.
• They coexist; migrate opportunistically.

Lab environment; opinions my own. Grounded in a VCF 9.1 / vSphere Supervisor with NSX VPC networking build-out; every row was deployed.

So: two clusters, one supervisor, one ClusterClass, two paths.

Path A: kubectl, the way we’ve always done it

kubectl vsphere login --server <supervisor> --tanzu-kubernetes-cluster-namespace demo
kubectl apply -f cluster.yaml

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
spec:
  clusterNetwork: { pods, services, serviceDomain }   # set all three — see below
  topology:
    class: builtin-generic-v3.6.0
    classNamespace: vmware-system-vks-public          # the gotcha
    version: v1.35.5+vmware.1
    controlPlane: {replicas: 1}
    workers: { one node pool, 2 replicas }
    variables: [ vmClass, storageClass ]

Fifteen minutes later: a cluster. Requires a vSphere namespace that somebody (an admin) created, with a content library attached, a VM class assigned and quota set — all in the vSphere Client, by hand.

Path B: the same manifest, as a catalog request

In All Apps the cluster is a CCI.Supervisor.Resource inside a blueprint, sitting next to a CCI.Supervisor.Namespace:

resources:
  namespace:
    type: CCI.Supervisor.Namespace
    properties:
      generateName: ${input.name}-
      className: large                      # quota comes from the class
      regionName: f06
      vpcName: ${input.vpc}
      contentSources: [{name: f06-vks-lib01, type: ContentLibrary}]
  cluster:
    type: CCI.Supervisor.Resource
    properties:
      context: ${resource.namespace.id}
      manifest: <the SAME Cluster object as above>

Publish it, and a tenant user requests it from a tile:

Fifteen minutes later: a cluster. Byte-identical Cluster object.

What path B adds for free

That last row is the one operations teams care about:

The VCFA-deployed cluster appears in Ops’ object model — supervisor → namespace → cluster → nodes → the workloads running on it — and its demo apps show up by name in the topology tab. The kubectl-built cluster is also visible to Ops (it’s the same supervisor, after all), but it has no deployment, no owner, no request history and no quota lineage. It’s a thing that exists, not a thing that was provided.

The four gotchas, in order of how much time they cost

Both paths share the same four traps on VKS 1.35 / VCF 9.1:

1. A VCFA-created namespace has no content library. Zero VirtualMachineImages → no cluster possible. contentSources in the blueprint, or attach by hand.
2. ClusterClass lives in vmware-system-vks-public. It 404s from the workload namespace unless the spec sets topology.classNamespace.
3. The default Quick Start namespace is 1000M CPU / 1000Mi. Unusable for a cluster. Use a real class: small 10000M/10000Mi, medium 20000M, large 40000M.
4. VKS 1.35 enforces PodSecurity restricted by default. Demo apps that run as root get a ReplicaSet and no pods; the events say FailedCreate. Label the app namespace pod-security.kubernetes.io/enforce=privileged (or fix the apps).

Plus one that only shows up later: set clusterNetwork.serviceDomain explicitly. It’s immutable after create, and a cluster without it produces a service DNS name that some add-ons build wrongly (....svc. with no domain). And pick a pod CIDR that doesn’t shadow your VPC’s external range — the stock 192.168.0.0/16 hid the org’s 192.168.144.0/21 from inside the cluster.

So which one?

Use kubectl when you’re the platform team proving something on a supervisor, or debugging. Use All Apps the moment a second person needs a cluster: the request form is the interface, the namespace class is the guardrail, the deployment is the audit trail, and Ops sees it as a provided service rather than a stray object.

The cluster’s the same either way. The service isn’t.

Why this matters outside the lab

The business case for the second path is governance without friction. Development teams get Kubernetes clusters on request; the platform team gets quotas, ownership, RBAC and a monitoring view of every cluster for free. That’s the difference between a managed Kubernetes service and a collection of clusters nobody can account for — and it’s typically the gap that stops organisations offering Kubernetes broadly at all. Everything the developers touch stays standard Kubernetes; the control lands around it, not on it.

Rules learned

• The Cluster object is identical across paths — All Apps wraps it, it doesn’t change it.
• What All Apps adds: catalog RBAC, class-based quota, library attach, VPC pinning, deployment history, and a place in the Ops object model.
• Four traps on 1.35: no default library, classNamespace, tiny default quota, PodSecurity restricted.
• Set serviceDomain and a non-shadowing pod CIDR at create; both are immutable.

Next in All Apps in Practice: the demo apps that live on this cluster.

Lab environment; opinions my own.