The Windows half of this series was a story about an agent that works on an OS the vendor hasn’t listed. This one is the opposite: a package on a fully supported platform that does nothing at all, silently, because of a dependency its documentation doesn’t mention.
The symptom
Install the Telegraf package on a VKS cluster with the metric proxy flag
set — isMetricProxyConfigured: true, which is what you want if the
metrics are going to VCF Operations — and every Telegraf pod stays in
ContainerCreating. Describe one and the reason is FailedMount: two
secrets don’t exist.
MountVolume.SetUp failed for volume "metrics-proxy-tls-config" : secret not found
MountVolume.SetUp failed for volume "metrics-proxy-http-config" : secret not found
Nothing creates them. The package doesn’t. The cluster doesn’t. The
PackageInstall reports ReconcileFailed and backs off, and that’s where
it stays.
The dependency
Those two secrets are published by the Supervisor Management Proxy,
a Supervisor Service you install on the supervisor, not in the guest
cluster. It exports the proxy’s TLS and HTTP config into every guest
cluster’s kube-system as a SecretExport; a matching SecretImport
in tanzu-system-telegraf pulls them across. No proxy on the supervisor,
no export, no secrets, no pods.
Supervisor Management Proxy (Supervisor Service)
└─ SecretExport (guest kube-system)
└─ SecretImport (guest tanzu-system-telegraf)
└─ metrics-proxy-tls-config + metrics-proxy-http-config
└─ Telegraf pods mount them → Running
The fix is one install on the supervisor, and it’s retroactive: once the
proxy exists the exports appear, the imports follow, and pods that had
been failing to mount for a day come up on their own. The only nudge
needed was an annotation bump on the PackageInstall to clear the stale
ReconcileFailed backoff.
The second trap: a URL with no domain
With pods running, Telegraf still wasn’t delivering. Its output URL had come out as:
https://supervisor-management-proxy.default.svc.:10093
Note the trailing dot and nothing after svc. The package builds that URL
from the cluster’s service domain, and this cluster had been created
without clusterNetwork.serviceDomain — so the domain was empty. That
field is immutable after create.
Two fixes, both applied:
- Live cluster: a CoreDNS
rewriterule that maps the domainless name onto the real one. Ugly, effective, documented in the cluster’s notes. - Every new cluster:
serviceDomain: cluster.localin the spec. Every add-on that constructs a service URL is assuming it’s there.
Why this matters outside the lab
Both traps have the same shape: a platform component that fails with a
generic symptom (FailedMount, a name that won’t resolve) whose cause is
a decision made somewhere else — a service not installed on the
supervisor, a field left blank at cluster create. For a customer that
means two things. Metrics from Kubernetes workloads into VCF Operations
are a platform feature, so the supervisor has to be built for it, not
just the cluster. And cluster specs need a baseline that includes the
fields add-ons assume, because the immutable ones can’t be fixed later.
This is exactly the kind of thing a standard cluster class and a
supervisor build checklist exist to encode.
Rules learned
- On VKS, the Telegraf package hard-depends on the Supervisor
Management Proxy whenever the metric proxy flag is set.
FailedMounton twometrics-proxy-*secrets is the tell. - The dependency heals retroactively: install the proxy, bump the PackageInstall annotation, wait.
- Set
serviceDomainat cluster create. It’s immutable, and every add-on that builds a service URL will assume it. - When a package “does nothing”, describe the pod, not the package.
Previously: Telegraf on Windows Server 2025. More in the Observability on VCF series.
Lab environment; opinions my own.