<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Telegraf on The Nested Lab</title>
    <link>https://thenestedlab.com/tags/telegraf/</link>
    <description>Recent content in Telegraf on The Nested Lab</description>
    <generator>Hugo</generator>
    <language>en-gb</language>
    <lastBuildDate>Wed, 16 Sep 2026 12:20:00 +0100</lastBuildDate>
    <atom:link href="https://thenestedlab.com/tags/telegraf/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Telegraf on Windows Server 2025: unsupported, works anyway</title>
      <link>https://thenestedlab.com/posts/telegraf-windows-2025/</link>
      <pubDate>Wed, 16 Sep 2026 12:20:00 +0100</pubDate>
      <guid>https://thenestedlab.com/posts/telegraf-windows-2025/</guid>
      <description>The VCF Operations agent support matrix doesn&amp;rsquo;t list Windows Server 2025. The Telegraf agent installs, runs and reports anyway. What &amp;lsquo;unsupported&amp;rsquo; really means, how to deploy it deliberately, what to watch because of it — and the same Telegraf on VKS, where a hidden proxy dependency bit me.</description>
      <content:encoded><![CDATA[<p>Two facts, both true:</p>
<ol>
<li>The VCF Operations application-monitoring agent (Telegraf, packaged by
Broadcom) does not list Windows Server 2025 as a supported OS.</li>
<li>It installs, runs, and reports on Windows Server 2025.</li>
</ol>
<p>This post is about the gap between those, because &ldquo;unsupported&rdquo; is a
statement about <em>who fixes it when it breaks</em>, not about whether it works
— and there is a right way to run unsupported software in production,
which starts with knowing exactly what you&rsquo;re relying on.</p>
<h2 id="what-unsupported-means-here">What &ldquo;unsupported&rdquo; means here</h2>
<p>The matrix is a promise: Broadcom has tested this combination and will
take a support case on it. Server 2025 wasn&rsquo;t in the test set at release.
Nothing in the agent is OS-version-gated; it&rsquo;s Telegraf with Windows
inputs (<code>win_perf_counters</code>, <code>win_services</code>, <code>win_eventlog</code>) and an output
to Ops. The Windows APIs those inputs use haven&rsquo;t changed in a decade.</p>
<p>So: it works. You just own it.</p>
<h2 id="deploying-it-deliberately">Deploying it deliberately</h2>
<p>On the <a href="/series/the-windows-build-pipeline/">pipeline-built W2025 server</a>
the agent went on from VCF Operations itself — <em>Applications → Manage
Telegraf Agents → Install</em> — and registered as a <strong>Product Managed Agent</strong>,
version 9.1.0.0.3033. Then the two things that make it <em>yours</em>:</p>
<p><strong>1. Record exactly what you&rsquo;re running.</strong> Agent build, Telegraf version,
OS build — in whatever you use for a CMDB. When the support matrix catches
up you want to know whether you&rsquo;re on the version they tested.</p>
<p><strong>2. Add a canary.</strong> Something trivially OS-dependent that will go flat
first if a Windows update changes an API under the agent. Ops makes this a
two-minute job with no editing of <code>telegraf.conf</code>: <em>Custom Monitoring →
Custom Script → Add</em>, pointing at a script that already exists on the box.
Mine reads the build number from the registry and prints it as a
key/value pair:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-powershell" data-lang="powershell"><span class="line"><span class="cl"><span class="c"># C:\temp\canary.ps1</span>
</span></span><span class="line"><span class="cl"><span class="nv">$b</span> <span class="p">=</span> <span class="p">(</span><span class="nb">Get-ItemProperty</span> <span class="s2">&#34;HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion&#34;</span><span class="p">).</span><span class="py">CurrentBuildNumber</span>
</span></span><span class="line"><span class="cl"><span class="s2">&#34;osbuild=</span><span class="nv">$b</span><span class="s2">&#34;</span>        <span class="c"># -&gt; osbuild=26100</span>
</span></span></code></pre></div><p>Prefix <code>powershell -NoProfile -ExecutionPolicy Bypass -File</code>, five-minute
timeout. (Registry, not <code>Get-ComputerInfo</code> — that cmdlet takes 20–30
seconds on Server 2025 and would trip the plugin&rsquo;s own timeout, which is a
very unhelpful way for a canary to die.)</p>
<p><img alt="Manage Telegraf Agents: Test-2025 - Agent Running, Product Managed Agent, Install Success, with Ping Check, the w2025-canary custom script and a Services check under it" loading="lazy" src="/images/ui/o3-ops-manage-telegraf-agents-w2025.jpg">
<em>The agent list with the row expanded. One Windows Server 2025 VM, agent running, product-managed, both collection ticks green - and under it the agent is doing real work on an OS the matrix doesn&rsquo;t list: a ping check, a service check, and the canary with its full command line. (On the first pass the <code>Last Operation Status</code> read &ldquo;Start Failed&rdquo; next to a green &ldquo;Agent Running&rdquo; - the operation&rsquo;s status check didn&rsquo;t recognise the platform, the service came up anyway. Re-running the operation cleared it. That contradiction is the first thing you own on an unsupported OS.)</em></p>
<p><img alt="The Windows OS on Windows 2025 object: one object, Normal, no alerts, with Custom Script, Ping Check and Services children and live CPU/memory properties" loading="lazy" src="/images/ui/o9-ops-w2025-windows-os-summary.jpg">
<em>The object the agent created, as Ops sees it: green, no alerts, CPU and memory properties populated, and three child objects for the checks. This is the picture that matters - not the install dialog.</em></p>
<p><img alt="Ping Check metrics for the W2025 agent: Availability flat at 100 and Average Response Time in a steady band across a three-hour window" loading="lazy" src="/images/ui/o10-ops-w2025-ping-check-availability.jpg">
<em>And the proof the agent is doing more than existing: the Ping Check it runs from Windows Server 2025, availability flat at 100 across the morning, response time steady at a couple of milliseconds. Metrics arriving on schedule from an OS the matrix doesn&rsquo;t list.</em></p>
<p><img alt="The VM object in Ops: Microsoft Windows Server 2025 (64-bit), tools running" loading="lazy" src="/images/ui/o2-ops-w2025-vm-summary.jpg"></p>
<p><img alt="Windows OS on Windows 2025: AgentManagedType = Product Managed, Tags|source = Windows_2025" loading="lazy" src="/images/ui/o1-ops-w2025-agent-metrics.jpg">
<em>The &ldquo;Windows OS on Windows 2025&rdquo; child object the agent created, with <code>Telegraf Availability</code> in the metric tree and <code>AgentManagedType</code> reading Product Managed.</em></p>
<h2 id="what-to-watch-because-its-unsupported">What to watch, <em>because</em> it&rsquo;s unsupported</h2>
<ul>
<li><strong>Agent upgrades from Ops.</strong> The upgrade path is tested on supported OSes.
Take a snapshot before pushing an agent upgrade to the W2025 fleet;
upgrade one first.</li>
<li><strong>Windows cumulative updates.</strong> Performance counter names are stable;
provider GUIDs occasionally aren&rsquo;t. Watch the canary after Patch Tuesday.</li>
<li><strong>Service account and WinRM hardening.</strong> W2025 tightens defaults; if the
install bootstrap fails it&rsquo;s almost always WinRM/TLS, not the agent.</li>
<li><strong>Don&rsquo;t file cases on it.</strong> Reproduce on a supported OS first. That&rsquo;s
the deal you made.</li>
</ul>
<h2 id="the-same-telegraf-on-vks--and-its-hidden-dependency">The same Telegraf on VKS — and its hidden dependency</h2>
<p>On VKS the Telegraf package has a dependency that isn&rsquo;t in its README:
with <code>isMetricProxyConfigured: true</code> it mounts two secrets
(<code>metrics-proxy-tls-config</code>, <code>metrics-proxy-http-config</code>) that <strong>only the
Supervisor Management Proxy service propagates</strong> into guest clusters. Without
the proxy installed on the supervisor, every Telegraf pod sits in
<code>ContainerCreating</code> / <code>FailedMount</code> forever, and nothing says why.</p>
<p>Install the proxy supervisor service, and the chain is retroactive:
<code>SecretExport</code> in the guest&rsquo;s <code>kube-system</code> → <code>SecretImport</code> into
<code>tanzu-system-telegraf</code> → pods Running. (The <code>PackageInstall</code> needed an
annotation bump to clear a stale <code>ReconcileFailed</code> backoff.)</p>
<p>Then the second trap: Telegraf&rsquo;s output URL came out as
<code>https://supervisor-management-proxy.default.svc.:10093</code> — domainless and
unresolvable — because the cluster was created without
<code>clusterNetwork.serviceDomain</code>. Immutable. A CoreDNS <code>rewrite</code> rule
patched the live cluster; every new cluster gets <code>serviceDomain: cluster.local</code> in its spec.</p>
<h2 id="why-this-matters-outside-the-lab">Why this matters outside the lab</h2>
<p>The practical lesson for customers is about <strong>how</strong> to adopt something the
vendor hasn&rsquo;t blessed yet. New operating systems arrive before support
matrices catch up, and &ldquo;wait&rdquo; is often not an option. The approach here —
run it, record exactly what you&rsquo;re running, add a canary that detects
breakage early, upgrade one node first — is how an operations team gets
Windows Server 2025 monitored on day one without taking on hidden risk.
The same discipline applies to any unsupported-but-working combination.</p>
<h2 id="rules-learned">Rules learned</h2>
<ul>
<li>&ldquo;Unsupported&rdquo; = <em>you</em> own the fix path. Decide that consciously, record
versions, add a canary, upgrade one node first.</li>
<li>The Windows inputs aren&rsquo;t version-gated; W2025 runs the agent fine.
Alert on metric <strong>absence</strong>, not just thresholds.</li>
<li>On VKS, Telegraf <strong>hard-depends on the Supervisor Management Proxy</strong>
when the metric proxy flag is set; <code>FailedMount</code> on two secrets is the
tell.</li>
<li>Set <code>serviceDomain</code> at cluster create. Every add-on that builds a
service URL will thank you.</li>
</ul>
<p><em>Previously: <a href="/posts/fluent-bit-two-ways/">fluent-bit two ways</a>.</em></p>
<hr>
<p><em>Lab environment; opinions my own. Support status as observed at time of
writing — check the current matrix.</em></p>
]]></content:encoded>
    </item>
  </channel>
</rss>
