<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Debugging on CodeBytes - The Curious Engineer</title><link>https://chris-ayers.com/tags/debugging/</link><description>Recent content in Debugging on CodeBytes - The Curious Engineer</description><generator>Hugo -- gohugo.io</generator><language>en-US</language><copyright>© 2026 Chris Ayers</copyright><lastBuildDate>Fri, 09 Oct 2026 09:00:00 -0400</lastBuildDate><atom:link href="https://chris-ayers.com/tags/debugging/index.xml" rel="self" type="application/rss+xml"/><item><title>Stop Losing the Bug When You Restart Aspire</title><link>https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/</link><pubDate>Fri, 09 Oct 2026 09:00:00 -0400</pubDate><guid>https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/</guid><description>&lt;p&gt;You reproduce an intermittent failure, change one line, and restart the application. This time the request works. Now you need the old trace to work out why.&lt;/p&gt;&#10;&lt;p&gt;If it disappeared with the restart, you&amp;rsquo;re left comparing the new run with whatever you remember. Aspire 13.6&amp;rsquo;s dashboard keeps completed runs, so you can go back to the failed request instead.&lt;sup id="fnref:1"&gt;&lt;a href="#fn:1" class="footnote-ref" role="doc-noteref"&gt;1&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;</description><content:encoded>&lt;p&gt;You reproduce an intermittent failure, change one line, and restart the application. This time the request works. Now you need the old trace to work out why.&lt;/p&gt;&#10;&lt;p&gt;If it disappeared with the restart, you&amp;rsquo;re left comparing the new run with whatever you remember. Aspire 13.6&amp;rsquo;s dashboard keeps completed runs, so you can go back to the failed request instead.&lt;sup id="fnref:1"&gt;&lt;a href="#fn:1" class="footnote-ref" role="doc-noteref"&gt;1&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#10;&lt;p&gt;&lt;a href="https://chris-ayers.com/series/aspire-field-notes/" &gt;Aspire Field Notes&lt;/a&gt; starts with that problem: keeping a failure around long enough to investigate it after the app has moved on.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Healthy resources, failed request&#10; &#10; &#10; &#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;A failed business request doesn&amp;rsquo;t have to show up in &lt;code&gt;/health&lt;/code&gt;. When a downstream dependency rejects a call, the request fails while the service keeps running and its health endpoint stays green.&lt;/p&gt;&#10;&lt;figure class="fn-motion fn-figure fn-request not-prose" data-fn-motion="request"&gt;&#10; &lt;div class="fn-board"&gt;&#10; &lt;div class="fn-board-head"&gt;&#10; &lt;span&gt;GET /api/catalog&lt;/span&gt;&#10; &lt;span class="fn-legend fn-caps"&gt;&lt;span class="fn-legend-dot"&gt;&lt;/span&gt;Healthy&lt;/span&gt;&#10; &lt;/div&gt;&#10; &lt;svg class="fn-request-map" viewbox="0 0 440 236" role="img" aria-label="The catalog request travels from web to api. Its catalogdb query succeeds, inventory returns HTTP 503, and the 503 travels back through api to web. All four resources stay healthy."&gt;&#10; &lt;path class="fn-link fn-link-web" d="M116 76H168" fill="none" stroke="#b91c1c" stroke-width="4" /&gt;&#10; &lt;path class="fn-link fn-link-inventory" d="M272 76H324" fill="none" stroke="#b91c1c" stroke-width="4" /&gt;&#10; &lt;path class="fn-link fn-link-db" d="M220 104V168" fill="none" stroke="#15803d" stroke-width="4" /&gt;&#10; &lt;circle class="fn-request-dot" r="8" opacity="0" /&gt;&#10; &lt;g class="fn-node"&gt;&#10; &lt;rect class="fn-node-box" x="12" y="48" width="104" height="56" rx="12" fill="#fff" stroke="#cbd5e1" /&gt;&#10; &lt;text class="fn-node-label" x="64" y="82" text-anchor="middle" font-family="monospace" font-size="17" fill="#1e293b"&gt;web&lt;/text&gt;&#10; &lt;circle class="fn-health" cx="104" cy="60" r="4.5" fill="#15803d" /&gt;&#10; &lt;circle class="fn-health-ring" cx="104" cy="60" r="4.5" fill="none" opacity="0" /&gt;&#10; &lt;/g&gt;&#10; &lt;g class="fn-node"&gt;&#10; &lt;rect class="fn-node-box" x="168" y="48" width="104" height="56" rx="12" fill="#fff" stroke="#cbd5e1" /&gt;&#10; &lt;text class="fn-node-label" x="220" y="82" text-anchor="middle" font-family="monospace" font-size="17" fill="#1e293b"&gt;api&lt;/text&gt;&#10; &lt;circle class="fn-health" cx="260" cy="60" r="4.5" fill="#15803d" /&gt;&#10; &lt;circle class="fn-health-ring" cx="260" cy="60" r="4.5" fill="none" opacity="0" /&gt;&#10; &lt;/g&gt;&#10; &lt;g class="fn-node"&gt;&#10; &lt;rect class="fn-node-box" x="324" y="48" width="104" height="56" rx="12" fill="#fff" stroke="#cbd5e1" /&gt;&#10; &lt;text class="fn-node-label" x="376" y="82" text-anchor="middle" font-family="monospace" font-size="17" fill="#1e293b"&gt;inventory&lt;/text&gt;&#10; &lt;circle class="fn-health" cx="416" cy="60" r="4.5" fill="#15803d" /&gt;&#10; &lt;circle class="fn-health-ring" cx="416" cy="60" r="4.5" fill="none" opacity="0" /&gt;&#10; &lt;/g&gt;&#10; &lt;g class="fn-node"&gt;&#10; &lt;rect class="fn-node-box" x="168" y="168" width="104" height="56" rx="12" fill="#fff" stroke="#cbd5e1" /&gt;&#10; &lt;text class="fn-node-label" x="220" y="202" text-anchor="middle" font-family="monospace" font-size="17" fill="#1e293b"&gt;catalogdb&lt;/text&gt;&#10; &lt;circle class="fn-health" cx="260" cy="180" r="4.5" fill="#15803d" /&gt;&#10; &lt;circle class="fn-health-ring" cx="260" cy="180" r="4.5" fill="none" opacity="0" /&gt;&#10; &lt;/g&gt;&#10; &lt;g class="fn-badge fn-badge-db"&gt;&#10; &lt;rect class="fn-pill-ok" x="284" y="183" width="92" height="26" rx="13" fill="#dcfce7" stroke="#15803d" /&gt;&#10; &lt;text class="fn-pill-text fn-pill-text-ok" x="330" y="201" text-anchor="middle" font-family="monospace" font-size="14" fill="#15803d"&gt;query OK&lt;/text&gt;&#10; &lt;/g&gt;&#10; &lt;g class="fn-badge fn-badge-inventory"&gt;&#10; &lt;rect class="fn-pill-bad" x="350" y="14" width="52" height="26" rx="13" fill="#fee2e2" stroke="#b91c1c" /&gt;&#10; &lt;text class="fn-pill-text fn-pill-text-bad" x="376" y="32" text-anchor="middle" font-family="monospace" font-size="14" fill="#b91c1c"&gt;503&lt;/text&gt;&#10; &lt;/g&gt;&#10; &lt;g class="fn-badge fn-badge-api"&gt;&#10; &lt;rect class="fn-pill-bad" x="194" y="14" width="52" height="26" rx="13" fill="#fee2e2" stroke="#b91c1c" /&gt;&#10; &lt;text class="fn-pill-text fn-pill-text-bad" x="220" y="32" text-anchor="middle" font-family="monospace" font-size="14" fill="#b91c1c"&gt;503&lt;/text&gt;&#10; &lt;/g&gt;&#10; &lt;g class="fn-badge fn-badge-web"&gt;&#10; &lt;rect class="fn-pill-bad" x="38" y="14" width="52" height="26" rx="13" fill="#fee2e2" stroke="#b91c1c" /&gt;&#10; &lt;text class="fn-pill-text fn-pill-text-bad" x="64" y="32" text-anchor="middle" font-family="monospace" font-size="14" fill="#b91c1c"&gt;503&lt;/text&gt;&#10; &lt;/g&gt;&#10; &lt;/svg&gt;&#10; &lt;/div&gt;&#10; &lt;div class="fn-controls" hidden&gt;&#10; &lt;button class="fn-button fn-button-primary" type="button" data-fn-action="toggle"&gt;Play request&lt;/button&gt;&#10; &lt;button class="fn-button" type="button" data-fn-action="replay" disabled&gt;Replay&lt;/button&gt;&#10; &lt;span class="fn-status" data-fn-status role="status" aria-live="polite"&gt;Ready · plays once&lt;/span&gt;&#10; &lt;/div&gt;&#10; &lt;figcaption&gt;All four resources stay healthy while &lt;code&gt;GET /api/catalog&lt;/code&gt; fails with a 503 from inventory.&lt;/figcaption&gt;&#10;&lt;/figure&gt;&#10;&#10;&lt;p&gt;Green indicators tell you the processes are running and passing their health checks. To diagnose the failed request, you need its downstream calls, durations, and errors.&lt;/p&gt;&#10;&lt;p&gt;So before changing code, make sure the application emits that telemetry. The dashboard can receive OpenTelemetry, but it can&amp;rsquo;t invent spans the app never produced. For .NET, Service Defaults is a good start. Node, Java, Python, and Rust services need their own instrumentation.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Choose the right kind of persistence&#10; &#10; &#10; &#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;Aspire 13.6 uses SQLite for dashboard storage. Its three modes have different purposes:&lt;sup id="fnref:2"&gt;&lt;a href="#fn:2" class="footnote-ref" role="doc-noteref"&gt;2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#10;&lt;table&gt;&#10;&#9;&lt;thead&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Mode&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Storage behavior&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;When I would use it&lt;/th&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/thead&gt;&#10;&#9;&lt;tbody&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;None&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;A temporary database for one dashboard process&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;A disposable standalone diagnostic session&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;Run&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;A separate database for each dashboard run, with a run selector&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Comparing a failing run with a later attempt&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;&lt;code&gt;Resume&lt;/code&gt;&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;One database reused across restarts, without separate run history&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Continuing a standalone dashboard session&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/tbody&gt;&#10;&lt;/table&gt;&#10;&lt;p&gt;When the AppHost launches the dashboard, it uses &lt;code&gt;Run&lt;/code&gt; by default. You don&amp;rsquo;t need to add a database resource to get this behavior.&lt;/p&gt;&#10;&lt;p&gt;A standalone dashboard still defaults to &lt;code&gt;None&lt;/code&gt;. To pick up where it left off after a restart, start it with &lt;code&gt;aspire dashboard run --persistence Resume&lt;/code&gt; and a stable &lt;code&gt;--application-name&lt;/code&gt;. Keep the name, data directory, and mode the same each time. &lt;code&gt;Resume&lt;/code&gt; keeps adding to one database, so it won&amp;rsquo;t give you separate runs to compare. Only one dashboard process can write to it at a time.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Capture a baseline, a failure, and a recovery&#10; &#10; &#10; &#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;The &lt;a href="https://github.com/codebytes/blog-samples/tree/main/aspire-field-notes" target="_blank" rel="noreferrer"&gt;companion catalog app&lt;/a&gt; is small. A Vite frontend named &lt;code&gt;web&lt;/code&gt; calls a catalog API named &lt;code&gt;api&lt;/code&gt;. The API reads PostgreSQL&amp;rsquo;s &lt;code&gt;catalogdb&lt;/code&gt;, then makes one instrumented HTTP call to a separate &lt;code&gt;inventory&lt;/code&gt; service, with no retry to hide a failure.&lt;/p&gt;&#10;&lt;p&gt;A development-only switch, &lt;code&gt;Inventory__FaultEnabled&lt;/code&gt;, makes inventory return a 503 while its health endpoint stays green. Flipping it gives you a failure and a recovery to compare without breaking anything shared. Here you already know the cause. With a real bug, the trace has to support whatever fix you propose.&lt;/p&gt;&#10;&lt;p&gt;If you&amp;rsquo;re following along, &lt;a href="https://github.com/codebytes/blog-samples/tree/main/aspire-field-notes/walkthroughs/01-keep-the-failing-run" target="_blank" rel="noreferrer"&gt;walkthrough 01&lt;/a&gt; has the commands and links to the one-time setup, including the database secret. The sample targets Aspire 13.6.1 and requires Aspire CLI 13.6 or later.&lt;/p&gt;&#10;&#10;&lt;h3 class="relative group"&gt;A request you can repeat&#10; &#10; &#10; &#10; &#10;&lt;/h3&gt;&#10;&lt;p&gt;Use the same request for each run. The companion&amp;rsquo;s AppHost adds a &lt;strong&gt;Load catalog&lt;/strong&gt; command to &lt;code&gt;web&lt;/code&gt; that sends one &lt;code&gt;GET /api/catalog&lt;/code&gt; through the frontend, the same path the browser takes. It reports the HTTP status, trace ID, and response. You can use the highlighted button in the dashboard or run &lt;code&gt;aspire resource web load-catalog&lt;/code&gt; from a terminal. Here&amp;rsquo;s the abridged registration:&lt;/p&gt;&#10;&lt;div class="highlight-wrapper"&gt;&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-csharp" data-lang="csharp"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="n"&gt;web&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WithHttpCommand&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;&amp;#34;/api/catalog&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;displayName&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;&amp;#34;Load catalog&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;endpointName&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;&amp;#34;http&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;commandName&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;&amp;#34;load-catalog&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;commandOptions&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="n"&gt;HttpCommandOptions&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="p"&gt;{&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;Method&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;HttpMethod&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Get&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="n"&gt;IsHighlighted&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="c1"&gt;// PrepareRequest sets a ten-second timeout.&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="c1"&gt;// GetCommandResult returns the status, trace ID, and response as JSON.&lt;/span&gt;&#10;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="p"&gt;});&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;&#10;&lt;p&gt;The &lt;a href="https://github.com/codebytes/blog-samples/blob/main/aspire-field-notes/catalog/Catalog.AppHost/AppHost.cs" target="_blank" rel="noreferrer"&gt;full registration&lt;/a&gt; also reports a non-success status as a failed command, so a broken request never looks like a passing one.&lt;/p&gt;&#10;&lt;p&gt;With the fault off, &lt;strong&gt;Load catalog&lt;/strong&gt; returns a 200 and three products. That&amp;rsquo;s your baseline.&lt;/p&gt;&#10;&#10;&lt;h3 class="relative group"&gt;Reproduce the failure&#10; &#10; &#10; &#10; &#10;&lt;/h3&gt;&#10;&lt;p&gt;Restart the same app with the fault on and run &lt;strong&gt;Load catalog&lt;/strong&gt; again. The frontend, API, inventory service, and database remain healthy, but the command fails with &lt;code&gt;HTTP 503: Inventory unavailable&lt;/code&gt; and a trace ID:&lt;/p&gt;&#10;&#10;&lt;figure class="full-width"&gt;&#10; &lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="auto"&#10; alt="Aspire dashboard Resources page with the Load catalog action highlighted on the web resource and a failure notification reading HTTP 503: Inventory unavailable, with the request&amp;#39;s trace ID"&#10; width="2560"&#10; height="1600"&#10; src="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/load-catalog-503_hu_83e1de259f5a28b.png"&#10; srcset="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/load-catalog-503_hu_83e1de259f5a28b.png 800w,/posts/aspire-field-notes-keep-the-failing-run/load-catalog-503_hu_be535d84ff3955e0.png 1280w"&#10; sizes="(min-width: 768px) 50vw, 65vw"&#10; data-zoom-src="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/load-catalog-503.png"&#10; /&gt;&#10; &#10; &lt;figcaption&gt;The resources are healthy, but Load catalog returns HTTP 503.&lt;/figcaption&gt;&#10; &lt;/figure&gt;&#10;&lt;p&gt;Open that trace. It has four spans: the API&amp;rsquo;s request, its PostgreSQL query, one HTTP call to inventory, and inventory&amp;rsquo;s own span. All three HTTP spans show 503, and there&amp;rsquo;s only one client span, so nothing retried the call:&lt;/p&gt;&#10;&#10;&lt;figure class="full-width"&gt;&#10; &lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="auto"&#10; alt="Trace detail for GET /api/catalog showing the API request, its PostgreSQL query to catalogdb, one HTTP GET call that returned 503, and the inventory service&amp;#39;s GET /inventory span"&#10; width="2560"&#10; height="560"&#10; src="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/failed-trace_hu_5e28a0ef172df9bd.png"&#10; srcset="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/failed-trace_hu_5e28a0ef172df9bd.png 800w,/posts/aspire-field-notes-keep-the-failing-run/failed-trace_hu_27c84a320de3a359.png 1280w"&#10; sizes="(min-width: 768px) 50vw, 65vw"&#10; data-zoom-src="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/failed-trace.png"&#10; /&gt;&#10; &#10; &lt;figcaption&gt;The database query completes. The inventory call returns 503, and the API passes that failure back.&lt;/figcaption&gt;&#10; &lt;/figure&gt;&#10;&lt;p&gt;Before stopping the failing run, open the dashboard&amp;rsquo;s &lt;strong&gt;Console logs&lt;/strong&gt; page for both &lt;code&gt;api&lt;/code&gt; and &lt;code&gt;inventory&lt;/code&gt;. &lt;strong&gt;The dashboard stores a console stream in a run&amp;rsquo;s history only after you view or export it there.&lt;/strong&gt; Reading the same output in a terminal with &lt;code&gt;aspire logs&lt;/code&gt; doesn&amp;rsquo;t count. Structured logs sent through OpenTelemetry are stored either way.&lt;sup id="fnref1:2"&gt;&lt;a href="#fn:2" class="footnote-ref" role="doc-noteref"&gt;2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#10;&lt;p&gt;That&amp;rsquo;s easy to miss when a startup message is the clue you need.&lt;/p&gt;&#10;&lt;p&gt;CLI queries such as &lt;code&gt;aspire otel traces --has-error&lt;/code&gt; read the live run. Picking a historical run in the browser doesn&amp;rsquo;t change what the CLI sees, so do the comparison below in the dashboard.&lt;/p&gt;&#10;&#10;&lt;h3 class="relative group"&gt;Keep the failure and compare recovery&#10; &#10; &#10; &#10; &#10;&lt;/h3&gt;&#10;&lt;p&gt;Open the run selector in the dashboard header. Hover or focus the &lt;strong&gt;Live run&lt;/strong&gt; row to reveal &lt;strong&gt;Pin run&lt;/strong&gt;, then pin the failing run. Stop the app, turn the fault off, start it again, and run &lt;strong&gt;Load catalog&lt;/strong&gt;. The request returns a 200 and three products through the same call path.&lt;/p&gt;&#10;&lt;figure class="fn-motion fn-figure fn-runs not-prose" data-fn-motion="runs"&gt;&#10; &lt;div class="fn-runs-layout"&gt;&#10; &lt;div&gt;&#10; &lt;div class="fn-board"&gt;&#10; &lt;div class="fn-board-head"&gt;&#10; &lt;span class="fn-caps"&gt;Run selector&lt;/span&gt;&#10; &lt;span data-fn-step-label&gt;Step 5 of 5&lt;/span&gt;&#10; &lt;/div&gt;&#10; &lt;div class="fn-run-stage"&gt;&#10; &lt;span class="fn-lane fn-lane-live"&gt;Live run&lt;/span&gt;&#10; &lt;span class="fn-lane fn-lane-earlier"&gt;Earlier runs&lt;/span&gt;&#10; &lt;div class="fn-run fn-run-1" data-fn-from="1"&gt;&#10; &lt;span class="fn-run-name"&gt;Run 1&lt;/span&gt;&#10; &lt;span class="fn-run-meta"&gt;fault off&lt;/span&gt;&#10; &lt;span class="fn-chip fn-chip-ok"&gt;200&lt;/span&gt;&#10; &lt;/div&gt;&#10; &lt;div class="fn-run fn-run-2" data-fn-from="2"&gt;&#10; &lt;span class="fn-run-name"&gt;Run 2&lt;/span&gt;&#10; &lt;span class="fn-run-meta"&gt;fault on&lt;/span&gt;&#10; &lt;span class="fn-chip fn-chip-pin"&gt;Pinned&lt;/span&gt;&#10; &lt;span class="fn-chip fn-chip-bad"&gt;503&lt;/span&gt;&#10; &lt;/div&gt;&#10; &lt;div class="fn-run fn-run-3" data-fn-from="4"&gt;&#10; &lt;span class="fn-run-name"&gt;Run 3&lt;/span&gt;&#10; &lt;span class="fn-run-meta"&gt;fault off&lt;/span&gt;&#10; &lt;span class="fn-chip fn-chip-ok"&gt;200&lt;/span&gt;&#10; &lt;/div&gt;&#10; &lt;/div&gt;&#10; &lt;/div&gt;&#10; &lt;div class="fn-controls" hidden&gt;&#10; &lt;button class="fn-button fn-button-primary" type="button" data-fn-action="play"&gt;Play steps&lt;/button&gt;&#10; &lt;button class="fn-button" type="button" data-fn-action="back"&gt;Back&lt;/button&gt;&#10; &lt;button class="fn-button" type="button" data-fn-action="next"&gt;Next&lt;/button&gt;&#10; &lt;span class="fn-sr" data-fn-status role="status" aria-live="polite"&gt;&lt;/span&gt;&#10; &lt;/div&gt;&#10; &lt;/div&gt;&#10; &lt;ol class="fn-steps" role="list"&gt;&#10; &lt;li&gt;Run 1 is the baseline: &lt;strong&gt;Load catalog&lt;/strong&gt; returns 200.&lt;/li&gt;&#10; &lt;li&gt;Restart with the fault on. Run 2 returns 503, and Run 1 moves to earlier runs.&lt;/li&gt;&#10; &lt;li&gt;Pin Run 2 and open its &lt;code&gt;api&lt;/code&gt; and &lt;code&gt;inventory&lt;/code&gt; console logs.&lt;/li&gt;&#10; &lt;li&gt;Restart with the fault off. Run 3 returns 200, and the pinned run stays.&lt;/li&gt;&#10; &lt;li&gt;Select Run 2 to compare its trace and logs with the recovery.&lt;/li&gt;&#10; &lt;/ol&gt;&#10; &lt;/div&gt;&#10; &lt;figcaption&gt;A restart ends the live run without discarding it. Pinning keeps the failure from aging out as newer runs arrive.&lt;/figcaption&gt;&#10;&lt;/figure&gt;&#10;&#10;&lt;p&gt;In the new dashboard, the run selector lists the pinned failing run beside the live one:&lt;/p&gt;&#10;&#10;&lt;figure class="full-width"&gt;&#10; &lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="auto"&#10; alt="Run selector open in the recovered dashboard, listing the live run and the pinned 10:08:18 PM failing run"&#10; width="1040"&#10; height="520"&#10; src="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/pinned-run-selector_hu_8c3d618cbbaa30d1.png"&#10; srcset="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/pinned-run-selector_hu_8c3d618cbbaa30d1.png 800w,/posts/aspire-field-notes-keep-the-failing-run/pinned-run-selector_hu_fd8d309f42fd3f48.png 1280w"&#10; sizes="(min-width: 768px) 50vw, 65vw"&#10; data-zoom-src="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/pinned-run-selector.png"&#10; /&gt;&#10; &#10; &lt;figcaption&gt;The pinned failure is still available beside the new live run.&lt;/figcaption&gt;&#10; &lt;/figure&gt;&#10;&lt;p&gt;Select the pinned run. Its failed trace and the &lt;code&gt;api&lt;/code&gt; and &lt;code&gt;inventory&lt;/code&gt; &lt;strong&gt;Console logs&lt;/strong&gt; you viewed earlier are still there:&lt;/p&gt;&#10;&#10;&lt;figure class="full-width"&gt;&#10; &lt;img&#10; class="my-0 rounded-md"&#10; loading="lazy"&#10; decoding="async"&#10; fetchpriority="auto"&#10; alt="Console logs for api in the pinned 10:08:18 PM run, filtered to 503, ending with the warning that inventory returned HTTP 503 with no retry, followed by the same trace ID"&#10; width="2560"&#10; height="400"&#10; src="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/pinned-run-console_hu_3246522501f34435.png"&#10; srcset="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/pinned-run-console_hu_3246522501f34435.png 800w,/posts/aspire-field-notes-keep-the-failing-run/pinned-run-console_hu_94423fe7fc0c854b.png 1280w"&#10; sizes="(min-width: 768px) 50vw, 65vw"&#10; data-zoom-src="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/pinned-run-console.png"&#10; /&gt;&#10; &#10; &lt;figcaption&gt;The historical API console contains the original 503 and its trace ID.&lt;/figcaption&gt;&#10; &lt;/figure&gt;&#10;&lt;p&gt;History is on by default. Pinning only protects a run from being pruned.&lt;/p&gt;&#10;&lt;p&gt;Change one thing between runs. If you update packages, request data, storage, and the fault setting at once, you won&amp;rsquo;t know which change mattered.&lt;/p&gt;&#10;&lt;p&gt;Go into the comparison with a question:&lt;/p&gt;&#10;&lt;table&gt;&#10;&#9;&lt;thead&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;Question&lt;/th&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;th&gt;What to compare&lt;/th&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/thead&gt;&#10;&#9;&lt;tbody&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Did the request succeed?&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Status code and response body the caller received&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Did the failure move elsewhere?&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Errors and span relationships across the request&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Did a retry hide the problem?&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Downstream attempts and elapsed time, where instrumented&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&#9;&#9;&lt;tr&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Did the environment change?&lt;/td&gt;&#10;&#9;&#9;&#9;&#9;&#9;&lt;td&gt;Resource properties, endpoints, and configuration relevant to the failure&lt;/td&gt;&#10;&#9;&#9;&#9;&lt;/tr&gt;&#10;&#9;&lt;/tbody&gt;&#10;&lt;/table&gt;&#10;&lt;p&gt;Be careful with duration alone. Warm caches and connection pools can make a second request faster even if the fix changed nothing.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;How much history the dashboard keeps&#10; &#10; &#10; &#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;By default, the dashboard keeps up to 10 unpinned runs per application and prunes the oldest when a new run starts. Pinned runs don&amp;rsquo;t count toward that limit.&lt;sup id="fnref2:2"&gt;&lt;a href="#fn:2" class="footnote-ref" role="doc-noteref"&gt;2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#10;&lt;p&gt;The dashboard groups runs by application name, so two clones of the same AppHost share one history and the same 10-run limit. A burst of runs in one checkout can prune an unpinned reproduction from the other.&lt;/p&gt;&#10;&lt;p&gt;Each run&amp;rsquo;s database also caps console logs, structured logs, and traces at 100,000 each by default, shared across resources. Metric points have their own limit. If a run exceeds those caps, its oldest data is gone, pinned or not.&lt;/p&gt;&#10;&lt;p&gt;Those caps count records, so large attributes and high-cardinality metrics can still take up a lot of disk space. Deleted rows free space inside the database file without shrinking it.&lt;/p&gt;&#10;&lt;p&gt;Unpin runs when you&amp;rsquo;ve finished investigating. If you need longer retention, use a telemetry backend with a retention policy and backups.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;What a historical run can tell you&#10; &#10; &#10; &#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;Historical runs are read-only. In last Tuesday&amp;rsquo;s run, you can&amp;rsquo;t restart a resource, run &lt;strong&gt;Load catalog&lt;/strong&gt;, or change a parameter.&lt;/p&gt;&#10;&lt;p&gt;The dashboard starts watching the AppHost&amp;rsquo;s resources only when a page first needs them. A run that nobody opens in a browser can keep traces and structured logs but no resource snapshot.&lt;sup id="fnref:3"&gt;&lt;a href="#fn:3" class="footnote-ref" role="doc-noteref"&gt;3&lt;/a&gt;&lt;/sup&gt; When there is a snapshot, it shows each resource as the dashboard last saw it.&lt;/p&gt;&#10;&lt;p&gt;Upgrades can strand old runs. The dashboard doesn&amp;rsquo;t migrate its schema. A &lt;code&gt;Run&lt;/code&gt; database from an incompatible version stays in the selector but won&amp;rsquo;t open, and an incompatible &lt;code&gt;Resume&lt;/code&gt; database is replaced. Before upgrading Aspire, save anything you still need from an old run.&lt;sup id="fnref3:2"&gt;&lt;a href="#fn:2" class="footnote-ref" role="doc-noteref"&gt;2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#10;&lt;p&gt;If you need to inspect or copy a database, include SQLite&amp;rsquo;s write-ahead log files, and don&amp;rsquo;t edit it with outside tools.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Under the hood: SQLite and Native AOT&#10; &#10; &#10; &#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;James Newton-King&amp;rsquo;s persistence deep dive explains the design. SQLite runs inside the dashboard process, with no database server to manage. Dapper queries filter, count, sort, and page telemetry in the database, so the dashboard doesn&amp;rsquo;t hold a run&amp;rsquo;s whole history as live .NET objects.&lt;sup id="fnref:4"&gt;&lt;a href="#fn:4" class="footnote-ref" role="doc-noteref"&gt;4&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#10;&lt;p&gt;In his large-telemetry test, private memory dropped from about 1,007 MB in Aspire 13.5 to 241 MB in 13.6. He measured each version once, after a forced garbage collection. Expect different numbers from this sample and from your own apps.&lt;/p&gt;&#10;&lt;p&gt;The 13.6 dashboard also ships as a Native AOT executable. His AOT write-up covers the work across Blazor, Fluent UI, serialization, and Dapper. For this workflow, the payoff is less startup and JIT work each time you stop and start the dashboard.&lt;sup id="fnref:5"&gt;&lt;a href="#fn:5" class="footnote-ref" role="doc-noteref"&gt;5&lt;/a&gt;&lt;/sup&gt; Dapper.AOT ties the two changes together by generating query and mapping code at build time.&lt;sup id="fnref1:4"&gt;&lt;a href="#fn:4" class="footnote-ref" role="doc-noteref"&gt;4&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#10;&lt;p&gt;Two caveats:&lt;/p&gt;&#10;&lt;ul&gt;&#10;&lt;li&gt;The native dashboard doesn&amp;rsquo;t need a separately installed .NET runtime, but the Aspire CLI still makes sure one is available for the rest of Aspire.&lt;/li&gt;&#10;&lt;li&gt;The dashboard&amp;rsquo;s experimental Blazor AOT work doesn&amp;rsquo;t make Native AOT a supported publishing option for Blazor Web Apps in general.&lt;/li&gt;&#10;&lt;/ul&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Protect what you keep&#10; &#10; &#10; &#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;Resource snapshots and telemetry can contain credentials, user data, and internal addresses. A value the dashboard masks on screen is still stored unredacted in the database.&lt;/p&gt;&#10;&lt;p&gt;The dashboard doesn&amp;rsquo;t add database authentication, encryption at rest, replication, or backups, so protect the data directory and any copies. Redact anything you share with a colleague, attach to an issue, or hand to an external tool.&lt;/p&gt;&#10;&lt;p&gt;For production retention and access control, use Application Insights or another telemetry backend built for that job.&lt;/p&gt;&#10;&#10;&lt;h2 class="relative group"&gt;Try this before the next refactor&#10; &#10; &#10; &#10; &#10;&lt;/h2&gt;&#10;&lt;p&gt;Stopping the app leaves dashboard history in place, so the pinned run is still there when you come back.&lt;/p&gt;&#10;&lt;p&gt;Next time you chase a bug, capture the failing request and pin its run before you change any code. After the fix, repeat the same request and compare the two traces. You&amp;rsquo;ll have something better to show in the review than &amp;ldquo;it worked after I restarted.&amp;rdquo;&lt;/p&gt;&#10;&lt;div class="footnotes" role="doc-endnotes"&gt;&#10;&lt;hr&gt;&#10;&lt;ol&gt;&#10;&lt;li id="fn:1"&gt;&#10;&lt;p&gt;Maddy Montaquila, &lt;a href="https://devblogs.microsoft.com/aspire/whats-new-aspire-13-6/" target="_blank" rel="noreferrer"&gt;Aspire 13.6: Your dashboard gets memory&lt;/a&gt;, September 29, 2026.&amp;#160;&lt;a href="#fnref:1" class="footnote-backref" role="doc-backlink"&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;&#10;&lt;/li&gt;&#10;&lt;li id="fn:2"&gt;&#10;&lt;p&gt;&lt;a href="https://aspire.dev/dashboard/data-persistence/" target="_blank" rel="noreferrer"&gt;Dashboard persistence modes, captured data, retention, compatibility, and security&lt;/a&gt;.&amp;#160;&lt;a href="#fnref:2" class="footnote-backref" role="doc-backlink"&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&amp;#160;&lt;a href="#fnref1:2" class="footnote-backref" role="doc-backlink"&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&amp;#160;&lt;a href="#fnref2:2" class="footnote-backref" role="doc-backlink"&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&amp;#160;&lt;a href="#fnref3:2" class="footnote-backref" role="doc-backlink"&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;&#10;&lt;/li&gt;&#10;&lt;li id="fn:3"&gt;&#10;&lt;p&gt;The 13.6.1 dashboard&amp;rsquo;s &lt;a href="https://github.com/microsoft/aspire/blob/v13.6.1/src/Aspire.Dashboard/ServiceClient/DashboardClient.cs" target="_blank" rel="noreferrer"&gt;&lt;code&gt;DashboardClient&lt;/code&gt;&lt;/a&gt; connects to the AppHost&amp;rsquo;s resource service on first use, then watches resources for the dashboard&amp;rsquo;s lifetime.&amp;#160;&lt;a href="#fnref:3" class="footnote-backref" role="doc-backlink"&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;&#10;&lt;/li&gt;&#10;&lt;li id="fn:4"&gt;&#10;&lt;p&gt;James Newton-King, &lt;a href="https://devblogs.microsoft.com/aspire/aspire-dashboard-persistence/" target="_blank" rel="noreferrer"&gt;Adding persistence to the Aspire dashboard&lt;/a&gt;, October 8, 2026.&amp;#160;&lt;a href="#fnref:4" class="footnote-backref" role="doc-backlink"&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&amp;#160;&lt;a href="#fnref1:4" class="footnote-backref" role="doc-backlink"&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;&#10;&lt;/li&gt;&#10;&lt;li id="fn:5"&gt;&#10;&lt;p&gt;James Newton-King, &lt;a href="https://devblogs.microsoft.com/aspire/aspire-dashboard-native-aot/" target="_blank" rel="noreferrer"&gt;Bringing Native AOT to the Aspire dashboard&lt;/a&gt;, October 6, 2026.&amp;#160;&lt;a href="#fnref:5" class="footnote-backref" role="doc-backlink"&gt;&amp;#x21a9;&amp;#xfe0e;&lt;/a&gt;&lt;/p&gt;&#10;&lt;/li&gt;&#10;&lt;/ol&gt;&#10;&lt;/div&gt;</content:encoded><media:content xmlns:media="http://search.yahoo.com/mrss/" url="https://chris-ayers.com/posts/aspire-field-notes-keep-the-failing-run/featured.png"/></item></channel></rss>