wefio/dsh-cache-miss

中文

DSH web plugin: a yellow one-line prompt-cache-miss notice under an assistant reply whose request rebuilt the prompt cache — shown the moment the miss is detected from the stream's usage, usually the turn's first request.

What it does

An agent turn runs assistant -> tool -> assistant -> tool .... The turn's first model call is the moment the provider's prompt cache may have expired and needs a full re-prefill (cache rebuild); later calls in the same turn usually hit the just-rebuilt cache. DSH's own token/cache stats sit under the composer and the produced-files row sits at the turn tail, neither of which indicates a miss where it actually happens.

This plugin renders, under each assistant reply whose request missed, a single yellow line, live, as soon as that step's usage chunk arrives:

Cache miss after 3m idle: 182k tokens re-billed · 0.8k cached · ttft 2.1s ↑

• idle — gap from the previous turn's end to this one's start.
• re-billed — the request's uncached input tokens (abbreviated to k).
• cached — the cached-read portion of the same prefill (when the provider reports it), so the rebilled-vs-cached split stays unambiguous.
• ttft — first-token latency, when available; the up arrow hints a rebuild prefill usually ran slower.

It is pure presentation: nothing is written to the session log, no DSH source is modified, and it does not take the turn-tail chain, so it does not collide with the produced-files row (e.g. DSH-better-sidebar).

Miss definition

inputTokens is the disjoint "uncached input only" count and cacheReadTokens the cached part of the same prefill, so the cache-hit ratio is hitRatio = cacheReadTokens / (inputTokens + cacheReadTokens). A request is a miss when, all together:

• inputTokens > 0,
• hitRatio < 80% (over 20% of the prefill was uncached — significant because context accumulates), and
• inputTokens >= 1000 (at least 1k tokens actually re-billed).

A provider that reports usage but no cache fields cannot be classified until it has ever shown a cache field. Before that evidence exists, the plugin does not misreport the request as a miss; it shows a grey notice Provider reports no cache fields — cannot confirm cache status. Each such provider is noticed once; switching to another no-evidence provider surfaces a fresh notice, with a more detailed per-provider warning in the console. Once a provider has shown any cache field, later requests without fields are treated as full misses and show the yellow miss line normally. A provider that reports no usage at all stays silent. A normal continuation reusing the just-built cache therefore stays quiet (hit ratio ≥ 80%), and re-billed shows only the uncached inputTokens.

The miss is read from the stream's own usage chunk — adapters emit it before the terminal finish — so the notice appears as soon as the usage lands (while the reply is still streaming), not only once the assistant message settles. The line is logged to the browser console once per step, stamped in the browser's local time zone.

Install

dsh plugin --profile web add dsh-cache-miss

Restart dsh web (or hard-refresh the running GUI) to load the client bundle.

Known limitations

• TTFT is the wall-clock gap between the step's step/start event and its first non-empty token delta, computed from session event times. It is absent (rendered as if no timing) when a boundary fell outside the loaded window.
• The node publishes for every assistant step, but renders nothing for cache hits, so a hit turn contributes no visible row. Each step logs at most one console line regardless of how many renders follow.
• Providers that have never reported cache fields get one grey notice per provider and no heuristic guess: TTFT is too sensitive to network/load to reliably separate "slow because of a cache miss" from "just slow".