Skip to content

feat: cache-hit-ratio analyzer with underutilization warnings#35

Open
jakduch wants to merge 5 commits into
josepe98:mainfrom
jakduch:feat/cache-hit-ratio
Open

feat: cache-hit-ratio analyzer with underutilization warnings#35
jakduch wants to merge 5 commits into
josepe98:mainfrom
jakduch:feat/cache-hit-ratio

Conversation

@jakduch
Copy link
Copy Markdown
Collaborator

@jakduch jakduch commented May 23, 2026

Glad to see this fork is actively maintained — the original phuryn/claude-usage had stopped accepting PRs (30+ open). Switching here solved real problems; more contributions on the way.

What does this add and why do you believe it belongs in this dashboard?

Adds a cache-hit-ratio analyzer so a personal Claude Code usage dashboard can finally answer the single question that drives most of the bill: am I actually getting prompt caching, and where am I leaving money on the table?

For every session we compute one number, cache_read / (input + cache_read) — the overall share of "fresh" prompt context that was served from cache instead of paid input — and bucket it into low / medium / high. The dashboard exposes a global cache_hit_summary (overall effectiveness, avg_ratio_pct, and counts by category) in /api/data, surfaces an insight card under the stats row, and tags individual sessions in Recent Sessions with a cache underused badge when they pull >50k input tokens but hit cache <30% of the time. Those are exactly the sessions paying 2–3× the necessary price for repeat context, and today they're invisible in this dashboard. With this card a user can immediately see "my big sessions aren't caching — restructure prompts here" without leaving the page. It fits the dashboard's existing insight-card pattern (next to the Pareto card) and adds no dependencies.

Checklist

Code correctness

  • All calcCost() calls pass 6 arguments: (model, inp, out, cache_read, cache_creation, cache_1h)no calcCost() calls added or modified
  • JavaScript template literals use bare backticks (`), not escaped ones (\`)
  • No JS variables referenced before they are defined (renderCacheHit is a hoisted function declaration; uses pre-existing fmt / rawData)
  • No new third-party dependencies introduced

Tests

  • python3 -m unittest discover -s tests -v — all passing on this branch (any test_health socket-bind flakiness is pre-existing on main and unrelated to this PR)
  • python3 -m unittest tests.test_browser -v — all passing (6/6)
  • New behaviour is covered by at least one test — 15 new tests in tests/test_cache_hit.py cover ratio math (including None inputs), bucket boundaries, summary aggregation, configurable input threshold, HTML wiring, and an end-to-end check through get_dashboard_data against a real SQLite schema

Scope

  • This is a single concern — one feature or fix per PR (cache-hit-ratio analyzer only)
  • Only touches existing files (dashboard.py, scanner.py, cli.py, pricing.py, cowork.py, tests/) — or I've explained below why a new file is needed: tests/test_cache_hit.py is added under tests/ per the existing one-file-per-feature test convention. .github/PULL_REQUEST_TEMPLATE.md was restored from main after an accidental delete on this branch.

jakduch added 2 commits May 23, 2026 17:51
Compute cache_read / (input + cache_read) per session and bucket into
low (<30%), medium, high (>70%). Surface a global avg and flag sessions
with input > 50k tokens but a sub-30% cache hit ratio (they're paying
full price for repeat context).

- dashboard.py: _cache_hit_ratio / _cache_hit_category / _cache_hit_analysis
  helpers; attach cache_hit_ratio + cache_underusing to each session in
  sessions_all; expose cache_hit_summary in /api/data; render an insight
  card and a 'cache underused' badge in the Recent Sessions table.
- tests/test_cache_hit.py: 15 tests (ratio math, bucketing, summary,
  threshold, HTML wiring, end-to-end via get_dashboard_data).
@josepe98 josepe98 closed this May 25, 2026
@josepe98 josepe98 reopened this May 25, 2026
@josepe98 josepe98 closed this May 25, 2026
@josepe98 josepe98 reopened this May 25, 2026
@josepe98 josepe98 closed this May 25, 2026
@josepe98 josepe98 reopened this May 25, 2026
@josepe98 josepe98 closed this May 25, 2026
@josepe98 josepe98 reopened this May 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants