feat: cache-hit-ratio analyzer with underutilization warnings#35
Open
jakduch wants to merge 5 commits into
Open
feat: cache-hit-ratio analyzer with underutilization warnings#35jakduch wants to merge 5 commits into
jakduch wants to merge 5 commits into
Conversation
Compute cache_read / (input + cache_read) per session and bucket into low (<30%), medium, high (>70%). Surface a global avg and flag sessions with input > 50k tokens but a sub-30% cache hit ratio (they're paying full price for repeat context). - dashboard.py: _cache_hit_ratio / _cache_hit_category / _cache_hit_analysis helpers; attach cache_hit_ratio + cache_underusing to each session in sessions_all; expose cache_hit_summary in /api/data; render an insight card and a 'cache underused' badge in the Recent Sessions table. - tests/test_cache_hit.py: 15 tests (ratio math, bucketing, summary, threshold, HTML wiring, end-to-end via get_dashboard_data).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this add and why do you believe it belongs in this dashboard?
Adds a cache-hit-ratio analyzer so a personal Claude Code usage dashboard can finally answer the single question that drives most of the bill: am I actually getting prompt caching, and where am I leaving money on the table?
For every session we compute one number,
cache_read / (input + cache_read)— the overall share of "fresh" prompt context that was served from cache instead of paid input — and bucket it into low / medium / high. The dashboard exposes a globalcache_hit_summary(overall effectiveness,avg_ratio_pct, and counts by category) in/api/data, surfaces an insight card under the stats row, and tags individual sessions in Recent Sessions with acache underusedbadge when they pull >50k input tokens but hit cache <30% of the time. Those are exactly the sessions paying 2–3× the necessary price for repeat context, and today they're invisible in this dashboard. With this card a user can immediately see "my big sessions aren't caching — restructure prompts here" without leaving the page. It fits the dashboard's existing insight-card pattern (next to the Pareto card) and adds no dependencies.Checklist
Code correctness
calcCost()calls pass 6 arguments:(model, inp, out, cache_read, cache_creation, cache_1h)— nocalcCost()calls added or modified`), not escaped ones (\`)renderCacheHitis a hoisted function declaration; uses pre-existingfmt/rawData)Tests
python3 -m unittest discover -s tests -v— all passing on this branch (anytest_healthsocket-bind flakiness is pre-existing onmainand unrelated to this PR)python3 -m unittest tests.test_browser -v— all passing (6/6)tests/test_cache_hit.pycover ratio math (includingNoneinputs), bucket boundaries, summary aggregation, configurable input threshold, HTML wiring, and an end-to-end check throughget_dashboard_dataagainst a real SQLite schemaScope
dashboard.py,scanner.py,cli.py,pricing.py,cowork.py,tests/) — or I've explained below why a new file is needed:tests/test_cache_hit.pyis added undertests/per the existing one-file-per-feature test convention..github/PULL_REQUEST_TEMPLATE.mdwas restored frommainafter an accidental delete on this branch.