perf(retrieval): memoize per-entry tokens on the entry itself#9
Merged
Conversation
Code-review 2026-05-07: the keyword retriever's ``_score_entry``
re-tokenized path / summary / content-preview / aliases — and looped
over related entries' summaries — on EVERY query against EVERY entry.
For a corpus of N entries answering Q queries, that's N*Q tokenization
passes, all redoing identical work.
Add a ``_tokens_cache`` field on :class:`RetrievalEntry` (frozen
dataclass; ``compare=False, hash=False, repr=False`` so identity
semantics are unchanged) and route the retriever through two
helpers:
- ``_entry_field_tokens`` — keyed by ``("field_tokens", CONTENT_PREVIEW_CHARS)``.
Computes path/summary/content-preview/aliases once per entry per
retriever-class. Subclasses with different preview sizes get
independent cache slots automatically.
- ``_related_summary_tokens`` — keyed by ``("related_tokens", corpus.name)``.
Computes the union of related-entry summary tokens once per
(entry, corpus) pair. Cache lives on the entry, so when a corpus
rebuilds (new entry instances), it's naturally fresh.
Three new tests cover: cache populated on first call (same dict
returned on subsequent calls), preview-size-keyed independence, and
the bottom line — repeated ``_score_entry`` calls against the same
entry don't re-tokenize the entry's fields.
306 passed (up from 303).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Code-review finding (retrieval.py:179). The keyword retriever re-tokenized every entry's path/summary/content/aliases on every query, plus looped over related entries' summaries. N×Q tokenizations for N entries × Q queries.
Add a _tokens_cache sidecar on
RetrievalEntry(frozen + non-comparing/hashing/repr-ing) and route the scorer through two memoizing helpers. Cache is keyed by CONTENT_PREVIEW_CHARSand corpus.nameso subclasses and cross-corpus lookups don't collide.Three new tests. 306 passed (was 303).
🤖 Generated with Claude Code