feat(rag): native Anthropic Citations API (opt-in)#11
Merged
Conversation
New opt-in `use_native_citations=True` kwarg on
RagPipeline.run_and_generate sends retrieved hits as Anthropic
custom_content document blocks and parses claim-level citations
back into a new ClaimCitation type. The legacy [P{n}]-marker
path is preserved unchanged (it has a 2026-04-19 A/B-validated
hallucination win), so this is purely additive.
Surface added:
- attune_rag.ClaimCitation (response_span, document_index,
document_title, cited_text, cited_block_index)
- attune_rag.format_claim_citations_markdown — footnote-style
rendering of response text + citations
- attune_rag.providers.base.{CitationDocument, CitedResponse}
- LLMProvider.supports_native_citations flag +
generate_with_citations method (default NotImplementedError)
- ClaudeProvider implements both; GeminiProvider declares the
flag = False, pipeline detects this and falls back
Surface modified (additive only):
- RagResult gains claim_citations and used_native_citations,
both default empty/False so existing callers are unaffected
- attune-rag query --native-citations CLI flag
- attune-rag-benchmark --native-citations side-by-side column
Verification gates V2 (cache_control on document blocks) and V3
(document-count ceiling) are deferred — they require live API
spend. Conservative defaults: cache_control OFF on the new path
in v1, MAX_CITATION_DOCUMENTS=20 hardcoded with a clean
ValueError. Both flagged in docs/rag/native-citations.md.
Tests: +24 unit tests (8 ClaimCitation, 10 format_claim_citations
helper, 16 ClaudeProvider citations parser+payload, 8 pipeline
behavior matrix + parity). Full suite 348 passed. Retrieval P@1
unchanged at 95% (well above 73% workspace floor).
Spec: specs/rag-native-citations/{requirements,design,tasks}.md
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced May 8, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds opt-in native Anthropic Citations API support to attune-rag, sequenced as item (2) from the 2026-05-08 enhancements briefing.
use_native_citations=TrueonRagPipeline.run_and_generatesends retrieved hits ascustom_contentdocument blocks; the model emits claim-level citations attached to its response text. Returns claim-level attribution with character precision.[P{n}]path is preserved unchanged — it has a 2026-04-19 A/B-validated hallucination win that we don't throw away. New path is purely additive, default off, until benchmark validates.Spec:
specs/rag-native-citations/{requirements,design,tasks}.md.What landed
attune_rag.ClaimCitation— claim-level attribution dataclassattune_rag.format_claim_citations_markdown— footnote-style rendererLLMProvider.supports_native_citations+generate_with_citations(defaultNotImplementedError)ClaudeProvider.generate_with_citations(verified against SDK 0.96.0 —CitationContentBlockLocationfield names)RagResult.claim_citations+used_native_citations(additive; defaults preserve all existing callers)attune-rag query --native-citationsCLI flagattune-rag-benchmark --native-citationsside-by-side column (mean faithfulness, refusal rate, hallucination rate, citation emit rate, p95 latency)docs/rag/native-citations.mddesign noteTests
ClaimCitationdataclass + re-exportformat_claim_citations_markdown(footnote rendering, base_url, marker order, deduped footnote numbers, empty-citations note)ClaudeProvider.generate_with_citations(recorded fixture parser + payload shape;MAX_CITATION_DOCUMENTSboundary; defensive parsing for missing fields)These are documented in
docs/rag/native-citations.mdand were intentionally not run autonomously because they require Anthropic API credits:cache_controlon document blocks: empirically confirm a 2-call test yields cache hits when documents are identical. Currentlycache_controlis off on the native path; if V2 confirms parity with text-block caching, attachcache_control: ephemeralto the first document.MAX_CITATION_DOCUMENTS = 20inClaudeProvider. Re-verify against current API limits and adjust if needed.Both gates affect optional polish, not correctness. Native citations works end-to-end as committed; v2 just leaves performance on the table until verified.
Test plan
attune-rag-benchmark --with-faithfulness --native-citationson a real corpus to populate the side-by-side table🤖 Generated with Claude Code