diff --git a/js/plugins/anthropic/README.md b/js/plugins/anthropic/README.md index 99bb31e688..2f664d54ba 100644 --- a/js/plugins/anthropic/README.md +++ b/js/plugins/anthropic/README.md @@ -4,11 +4,16 @@

Anthropic AI plugin for Google Firebase Genkit

-`@genkit-ai/anthropic` is the official Anthropic plugin for [Firebase Genkit](https://github.com/firebase/genkit). It supersedes the earlier community package `genkitx-anthropic` and is now maintained by Google. +`@genkit-ai/anthropic` is the official Anthropic plugin for +[Firebase Genkit](https://github.com/firebase/genkit). It supersedes the earlier +community package `genkitx-anthropic` and is now maintained by Google. ## Supported models -The plugin supports the most recent Anthropic models: **Claude Haiku 4.5**, **Claude Sonnet 4.5**, and **Claude Opus 4.5**. Additionally, the plugin supports all of the [non-retired older models](https://platform.claude.com/docs/en/about-claude/model-deprecations#model-status). +The plugin supports the most recent Anthropic models: **Claude Haiku 4.5**, +**Claude Sonnet 4.5**, and **Claude Opus 4.5**. Additionally, the plugin +supports all of the +[non-retired older models](https://platform.claude.com/docs/en/about-claude/model-deprecations#model-status). ## Installation @@ -23,13 +28,13 @@ Install the plugin in your project with your favorite package manager: ### Initialize ```typescript -import { genkit } from 'genkit'; -import { anthropic } from '@genkit-ai/anthropic'; +import { genkit } from "genkit"; +import { anthropic } from "@genkit-ai/anthropic"; const ai = genkit({ plugins: [anthropic({ apiKey: process.env.ANTHROPIC_API_KEY })], // specify a default model for generate here if you wish: - model: anthropic.model('claude-sonnet-4-5'), + model: anthropic.model("claude-sonnet-4-5"), }); ``` @@ -39,8 +44,8 @@ The simplest way to generate text is by using the `generate` method: ```typescript const response = await ai.generate({ - model: anthropic.model('claude-haiku-4-5'), - prompt: 'Tell me a joke.', + model: anthropic.model("claude-haiku-4-5"), + prompt: "Tell me a joke.", }); console.log(response.text); @@ -53,7 +58,7 @@ console.log(response.text); const response = await ai.generate({ prompt: [ - { text: 'What animal is in the photo?' }, + { text: "What animal is in the photo?" }, { media: { url: imageUrl } }, ], }); @@ -62,11 +67,12 @@ console.log(response.text); ### Extended thinking -Claude 4.5 models can expose their internal reasoning. Enable it per-request with the Anthropic thinking config and read the reasoning from the response: +Claude 4.5 models can expose their internal reasoning. Enable it per-request +with the Anthropic thinking config and read the reasoning from the response: ```typescript const response = await ai.generate({ - prompt: 'Walk me through your reasoning for Fermat’s little theorem.', + prompt: "Walk me through your reasoning for Fermat’s little theorem.", config: { thinking: { enabled: true, @@ -75,15 +81,76 @@ const response = await ai.generate({ }, }); -console.log(response.text); // Final assistant answer -console.log(response.reasoning); // Summarized thinking steps +console.log(response.text); // Final assistant answer +console.log(response.reasoning); // Summarized thinking steps ``` -When thinking is enabled, request bodies sent through the plugin include the `thinking` payload (`{ type: 'enabled', budget_tokens: … }`) that Anthropic's API expects, and streamed responses deliver `reasoning` parts as they arrive so you can render the chain-of-thought incrementally. +When thinking is enabled, request bodies sent through the plugin include the +`thinking` payload (`{ type: 'enabled', budget_tokens: … }`) that Anthropic's +API expects, and streamed responses deliver `reasoning` parts as they arrive so +you can render the chain-of-thought incrementally. + +### Document Citations + +Claude can cite specific parts of documents you provide, making it easy to trace +where information in the response came from. Use the `anthropicDocument()` +helper to create citable documents. For more details, see the +[Anthropic Citations documentation](https://platform.claude.com/docs/en/build-with-claude/citations). + +```typescript +import { anthropic, anthropicDocument } from "@genkit-ai/anthropic"; + +const response = await ai.generate({ + model: anthropic.model("claude-sonnet-4-5"), + messages: [ + { + role: "user", + content: [ + anthropicDocument({ + source: { + type: "text", + data: "The grass is green. The sky is blue. Water is wet.", + }, + title: "Basic Facts", + citations: { enabled: true }, + }), + { text: "What color is the grass? Cite your source." }, + ], + }, + ], +}); + +// Access citations from response parts +const citations = response.message?.content?.flatMap( + (part) => part.metadata?.citations || [], +) ?? []; + +console.log("Citations:", citations); +``` + +**Important:** Citations must be enabled on all documents in a request, or on +none of them. You cannot mix documents with citations enabled and disabled in +the same request. + +Supported document source types: + +- `text` - Plain text documents (returns `char_location` citations) +- `base64` - Base64-encoded PDFs (returns `page_location` citations) +- `url` - PDFs accessible via URL (returns `page_location` citations) +- `content` - Custom content blocks with text/images (returns + `content_block_location` citations) +- `file` - File references from Anthropic's Files API, beta API only (returns + `page_location` citations) + +Citations are returned in the response parts' metadata and include information +about the document index, cited text, and location (character indices, page +numbers, or block indices depending on the source type). ### Beta API Limitations -The beta API surface provides access to experimental features, but some server-managed tool blocks are not yet supported by this plugin. The following beta API features will cause an error if encountered: +The beta API surface provides access to experimental features, but some +server-managed tool blocks are not yet supported by this plugin. The following +beta API features will cause an error if encountered: - `web_fetch_tool_result` - `code_execution_tool_result` @@ -93,18 +160,19 @@ The beta API surface provides access to experimental features, but some server-m - `mcp_tool_use` - `container_upload` -Note that `server_tool_use` and `web_search_tool_result` ARE supported and work with both stable and beta APIs. +Note that `server_tool_use` and `web_search_tool_result` ARE supported and work +with both stable and beta APIs. ### Within a flow ```typescript -import { z } from 'genkit'; +import { z } from "genkit"; // ...initialize Genkit instance (as shown above)... export const jokeFlow = ai.defineFlow( { - name: 'jokeFlow', + name: "jokeFlow", inputSchema: z.string(), outputSchema: z.string(), }, @@ -113,26 +181,27 @@ export const jokeFlow = ai.defineFlow( prompt: `tell me a joke about ${subject}`, }); return llmResponse.text; - } + }, ); ``` ### Direct model usage (without Genkit instance) -The plugin supports Genkit Plugin API v2, which allows you to use models directly without initializing the full Genkit framework: +The plugin supports Genkit Plugin API v2, which allows you to use models +directly without initializing the full Genkit framework: ```typescript -import { anthropic } from '@genkit-ai/anthropic'; +import { anthropic } from "@genkit-ai/anthropic"; // Create a model reference directly -const claude = anthropic.model('claude-sonnet-4-5'); +const claude = anthropic.model("claude-sonnet-4-5"); // Use the model directly const response = await claude({ messages: [ { - role: 'user', - content: [{ text: 'Tell me a joke.' }], + role: "user", + content: [{ text: "Tell me a joke." }], }, ], }); @@ -143,19 +212,19 @@ console.log(response); You can also create model references using the plugin's `model()` method: ```typescript -import { anthropic } from '@genkit-ai/anthropic'; +import { anthropic } from "@genkit-ai/anthropic"; // Create model references -const claudeHaiku45 = anthropic.model('claude-haiku-4-5'); -const claudeSonnet45 = anthropic.model('claude-sonnet-4-5'); -const claudeOpus45 = anthropic.model('claude-opus-4-5'); +const claudeHaiku45 = anthropic.model("claude-haiku-4-5"); +const claudeSonnet45 = anthropic.model("claude-sonnet-4-5"); +const claudeOpus45 = anthropic.model("claude-opus-4-5"); // Use the model reference directly const response = await claudeSonnet45({ messages: [ { - role: 'user', - content: [{ text: 'Hello!' }], + role: "user", + content: [{ text: "Hello!" }], }, ], }); @@ -169,22 +238,29 @@ This approach is useful for: ## Acknowledgements -This plugin builds on the community work published as [`genkitx-anthropic`](https://github.com/BloomLabsInc/genkit-plugins/blob/main/plugins/anthropic/README.md) by Bloom Labs Inc. Their Apache 2.0–licensed implementation provided the foundation for this maintained package. +This plugin builds on the community work published as +[`genkitx-anthropic`](https://github.com/BloomLabsInc/genkit-plugins/blob/main/plugins/anthropic/README.md) +by Bloom Labs Inc. Their Apache 2.0–licensed implementation provided the +foundation for this maintained package. ## Contributing -Want to contribute to the project? That's awesome! Head over to our [Contribution Guidelines](CONTRIBUTING.md). +Want to contribute to the project? That's awesome! Head over to our +[Contribution Guidelines](CONTRIBUTING.md). ## Need support? > [!NOTE] -> This repository depends on Google's Firebase Genkit. For issues and questions related to Genkit, please refer to instructions available in [Genkit's repository](https://github.com/firebase/genkit). - +> This repository depends on Google's Firebase Genkit. For issues and questions +> related to Genkit, please refer to instructions available in +> [Genkit's repository](https://github.com/firebase/genkit). ## Credits -This plugin is maintained by Google with acknowledgement to the community contributions from [Bloom Labs Inc](https://github.com/BloomLabsInc). +This plugin is maintained by Google with acknowledgement to the community +contributions from [Bloom Labs Inc](https://github.com/BloomLabsInc). ## License -This project is licensed under the [Apache 2.0 License](https://github.com/BloomLabsInc/genkit-plugins/blob/main/LICENSE). +This project is licensed under the +[Apache 2.0 License](https://github.com/BloomLabsInc/genkit-plugins/blob/main/LICENSE). diff --git a/js/plugins/anthropic/src/index.ts b/js/plugins/anthropic/src/index.ts index d5a0fef9cb..1c4986c680 100644 --- a/js/plugins/anthropic/src/index.ts +++ b/js/plugins/anthropic/src/index.ts @@ -18,6 +18,7 @@ import Anthropic from '@anthropic-ai/sdk'; import { genkitPluginV2, type GenkitPluginV2 } from 'genkit/plugin'; +import type { Part } from 'genkit'; import { ActionMetadata, ModelReference, z } from 'genkit'; import { ModelAction } from 'genkit/model'; import { ActionType } from 'genkit/registry'; @@ -31,7 +32,15 @@ import { claudeModel, claudeModelReference, } from './models.js'; -import { InternalPluginOptions, PluginOptions, __testClient } from './types.js'; +import { + InternalPluginOptions, + PluginOptions, + __testClient, + type AnthropicDocumentOptions, +} from './types.js'; + +// Re-export citation type for consumers (AnthropicDocumentOptions is inferred via anthropicDocument()) +export type { AnthropicCitation } from './types.js'; const PROMPT_CACHING_BETA_HEADER_VALUE = 'prompt-caching-2024-07-31'; @@ -152,4 +161,38 @@ export const anthropic = anthropicPlugin as AnthropicPlugin; return claudeModelReference(name, config); }; +/** + * Creates a custom part representing an Anthropic document with optional citations support. + * + * Use this to provide documents to Claude that can be cited in responses. + * Citations must be enabled on all or none of the documents in a request. + * + * @example + * ```ts + * import { anthropic, anthropicDocument } from '@genkit-ai/anthropic'; + * + * const { text } = await ai.generate({ + * model: anthropic.model('claude-sonnet-4-5'), + * messages: [{ + * role: 'user', + * content: [ + * anthropicDocument({ + * source: { type: 'text', data: 'The grass is green. The sky is blue.' }, + * title: 'Nature Facts', + * citations: { enabled: true } + * }), + * { text: 'What color is the grass?' } + * ] + * }] + * }); + * ``` + */ +export function anthropicDocument(options: AnthropicDocumentOptions): Part { + return { + custom: { + anthropicDocument: options, + }, + }; +} + export default anthropic; diff --git a/js/plugins/anthropic/src/runner/beta.ts b/js/plugins/anthropic/src/runner/beta.ts index 4efcd1fd13..69a7e41755 100644 --- a/js/plugins/anthropic/src/runner/beta.ts +++ b/js/plugins/anthropic/src/runner/beta.ts @@ -43,14 +43,20 @@ import type { import { logger } from 'genkit/logging'; import { KNOWN_CLAUDE_MODELS, extractVersion } from '../models.js'; -import { AnthropicConfigSchema, type ClaudeRunnerParams } from '../types.js'; +import { + AnthropicConfigSchema, + type AnthropicDocumentOptions, + type ClaudeRunnerParams, +} from '../types.js'; import { removeUndefinedProperties } from '../utils.js'; import { BaseRunner } from './base.js'; import { betaServerToolUseBlockToPart, + toBetaDocumentBlock, unsupportedServerToolError, } from './converters/beta.js'; import { + citationsDeltaToPart, inputJsonDeltaError, redactedThinkingBlockToPart, textBlockToPart, @@ -182,6 +188,13 @@ export class BetaRunner extends BaseRunner { return { type: 'text', text: part.text }; } + // Custom document (for citations support) + if (part.custom?.anthropicDocument) { + return toBetaDocumentBlock( + part.custom.anthropicDocument as AnthropicDocumentOptions + ); + } + // Media if (part.media) { if (part.media.contentType === 'anthropic/file') { @@ -463,6 +476,9 @@ export class BetaRunner extends BaseRunner { if (event.delta.type === 'thinking_delta') { return thinkingDeltaToPart(event.delta); } + if (event.delta.type === 'citations_delta') { + return citationsDeltaToPart(event.delta); + } if (event.delta.type === 'input_json_delta') { throw inputJsonDeltaError(); } diff --git a/js/plugins/anthropic/src/runner/converters/beta.ts b/js/plugins/anthropic/src/runner/converters/beta.ts index c597dbb848..4b1fa49bb4 100644 --- a/js/plugins/anthropic/src/runner/converters/beta.ts +++ b/js/plugins/anthropic/src/runner/converters/beta.ts @@ -18,7 +18,10 @@ * Converters for beta API content blocks. */ +import type { BetaRequestDocumentBlock } from '@anthropic-ai/sdk/resources/beta/messages'; import type { Part } from 'genkit'; +import type { AnthropicDocumentOptions } from '../../types.js'; +import { convertDocumentSource, createDocumentBlock } from './shared.js'; /** * Converts a server_tool_use block to a Genkit Part. @@ -52,3 +55,22 @@ export function betaServerToolUseBlockToPart(block: { export function unsupportedServerToolError(blockType: string): string { return `Anthropic beta runner does not yet support server-managed tool block '${blockType}'. Please retry against the stable API or wait for dedicated support.`; } + +/** + * Converts AnthropicDocumentOptions to Anthropic's beta API document block format. + * The beta API supports file-based sources via the Files API. + */ +export function toBetaDocumentBlock( + options: AnthropicDocumentOptions +): BetaRequestDocumentBlock { + return createDocumentBlock(options, (source) => + convertDocumentSource( + source, + (fileId) => + ({ + type: 'file', + file_id: fileId, + }) as BetaRequestDocumentBlock['source'] + ) + ); +} diff --git a/js/plugins/anthropic/src/runner/converters/citations.ts b/js/plugins/anthropic/src/runner/converters/citations.ts new file mode 100644 index 0000000000..a6d5ab9a94 --- /dev/null +++ b/js/plugins/anthropic/src/runner/converters/citations.ts @@ -0,0 +1,144 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/** + * Citation conversion utilities for Anthropic API responses. + * Handles validation and conversion of citations from Anthropic's format to Genkit format. + */ + +import { z } from 'genkit'; +import type { AnthropicCitation } from '../../types.js'; + +/** Structural type for Anthropic citations (works with both stable and beta APIs). */ +export interface AnthropicCitationInput { + type: string; + cited_text: string; + // document_index is optional since web search citations don't have it + document_index?: number; + document_title?: string | null; + file_id?: string | null; + start_char_index?: number; + end_char_index?: number; + start_page_number?: number; + end_page_number?: number; + start_block_index?: number; + end_block_index?: number; +} + +// --- Citation validation schemas --- + +/** + * Base citation schema with common fields shared across all citation types. + */ +const baseCitationSchema = z.object({ + cited_text: z.string(), + document_index: z.number(), + document_title: z.string().nullable().optional(), + file_id: z.string().nullable().optional(), +}); + +/** + * Schema for character location citations (plain text documents). + */ +const charLocationCitationSchema = baseCitationSchema.extend({ + type: z.literal('char_location'), + start_char_index: z.number(), + end_char_index: z.number(), +}); + +/** + * Schema for page location citations (PDF documents). + */ +const pageLocationCitationSchema = baseCitationSchema.extend({ + type: z.literal('page_location'), + start_page_number: z.number(), + end_page_number: z.number(), +}); + +/** + * Schema for content block location citations (custom content documents). + */ +const contentBlockLocationCitationSchema = baseCitationSchema.extend({ + type: z.literal('content_block_location'), + start_block_index: z.number(), + end_block_index: z.number(), +}); + +/** + * Discriminated union schema for all supported citation types. + */ +const citationSchema = z.discriminatedUnion('type', [ + charLocationCitationSchema, + pageLocationCitationSchema, + contentBlockLocationCitationSchema, +]); + +/** + * Converts Anthropic's citation format (snake_case) to genkit format (camelCase). + * Only handles document-based citations (char_location, page_location, content_block_location). + * Skips web search and other citation types that don't reference documents. + */ +export function fromAnthropicCitation( + citation: AnthropicCitationInput +): AnthropicCitation | undefined { + // Skip citations without document_index (e.g., web search results) + if (citation.document_index === undefined) { + return undefined; + } + + // Validate and parse citation with Zod + const result = citationSchema.safeParse(citation); + if (!result.success) { + // Invalid citation structure, skip it gracefully + return undefined; + } + + const parsed = result.data; + + // Convert validated citation to Genkit format + switch (parsed.type) { + case 'char_location': + return { + type: 'char_location', + citedText: parsed.cited_text, + documentIndex: parsed.document_index, + documentTitle: parsed.document_title ?? undefined, + fileId: parsed.file_id ?? undefined, + startCharIndex: parsed.start_char_index, + endCharIndex: parsed.end_char_index, + }; + case 'page_location': + return { + type: 'page_location', + citedText: parsed.cited_text, + documentIndex: parsed.document_index, + documentTitle: parsed.document_title ?? undefined, + fileId: parsed.file_id ?? undefined, + startPageNumber: parsed.start_page_number, + endPageNumber: parsed.end_page_number, + }; + case 'content_block_location': + return { + type: 'content_block_location', + citedText: parsed.cited_text, + documentIndex: parsed.document_index, + documentTitle: parsed.document_title ?? undefined, + fileId: parsed.file_id ?? undefined, + startBlockIndex: parsed.start_block_index, + endBlockIndex: parsed.end_block_index, + }; + } +} diff --git a/js/plugins/anthropic/src/runner/converters/shared.ts b/js/plugins/anthropic/src/runner/converters/shared.ts index 6d6faf6091..c40cc82d4d 100644 --- a/js/plugins/anthropic/src/runner/converters/shared.ts +++ b/js/plugins/anthropic/src/runner/converters/shared.ts @@ -20,11 +20,41 @@ */ import type { Part } from 'genkit'; +import type { + AnthropicCitation, + AnthropicDocumentOptions, +} from '../../types.js'; +import { MEDIA_TYPES, MediaTypeSchema } from '../../types.js'; +import { + fromAnthropicCitation, + type AnthropicCitationInput, +} from './citations.js'; + +// Re-export citation utilities for backward compatibility +export { + fromAnthropicCitation, + type AnthropicCitationInput, +} from './citations.js'; /** - * Converts a text block to a Genkit Part. + * Converts a text block to a Genkit Part, including citations if present. + * Uses structural typing for compatibility with both stable and beta APIs. */ -export function textBlockToPart(block: { text: string }): Part { +export function textBlockToPart(block: { + text: string; + citations?: AnthropicCitationInput[] | null; +}): Part { + if (block.citations && block.citations.length > 0) { + const citations = block.citations + .map((c) => fromAnthropicCitation(c)) + .filter((c): c is AnthropicCitation => c !== undefined); + if (citations.length > 0) { + return { + text: block.text, + metadata: { citations }, + }; + } + } return { text: block.text }; } @@ -103,6 +133,27 @@ export function thinkingDeltaToPart(delta: { thinking: string }): Part { return { reasoning: delta.thinking }; } +/** + * Converts a citations_delta to a Genkit Part for streaming. + * Returns a text part with empty text and citation data in metadata. + * Empty text is intentional: genkit's `.text` getter concatenates all text parts, + * so empty strings contribute nothing to the final text while preserving the citation + * in the parts array for consumers who need to access citation metadata. + */ +export function citationsDeltaToPart(delta: { + type: 'citations_delta'; + citation: AnthropicCitationInput; +}): Part | undefined { + const citation = fromAnthropicCitation(delta.citation); + if (citation) { + return { + text: '', + metadata: { citations: [citation] }, + }; + } + return undefined; +} + /** * Error for unsupported input_json_delta in streaming. */ @@ -111,3 +162,96 @@ export function inputJsonDeltaError(): Error { 'Anthropic streaming tool input (input_json_delta) is not yet supported. Please disable streaming or upgrade this plugin.' ); } + +// --- Document block converters (shared between stable and beta APIs) --- + +/** + * Document block type constraint for generics. + */ +type DocumentBlockBase = { + type: 'document'; + source: unknown; + title?: string | null; + context?: string | null; + citations?: { enabled?: boolean } | null; +}; + +/** + * Converts AnthropicDocumentOptions to Anthropic's document block format. + * Works for both stable and beta APIs via generics. + */ +export function createDocumentBlock( + options: AnthropicDocumentOptions, + sourceConverter: (source: AnthropicDocumentOptions['source']) => T['source'] +): T { + return { + type: 'document' as const, + source: sourceConverter(options.source), + ...(options.title && { title: options.title }), + ...(options.context && { context: options.context }), + ...(options.citations && { citations: options.citations }), + } as T; +} + +/** + * Converts document source options to Anthropic's source format. + * Works for both stable and beta APIs via a file handler callback. + * The file handler is called for 'file' type sources, allowing different + * behavior (error for stable, conversion for beta). + */ +export function convertDocumentSource( + source: AnthropicDocumentOptions['source'], + fileHandler: (fileId: string) => T +): T { + switch (source.type) { + case 'text': + return { + type: 'text', + media_type: (source.mediaType ?? 'text/plain') as 'text/plain', + data: source.data, + } as T; + case 'base64': + return { + type: 'base64', + media_type: source.mediaType as 'application/pdf', + data: source.data, + } as T; + case 'file': + return fileHandler(source.fileId); + case 'content': + return { + type: 'content', + content: source.content.map((item) => { + if (item.type === 'text') { + return item; + } + // Validate media type with Zod + const mediaTypeResult = MediaTypeSchema.safeParse( + item.source.mediaType + ); + if (!mediaTypeResult.success) { + throw new Error( + `Unsupported image media type for Anthropic document content: ${item.source.mediaType}. Supported types: ${Object.values(MEDIA_TYPES).join(', ')}` + ); + } + return { + type: 'image' as const, + source: { + type: 'base64' as const, + media_type: mediaTypeResult.data, + data: item.source.data, + }, + }; + }), + } as T; + case 'url': + return { + type: 'url', + url: source.url, + } as T; + default: + throw new Error( + `Unsupported document source type: ${(source as { type: string }).type}` + ); + } +} diff --git a/js/plugins/anthropic/src/runner/converters/stable.ts b/js/plugins/anthropic/src/runner/converters/stable.ts index d6b3508d42..996ffa208d 100644 --- a/js/plugins/anthropic/src/runner/converters/stable.ts +++ b/js/plugins/anthropic/src/runner/converters/stable.ts @@ -18,7 +18,10 @@ * Converters for stable API content blocks. */ +import type { DocumentBlockParam } from '@anthropic-ai/sdk/resources/messages'; import type { Part } from 'genkit'; +import type { AnthropicDocumentOptions } from '../../types.js'; +import { convertDocumentSource, createDocumentBlock } from './shared.js'; /** * Converts a server_tool_use block to a Genkit Part. @@ -40,3 +43,19 @@ export function serverToolUseBlockToPart(block: { }, }; } + +/** + * Converts AnthropicDocumentOptions to Anthropic's stable API document block format. + * Note: The stable API does not support file-based sources (Files API). + */ +export function toDocumentBlock( + options: AnthropicDocumentOptions +): DocumentBlockParam { + return createDocumentBlock(options, (source) => + convertDocumentSource(source, () => { + throw new Error( + 'File-based document sources require the beta API. Set apiVersion: "beta" in your plugin config or request config.' + ); + }) + ); +} diff --git a/js/plugins/anthropic/src/runner/stable.ts b/js/plugins/anthropic/src/runner/stable.ts index 61c921b97c..7e9907778a 100644 --- a/js/plugins/anthropic/src/runner/stable.ts +++ b/js/plugins/anthropic/src/runner/stable.ts @@ -41,10 +41,15 @@ import type { import { logger } from 'genkit/logging'; import { KNOWN_CLAUDE_MODELS, extractVersion } from '../models.js'; -import { AnthropicConfigSchema, type ClaudeRunnerParams } from '../types.js'; +import { + AnthropicConfigSchema, + type AnthropicDocumentOptions, + type ClaudeRunnerParams, +} from '../types.js'; import { removeUndefinedProperties } from '../utils.js'; import { BaseRunner } from './base.js'; import { + citationsDeltaToPart, inputJsonDeltaError, redactedThinkingBlockToPart, textBlockToPart, @@ -54,7 +59,10 @@ import { toolUseBlockToPart, webSearchToolResultBlockToPart, } from './converters/shared.js'; -import { serverToolUseBlockToPart } from './converters/stable.js'; +import { + serverToolUseBlockToPart, + toDocumentBlock, +} from './converters/stable.js'; import { RunnerTypes as BaseRunnerTypes } from './types.js'; interface RunnerTypes extends BaseRunnerTypes { @@ -121,6 +129,13 @@ export class Runner extends BaseRunner { }; } + // Custom document (for citations support) + if (part.custom?.anthropicDocument) { + return toDocumentBlock( + part.custom.anthropicDocument as AnthropicDocumentOptions + ); + } + if (part.media) { if (part.media.contentType === 'application/pdf') { return { @@ -355,6 +370,10 @@ export class Runner extends BaseRunner { return thinkingDeltaToPart(delta); } + if (delta.type === 'citations_delta') { + return citationsDeltaToPart(delta); + } + if (delta.type === 'input_json_delta') { throw inputJsonDeltaError(); } diff --git a/js/plugins/anthropic/src/types.ts b/js/plugins/anthropic/src/types.ts index 2f61464a10..77744be2c1 100644 --- a/js/plugins/anthropic/src/types.ts +++ b/js/plugins/anthropic/src/types.ts @@ -195,3 +195,96 @@ export function resolveBetaEnabled( if (pluginDefaultApiVersion === 'beta') return true; return false; } + +/** Plain text document source. */ +export interface AnthropicTextSource { + type: 'text'; + data: string; + mediaType?: string; +} + +/** Base64-encoded document source (e.g., PDF). */ +export interface AnthropicBase64Source { + type: 'base64'; + data: string; + mediaType: string; +} + +/** File reference source (from Files API). */ +export interface AnthropicFileSource { + type: 'file'; + fileId: string; +} + +/** Custom content blocks for granular citation control. */ +export interface AnthropicContentSource { + type: 'content'; + content: Array< + | { type: 'text'; text: string } + | { + type: 'image'; + source: { type: 'base64'; mediaType: string; data: string }; + } + >; +} + +/** URL source for PDFs. */ +export interface AnthropicURLSource { + type: 'url'; + url: string; +} + +/** Union of all document source types. */ +export type AnthropicDocumentSource = + | AnthropicTextSource + | AnthropicBase64Source + | AnthropicFileSource + | AnthropicContentSource + | AnthropicURLSource; + +/** Options for creating an Anthropic document with optional citations. */ +export interface AnthropicDocumentOptions { + source: AnthropicDocumentSource; + title?: string; + context?: string; + citations?: { enabled: boolean }; +} + +/** Citation from a plain text document (character indices). */ +export interface CharLocationCitation { + type: 'char_location'; + citedText: string; + documentIndex: number; + documentTitle?: string; + fileId?: string; + startCharIndex: number; + endCharIndex: number; +} + +/** Citation from a PDF document (page numbers). */ +export interface PageLocationCitation { + type: 'page_location'; + citedText: string; + documentIndex: number; + documentTitle?: string; + fileId?: string; + startPageNumber: number; + endPageNumber: number; +} + +/** Citation from a custom content document (block indices). */ +export interface ContentBlockLocationCitation { + type: 'content_block_location'; + citedText: string; + documentIndex: number; + documentTitle?: string; + fileId?: string; + startBlockIndex: number; + endBlockIndex: number; +} + +/** Union of all citation types for documents. */ +export type AnthropicCitation = + | CharLocationCitation + | PageLocationCitation + | ContentBlockLocationCitation; diff --git a/js/plugins/anthropic/tests/beta_runner_test.ts b/js/plugins/anthropic/tests/beta_runner_test.ts index 98e4226e86..a3eb5ae3c4 100644 --- a/js/plugins/anthropic/tests/beta_runner_test.ts +++ b/js/plugins/anthropic/tests/beta_runner_test.ts @@ -801,4 +801,319 @@ describe('BetaRunner', () => { const nullReason = exposed.fromBetaStopReason(null); assert.strictEqual(nullReason, 'unknown'); }); + + it('should convert citations_delta to citation part', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const citationPart = (runner as any).toGenkitPart({ + type: 'content_block_delta', + index: 0, + delta: { + type: 'citations_delta', + citation: { + type: 'char_location', + cited_text: 'The sky is blue.', + document_index: 0, + document_title: 'Facts', + start_char_index: 0, + end_char_index: 16, + }, + }, + }); + + assert.deepStrictEqual(citationPart, { + text: '', + metadata: { + citations: [ + { + type: 'char_location', + citedText: 'The sky is blue.', + documentIndex: 0, + documentTitle: 'Facts', + fileId: undefined, + startCharIndex: 0, + endCharIndex: 16, + }, + ], + }, + }); + }); + + it('should convert text block with citations', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const textPart = (runner as any).fromBetaContentBlock({ + type: 'text', + text: 'The answer is green.', + citations: [ + { + type: 'char_location', + cited_text: 'grass is green', + document_index: 0, + document_title: 'Nature', + start_char_index: 5, + end_char_index: 19, + }, + ], + }); + + assert.deepStrictEqual(textPart, { + text: 'The answer is green.', + metadata: { + citations: [ + { + type: 'char_location', + citedText: 'grass is green', + documentIndex: 0, + documentTitle: 'Nature', + fileId: undefined, + startCharIndex: 5, + endCharIndex: 19, + }, + ], + }, + }); + }); + + it('should convert page_location citations from PDF documents', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const textPart = (runner as any).fromBetaContentBlock({ + type: 'text', + text: 'According to the document...', + citations: [ + { + type: 'page_location', + cited_text: 'Important finding', + document_index: 0, + document_title: 'Report.pdf', + start_page_number: 1, + end_page_number: 2, + }, + ], + }); + + assert.deepStrictEqual(textPart, { + text: 'According to the document...', + metadata: { + citations: [ + { + type: 'page_location', + citedText: 'Important finding', + documentIndex: 0, + documentTitle: 'Report.pdf', + fileId: undefined, + startPageNumber: 1, + endPageNumber: 2, + }, + ], + }, + }); + }); + + it('should convert content_block_location citations', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const textPart = (runner as any).fromBetaContentBlock({ + type: 'text', + text: 'Based on the content...', + citations: [ + { + type: 'content_block_location', + cited_text: 'Block content', + document_index: 1, + document_title: 'Structured Doc', + start_block_index: 0, + end_block_index: 2, + }, + ], + }); + + assert.deepStrictEqual(textPart, { + text: 'Based on the content...', + metadata: { + citations: [ + { + type: 'content_block_location', + citedText: 'Block content', + documentIndex: 1, + documentTitle: 'Structured Doc', + fileId: undefined, + startBlockIndex: 0, + endBlockIndex: 2, + }, + ], + }, + }); + }); + + it('should skip web search citations (no document_index)', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const textPart = (runner as any).fromBetaContentBlock({ + type: 'text', + text: 'Search results say...', + citations: [ + { + type: 'web_search_result_location', + cited_text: 'From the web', + url: 'https://example.com', + title: 'Example Page', + // Note: no document_index - this is a web search citation + }, + ], + }); + + // Should return text without citations metadata since web search citations are filtered + assert.deepStrictEqual(textPart, { + text: 'Search results say...', + }); + }); + + it('should convert anthropicDocument custom part to beta document block', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const docBlock = (runner as any).toAnthropicMessageContent({ + custom: { + anthropicDocument: { + source: { + type: 'text', + data: 'Document content here.', + }, + title: 'My Document', + citations: { enabled: true }, + }, + }, + }); + + assert.strictEqual(docBlock.type, 'document'); + assert.strictEqual(docBlock.title, 'My Document'); + assert.deepStrictEqual(docBlock.citations, { enabled: true }); + assert.strictEqual(docBlock.source.type, 'text'); + assert.strictEqual(docBlock.source.data, 'Document content here.'); + }); + + it('should convert file source in beta document block', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const docBlock = (runner as any).toAnthropicMessageContent({ + custom: { + anthropicDocument: { + source: { + type: 'file', + fileId: 'file-abc123', + }, + title: 'Uploaded File', + }, + }, + }); + + assert.strictEqual(docBlock.type, 'document'); + assert.strictEqual(docBlock.source.type, 'file'); + assert.strictEqual(docBlock.source.file_id, 'file-abc123'); + }); + + it('should convert URL source in beta document block', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const docBlock = (runner as any).toAnthropicMessageContent({ + custom: { + anthropicDocument: { + source: { + type: 'url', + url: 'https://example.com/doc.pdf', + }, + }, + }, + }); + + assert.strictEqual(docBlock.type, 'document'); + assert.strictEqual(docBlock.source.type, 'url'); + assert.strictEqual(docBlock.source.url, 'https://example.com/doc.pdf'); + }); + + it('should convert base64 source in beta document block', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const docBlock = (runner as any).toAnthropicMessageContent({ + custom: { + anthropicDocument: { + source: { + type: 'base64', + data: 'JVBERi0xLjQ=', + mediaType: 'application/pdf', + }, + }, + }, + }); + + assert.strictEqual(docBlock.type, 'document'); + assert.strictEqual(docBlock.source.type, 'base64'); + assert.strictEqual(docBlock.source.data, 'JVBERi0xLjQ='); + assert.strictEqual(docBlock.source.media_type, 'application/pdf'); + }); + + it('should convert content source in beta document block', () => { + const mockClient = createMockAnthropicClient(); + const runner = new BetaRunner({ + name: 'claude-test', + client: mockClient as Anthropic, + }); + + const docBlock = (runner as any).toAnthropicMessageContent({ + custom: { + anthropicDocument: { + source: { + type: 'content', + content: [ + { type: 'text', text: 'First block' }, + { type: 'text', text: 'Second block' }, + ], + }, + citations: { enabled: true }, + }, + }, + }); + + assert.strictEqual(docBlock.type, 'document'); + assert.strictEqual(docBlock.source.type, 'content'); + assert.strictEqual(docBlock.source.content.length, 2); + assert.strictEqual(docBlock.source.content[0].text, 'First block'); + }); }); diff --git a/js/plugins/anthropic/tests/live_test.ts b/js/plugins/anthropic/tests/live_test.ts index 0c370196dd..631ffc81ed 100644 --- a/js/plugins/anthropic/tests/live_test.ts +++ b/js/plugins/anthropic/tests/live_test.ts @@ -24,7 +24,11 @@ import * as assert from 'assert'; import { genkit, z } from 'genkit'; import { describe, it } from 'node:test'; -import { anthropic } from '../src/index.js'; +import { + anthropic, + anthropicDocument, + type AnthropicCitation, +} from '../src/index.js'; const API_KEY = process.env.ANTHROPIC_API_KEY; @@ -126,4 +130,470 @@ describe('Live Anthropic API Tests', { skip: !API_KEY }, () => { ); } }); + + it('should return citations from a plain text document', async () => { + const ai = genkit({ + plugins: [anthropic({ apiKey: API_KEY, apiVersion: 'beta' })], + }); + + const result = await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + anthropicDocument({ + source: { + type: 'text', + data: 'The grass is green. The sky is blue. Water is wet.', + }, + title: 'Basic Facts', + citations: { enabled: true }, + }), + { text: 'What color is the grass? Cite your source.' }, + ], + }, + ], + }); + + assert.ok(result.text, 'Should have response text'); + assert.ok( + result.text.toLowerCase().includes('green'), + 'Response should mention green' + ); + + // Extract citations from response parts + const citations = + result.message?.content?.flatMap( + (part) => (part.metadata?.citations as AnthropicCitation[]) || [] + ) ?? []; + + assert.ok( + citations && citations.length > 0, + 'Should have at least one citation' + ); + + // Verify citation structure + const citation = citations[0]; + assert.strictEqual( + citation.type, + 'char_location', + 'Should be a char_location citation' + ); + assert.ok(citation.citedText, 'Citation should have cited text'); + assert.strictEqual( + citation.documentIndex, + 0, + 'Should reference first document' + ); + }); + + it('should return citations with streaming enabled', async () => { + const ai = genkit({ + plugins: [anthropic({ apiKey: API_KEY, apiVersion: 'beta' })], + }); + + const streamedChunks: string[] = []; + + const result = await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + anthropicDocument({ + source: { + type: 'text', + data: 'Cats are mammals. Dogs are also mammals. Birds have feathers.', + }, + title: 'Animal Facts', + citations: { enabled: true }, + }), + { text: 'Are cats mammals? Cite your source.' }, + ], + }, + ], + streamingCallback: (chunk) => { + if (chunk.text) { + streamedChunks.push(chunk.text); + } + }, + }); + + assert.ok(result.text, 'Should have response text'); + assert.ok( + streamedChunks.length > 0, + 'Should have received streaming chunks' + ); + + // Extract citations from final response + const citations = + result.message?.content?.flatMap( + (part) => (part.metadata?.citations as AnthropicCitation[]) || [] + ) ?? []; + + assert.ok( + citations && citations.length > 0, + 'Should have at least one citation' + ); + }); + + it('should return citations using stable API', async () => { + const ai = genkit({ + plugins: [anthropic({ apiKey: API_KEY })], // No apiVersion = stable + }); + + const result = await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + anthropicDocument({ + source: { + type: 'text', + data: 'The ocean is blue. The sun is yellow. Snow is white.', + }, + title: 'Color Facts', + citations: { enabled: true }, + }), + { text: 'What color is the ocean? Cite your source.' }, + ], + }, + ], + }); + + assert.ok(result.text, 'Should have response text'); + assert.ok( + result.text.toLowerCase().includes('blue'), + 'Response should mention blue' + ); + + // Extract citations from response parts + const citations = + result.message?.content?.flatMap( + (part) => (part.metadata?.citations as AnthropicCitation[]) || [] + ) ?? []; + + assert.ok( + citations && citations.length > 0, + 'Should have at least one citation with stable API' + ); + + // Verify citation structure + const citation = citations[0]; + assert.strictEqual( + citation.type, + 'char_location', + 'Should be a char_location citation' + ); + assert.ok(citation.citedText, 'Citation should have cited text'); + }); + + it('should return citations from a custom content document with content_block_location', async () => { + const ai = genkit({ + plugins: [anthropic({ apiKey: API_KEY, apiVersion: 'beta' })], + }); + + const result = await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + anthropicDocument({ + source: { + type: 'content', + content: [ + { type: 'text', text: 'Fact 1: Dogs are mammals.' }, + { type: 'text', text: 'Fact 2: Cats are also mammals.' }, + { type: 'text', text: 'Fact 3: Birds have feathers.' }, + { type: 'text', text: 'Fact 4: Fish live in water.' }, + ], + }, + title: 'Animal Facts', + citations: { enabled: true }, + }), + { + text: 'What do dogs and cats have in common? Cite your source with block references.', + }, + ], + }, + ], + }); + + assert.ok(result.text, 'Should have response text'); + assert.ok( + result.text.toLowerCase().includes('mammal'), + 'Response should mention mammals' + ); + + // Extract citations from response parts + const citations = + result.message?.content?.flatMap( + (part) => (part.metadata?.citations as AnthropicCitation[]) || [] + ) ?? []; + + assert.ok( + citations && citations.length > 0, + 'Should have at least one citation' + ); + + // Verify at least one citation is content_block_location type + const contentBlockCitations = citations.filter( + (c) => c.type === 'content_block_location' + ); + assert.ok( + contentBlockCitations.length > 0, + 'Should have at least one content_block_location citation' + ); + + // Verify content_block_location citation structure + const contentBlockCitation = contentBlockCitations[0]; + assert.strictEqual( + contentBlockCitation.type, + 'content_block_location', + 'Should be a content_block_location citation' + ); + assert.ok( + contentBlockCitation.citedText, + 'Citation should have cited text' + ); + assert.strictEqual( + contentBlockCitation.documentIndex, + 0, + 'Should reference first document' + ); + assert.ok( + typeof contentBlockCitation.startBlockIndex === 'number', + 'Citation should have startBlockIndex' + ); + assert.ok( + typeof contentBlockCitation.endBlockIndex === 'number', + 'Citation should have endBlockIndex' + ); + assert.ok( + contentBlockCitation.startBlockIndex >= 0, + 'startBlockIndex should be non-negative' + ); + assert.ok( + contentBlockCitation.endBlockIndex >= + contentBlockCitation.startBlockIndex, + 'endBlockIndex should be >= startBlockIndex' + ); + }); + + it('should return citations from multiple documents with correct document indexing', async () => { + const ai = genkit({ + plugins: [anthropic({ apiKey: API_KEY, apiVersion: 'beta' })], + }); + + const result = await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + anthropicDocument({ + source: { + type: 'text', + data: 'The capital of France is Paris. The capital of Germany is Berlin.', + }, + title: 'European Capitals', + citations: { enabled: true }, + }), + anthropicDocument({ + source: { + type: 'text', + data: 'The capital of Japan is Tokyo. The capital of China is Beijing.', + }, + title: 'Asian Capitals', + citations: { enabled: true }, + }), + { + text: 'What are the capitals of France and Japan? Cite your sources for each.', + }, + ], + }, + ], + }); + + assert.ok(result.text, 'Should have response text'); + assert.ok( + result.text.toLowerCase().includes('paris'), + 'Response should mention Paris' + ); + assert.ok( + result.text.toLowerCase().includes('tokyo'), + 'Response should mention Tokyo' + ); + + // Extract citations from response parts + const citations = + result.message?.content?.flatMap( + (part) => (part.metadata?.citations as AnthropicCitation[]) || [] + ) ?? []; + + assert.ok( + citations && citations.length > 0, + 'Should have at least one citation' + ); + + // Verify citations reference different documents + const documentIndices = new Set(citations.map((c) => c.documentIndex)); + assert.ok( + documentIndices.size >= 1, + 'Should have citations from at least one document' + ); + assert.ok( + documentIndices.has(0) || documentIndices.has(1), + 'Should have citations from document 0 or 1' + ); + + // Verify citation structure for each document index + for (const citation of citations) { + assert.ok( + citation.documentIndex === 0 || citation.documentIndex === 1, + `Citation documentIndex should be 0 or 1, got ${citation.documentIndex}` + ); + assert.ok(citation.citedText, 'Citation should have cited text'); + assert.strictEqual( + citation.type, + 'char_location', + 'Text document citations should be char_location type' + ); + + // Verify char_location specific fields + if (citation.type === 'char_location') { + assert.ok( + typeof citation.startCharIndex === 'number', + 'Citation should have startCharIndex' + ); + assert.ok( + typeof citation.endCharIndex === 'number', + 'Citation should have endCharIndex' + ); + assert.ok( + citation.endCharIndex >= citation.startCharIndex, + 'endCharIndex should be >= startCharIndex' + ); + } + } + + // If we have citations from both documents, verify they reference different content + if (documentIndices.size === 2) { + const doc0Citations = citations.filter((c) => c.documentIndex === 0); + const doc1Citations = citations.filter((c) => c.documentIndex === 1); + assert.ok( + doc0Citations.length > 0, + 'Should have citations from document 0' + ); + assert.ok( + doc1Citations.length > 0, + 'Should have citations from document 1' + ); + } + }); + + it('should throw descriptive error for invalid image media types in document content', async () => { + const ai = genkit({ + plugins: [anthropic({ apiKey: API_KEY, apiVersion: 'beta' })], + }); + + // Test that invalid media type throws descriptive error + await assert.rejects( + async () => { + await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + anthropicDocument({ + source: { + type: 'content', + content: [ + { + type: 'image', + source: { + type: 'base64', + mediaType: 'image/bmp', // Invalid - not supported + data: 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==', + }, + }, + ], + }, + citations: { enabled: true }, + }), + { text: 'What is in this image?' }, + ], + }, + ], + }); + }, + (error: Error) => { + assert.ok( + error.message.includes('Unsupported image media type'), + 'Error should mention unsupported media type' + ); + assert.ok( + error.message.includes('image/bmp'), + 'Error should include the invalid media type' + ); + assert.ok( + error.message.includes('image/jpeg') || + error.message.includes('image/png') || + error.message.includes('image/gif') || + error.message.includes('image/webp'), + 'Error should include supported types' + ); + return true; + } + ); + }); + + it('should handle valid image media types in document content gracefully', async () => { + const ai = genkit({ + plugins: [anthropic({ apiKey: API_KEY, apiVersion: 'beta' })], + }); + + // Test that valid media types work correctly + // Using a minimal valid PNG (1x1 transparent pixel) + const validPngBase64 = + 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=='; + + const result = await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + anthropicDocument({ + source: { + type: 'content', + content: [ + { type: 'text', text: 'This document contains an image.' }, + { + type: 'image', + source: { + type: 'base64', + mediaType: 'image/png', // Valid media type + data: validPngBase64, + }, + }, + ], + }, + title: 'Document with Image', + citations: { enabled: true }, + }), + { text: 'What does the document say?' }, + ], + }, + ], + }); + + // Should not throw and should return a response + assert.ok(result.text, 'Should have response text'); + assert.ok(result.text.length > 0, 'Response should not be empty'); + }); }); diff --git a/js/plugins/anthropic/tests/stable_runner_test.ts b/js/plugins/anthropic/tests/stable_runner_test.ts index 28e1834e2a..1caf6f4187 100644 --- a/js/plugins/anthropic/tests/stable_runner_test.ts +++ b/js/plugins/anthropic/tests/stable_runner_test.ts @@ -33,6 +33,7 @@ import type { CandidateData, ToolDefinition } from 'genkit/model'; import { describe, it, mock } from 'node:test'; import { claudeModel, claudeRunner } from '../src/models.js'; +import { toDocumentBlock } from '../src/runner/converters/stable.js'; import { Runner } from '../src/runner/stable.js'; import { AnthropicConfigSchema } from '../src/types.js'; import { @@ -724,6 +725,40 @@ describe('fromAnthropicContentBlockChunk', () => { }, expectedOutput: { reasoning: 'Step by step...' }, }, + { + should: 'should return citation part from citations_delta event', + event: { + index: 0, + type: 'content_block_delta', + delta: { + type: 'citations_delta', + citation: { + type: 'char_location', + cited_text: 'The grass is green.', + document_index: 0, + document_title: 'Basic Facts', + start_char_index: 0, + end_char_index: 19, + }, + }, + } as MessageStreamEvent, + expectedOutput: { + text: '', + metadata: { + citations: [ + { + type: 'char_location', + citedText: 'The grass is green.', + documentIndex: 0, + documentTitle: 'Basic Facts', + fileId: undefined, + startCharIndex: 0, + endCharIndex: 19, + }, + ], + }, + }, + }, { should: 'should return tool use requests', event: { @@ -914,6 +949,72 @@ describe('fromAnthropicResponse', () => { }, }, }, + { + should: 'should work with text content containing citations', + message: { + id: 'abc123', + model: 'whatever', + type: 'message', + role: 'assistant', + stop_reason: 'end_turn', + stop_sequence: null, + content: [ + { + type: 'text', + text: 'The grass is green.', + citations: [ + { + type: 'char_location', + cited_text: 'The grass is green.', + document_index: 0, + document_title: 'Basic Facts', + start_char_index: 0, + end_char_index: 19, + }, + ], + }, + ], + usage: createUsage({ + input_tokens: 10, + output_tokens: 20, + cache_creation_input_tokens: null, + cache_read_input_tokens: null, + }), + } as Message, + expectedOutput: { + candidates: [ + { + index: 0, + finishReason: 'stop', + message: { + role: 'model', + content: [ + { + text: 'The grass is green.', + metadata: { + citations: [ + { + type: 'char_location', + citedText: 'The grass is green.', + documentIndex: 0, + documentTitle: 'Basic Facts', + fileId: undefined, + startCharIndex: 0, + endCharIndex: 19, + }, + ], + }, + }, + ], + }, + }, + ], + usage: { + inputTokens: 10, + outputTokens: 20, + }, + }, + }, ]; for (const test of testCases) { @@ -2159,6 +2260,192 @@ describe('BaseRunner helper utilities', () => { }); }); +describe('toDocumentBlock (stable converter)', () => { + it('should convert text source to stable document block', () => { + const result = toDocumentBlock({ + source: { + type: 'text', + data: 'The grass is green. The sky is blue.', + }, + title: 'Basic Facts', + citations: { enabled: true }, + }); + + assert.deepStrictEqual(result, { + type: 'document', + source: { + type: 'text', + media_type: 'text/plain', + data: 'The grass is green. The sky is blue.', + }, + title: 'Basic Facts', + citations: { enabled: true }, + }); + }); + + it('should convert base64 PDF source to stable document block', () => { + const result = toDocumentBlock({ + source: { + type: 'base64', + mediaType: 'application/pdf', + data: 'JVBERi0xLjQK', + }, + title: 'Test PDF', + }); + + assert.deepStrictEqual(result, { + type: 'document', + source: { + type: 'base64', + media_type: 'application/pdf', + data: 'JVBERi0xLjQK', + }, + title: 'Test PDF', + }); + }); + + it('should convert URL source to stable document block', () => { + const result = toDocumentBlock({ + source: { + type: 'url', + url: 'https://example.com/document.pdf', + }, + context: 'This is a PDF about science.', + }); + + assert.deepStrictEqual(result, { + type: 'document', + source: { + type: 'url', + url: 'https://example.com/document.pdf', + }, + context: 'This is a PDF about science.', + }); + }); + + it('should convert content source with text blocks to stable document block', () => { + const result = toDocumentBlock({ + source: { + type: 'content', + content: [ + { type: 'text', text: 'Fact 1: Dogs are mammals.' }, + { type: 'text', text: 'Fact 2: Cats are also mammals.' }, + ], + }, + title: 'Animal Facts', + citations: { enabled: true }, + }); + + assert.deepStrictEqual(result, { + type: 'document', + source: { + type: 'content', + content: [ + { type: 'text', text: 'Fact 1: Dogs are mammals.' }, + { type: 'text', text: 'Fact 2: Cats are also mammals.' }, + ], + }, + title: 'Animal Facts', + citations: { enabled: true }, + }); + }); + + it('should convert content source with images to stable document block', () => { + const result = toDocumentBlock({ + source: { + type: 'content', + content: [ + { type: 'text', text: 'A picture of a cat:' }, + { + type: 'image', + source: { + type: 'base64', + mediaType: 'image/png', + data: 'iVBORw0KGgo=', + }, + }, + ], + }, + }); + + assert.deepStrictEqual(result, { + type: 'document', + source: { + type: 'content', + content: [ + { type: 'text', text: 'A picture of a cat:' }, + { + type: 'image', + source: { + type: 'base64', + media_type: 'image/png', + data: 'iVBORw0KGgo=', + }, + }, + ], + }, + }); + }); + + it('should throw error for file source on stable API', () => { + assert.throws( + () => + toDocumentBlock({ + source: { + type: 'file', + fileId: 'file-abc123', + }, + title: 'Uploaded Document', + }), + /File-based document sources require the beta API/ + ); + }); + + it('should handle anthropicDocument custom part via toAnthropicMessageContent', () => { + // Test that the stable runner correctly converts anthropicDocument custom parts + const result = testRunner.toAnthropicMessageContent({ + custom: { + anthropicDocument: { + source: { + type: 'text', + data: 'Hello world.', + }, + title: 'Test', + citations: { enabled: true }, + }, + }, + }); + + assert.deepStrictEqual(result, { + type: 'document', + source: { + type: 'text', + media_type: 'text/plain', + data: 'Hello world.', + }, + title: 'Test', + citations: { enabled: true }, + }); + }); + + it('should throw for file source via toAnthropicMessageContent on stable API', () => { + assert.throws( + () => + testRunner.toAnthropicMessageContent({ + custom: { + anthropicDocument: { + source: { + type: 'file', + fileId: 'file-abc123', + }, + }, + }, + }), + /File-based document sources require the beta API/ + ); + }); +}); + describe('Runner request bodies and error branches', () => { it('should include optional config fields in non-streaming request body', () => { const mockClient = createMockAnthropicClient(); diff --git a/js/testapps/anthropic/package.json b/js/testapps/anthropic/package.json index 0f7ac15001..7d7aff6664 100644 --- a/js/testapps/anthropic/package.json +++ b/js/testapps/anthropic/package.json @@ -14,6 +14,7 @@ "dev:beta:files-api": "genkit start -- npx tsx --watch src/beta/files_api.ts", "dev:beta:effort": "genkit start -- npx tsx --watch src/beta/effort.ts", "dev:beta:additional-params": "genkit start -- npx tsx --watch src/beta/additional_params.ts", + "dev:beta:citations": "genkit start -- npx tsx --watch src/beta/citations.ts", "dev:stable:text-plain": "genkit start -- npx tsx --watch src/stable/text-plain.ts", "dev:stable:webp": "genkit start -- npx tsx --watch src/stable/webp.ts", "dev:stable:pdf": "genkit start -- npx tsx --watch src/stable/pdf.ts", diff --git a/js/testapps/anthropic/src/beta/citations.ts b/js/testapps/anthropic/src/beta/citations.ts new file mode 100644 index 0000000000..dea9938e5a --- /dev/null +++ b/js/testapps/anthropic/src/beta/citations.ts @@ -0,0 +1,115 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { anthropic, anthropicDocument } from '@genkit-ai/anthropic'; +import { genkit } from 'genkit'; + +const ai = genkit({ + plugins: [ + anthropic({ + apiVersion: 'beta', + apiKey: process.env.ANTHROPIC_API_KEY, + }), + ], +}); + +/** + * This flow demonstrates citations with a plain text document. + * The response will include citations pointing back to the source text. + */ +ai.defineFlow('citations-text', async () => { + const response = await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + anthropicDocument({ + source: { + type: 'text', + data: 'The grass is green. The sky is blue. Water is wet. Fire is hot.', + }, + title: 'Basic Facts', + citations: { enabled: true }, + }), + { + text: 'What color is the grass and sky? Please cite your sources.', + }, + ], + }, + ], + }); + + // Log the response with citations + console.log('Response text:', response.text); + console.log( + 'Response content:', + JSON.stringify(response.message?.content, null, 2) + ); + + // Extract citations from the response + const citations = + response.message?.content?.flatMap( + (part) => part.metadata?.citations || [] + ) ?? []; + + console.log('Citations:', JSON.stringify(citations, null, 2)); + + return { + text: response.text, + citations, + }; +}); + +/** + * This flow demonstrates citations with custom content blocks + * for more granular citation control. + */ +ai.defineFlow('citations-custom-content', async () => { + const response = await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + anthropicDocument({ + source: { + type: 'content', + content: [ + { type: 'text', text: 'Fact 1: Dogs are mammals.' }, + { type: 'text', text: 'Fact 2: Cats are also mammals.' }, + { type: 'text', text: 'Fact 3: Birds have feathers.' }, + ], + }, + title: 'Animal Facts', + citations: { enabled: true }, + }), + { + text: 'What do dogs and cats have in common? Cite your source.', + }, + ], + }, + ], + }); + + console.log('Response text:', response.text); + console.log( + 'Response content:', + JSON.stringify(response.message?.content, null, 2) + ); + + return response.text; +}); diff --git a/js/testapps/anthropic/src/beta/files_api.ts b/js/testapps/anthropic/src/beta/files_api.ts index 3dd6b08d01..d74f96a007 100644 --- a/js/testapps/anthropic/src/beta/files_api.ts +++ b/js/testapps/anthropic/src/beta/files_api.ts @@ -24,6 +24,10 @@ const API_KEY = process.env.ANTHROPIC_API_KEY; // If you have a file ID, you can set it here. Otherwise, the flow will upload a new PDF to Anthropic. const FILE_ID = process.env.ANTHROPIC_FILE_ID; +/** + * Uploads a PDF file to Anthropic's Files API. + * @returns The file object containing `id` field. + */ export async function uploadPdfToAnthropic() { if (!API_KEY) throw new Error('Missing ANTHROPIC_API_KEY env variable'); @@ -34,7 +38,7 @@ export async function uploadPdfToAnthropic() { const form = new FormData(); form.append( 'file', - new Blob([fileBuffer], { type: 'application/pdf' }), + new Blob([new Uint8Array(fileBuffer)], { type: 'application/pdf' }), 'attention-first-page.pdf' ); @@ -53,62 +57,53 @@ export async function uploadPdfToAnthropic() { throw new Error(`Anthropic file upload failed: ${response.status} ${text}`); } const result = await response.json(); - return result as { id: string }; // Contains 'file_id', etc. + return result as { id: string }; } -async function main() { - const ai = genkit({ - plugins: [ - // Default all flows in this sample to the beta surface - anthropic({ - apiVersion: 'beta', - apiKey: API_KEY, - }), - ], - }); +const ai = genkit({ + plugins: [ + // Default all flows in this sample to the beta surface + anthropic({ + apiVersion: 'beta', + apiKey: API_KEY, + }), + ], +}); - /** - * This flow demonstrates PDF document processing via a public data URL along with a user prompt. - * The PDF is sent as a media part with the correct contentType and a URL, not base64. - */ - ai.defineFlow('beta-pdf-url', async () => { - let fileId = FILE_ID; +/** + * This flow demonstrates PDF document processing via the Anthropic Files API. + * The PDF is uploaded to Anthropic and referenced by file ID. + */ +ai.defineFlow('beta-pdf-url', async () => { + let fileId = FILE_ID; - if (!fileId) { - const fileResult = await uploadPdfToAnthropic(); - if (!fileResult || !fileResult.id) { - throw new Error('File ID not found'); - } - fileId = fileResult.id; + if (!fileId) { + const fileResult = await uploadPdfToAnthropic(); + if (!fileResult || !fileResult.id) { + throw new Error('File ID not found'); } + fileId = fileResult.id; + } - // Example: Use a (demo/test) PDF file accessible via public URL. - // Replace this with your actual PDF if needed. - const { text } = await ai.generate({ - model: anthropic.model('claude-sonnet-4-5'), - messages: [ - { - role: 'user', - content: [ - { - text: 'What are the key findings or main points in this document?', - }, - { - media: { - url: fileId, - contentType: 'anthropic/file', - }, + const { text } = await ai.generate({ + model: anthropic.model('claude-sonnet-4-5'), + messages: [ + { + role: 'user', + content: [ + { + text: 'What are the key findings or main points in this document?', + }, + { + media: { + url: fileId, + contentType: 'anthropic/file', }, - ], - }, - ], - }); - - return text; + }, + ], + }, + ], }); -} -main().catch((error) => { - console.error('Error:', error); - process.exit(1); + return text; });