Add native support for multiple LLM providers (Anthropic, Google, etc.)

[jhs]

Summary

Add native support for the top LangChain-supported LLM providers beyond OpenAI. Currently Chatfield requires openai: prefix and only uses ChatOpenAI, forcing users to rely on proxy solutions (LiteLLM, OpenRouter) for other providers.

Motivation

• LangChain already provides official integrations for major providers
• Users should be able to use Anthropic Claude, Google Gemini, etc. directly
• Simpler developer experience than setting up proxy infrastructure
• Better alignment with LangChain's multi-provider philosophy

Proposed LLM Support

Support the top 5 LangChain-supported providers:

1. OpenAI - openai:gpt-4o (existing)
2. Anthropic - anthropic:claude-3-5-sonnet-20241022
3. Google - google-genai:gemini-2.0-flash-exp
4. Azure OpenAI - azure-openai:gpt-4o
5. Groq - groq:llama-3.3-70b-versatile

Implementation Plan

1. Provider Detection and Instantiation

Update Interviewer.__init__() to detect provider prefix and instantiate appropriate chat model:

Python (interviewer.py):

from langchain_openai import ChatOpenAI, AzureChatOpenAI
from langchain_anthropic import ChatAnthropic
from langchain_google_genai import ChatGoogleGenerativeAI
from langchain_groq import ChatGroq

# Parse provider and model from llm_id
if ':' not in llm_id:
    raise ValueError(f'LLM ID must have format "provider:model", got {llm_id!r}')

provider, model_name = llm_id.split(':', 1)

# Instantiate appropriate chat model
if provider == 'openai':
    self.llm = ChatOpenAI(
        model=model_name,
        temperature=temperature,
        base_url=base_url,
        api_key=api_key,
    )
elif provider == 'anthropic':
    self.llm = ChatAnthropic(
        model=model_name,
        temperature=temperature,
        base_url=base_url,
        api_key=api_key,
    )
elif provider == 'google-genai':
    self.llm = ChatGoogleGenerativeAI(
        model=model_name,
        temperature=temperature,
        api_key=api_key,
    )
elif provider == 'azure-openai':
    self.llm = AzureChatOpenAI(
        model=model_name,
        temperature=temperature,
        azure_endpoint=base_url,
        api_key=api_key,
    )
elif provider == 'groq':
    self.llm = ChatGroq(
        model=model_name,
        temperature=temperature,
        base_url=base_url,
        api_key=api_key,
    )
else:
    raise ValueError(
        f'Unsupported LLM provider: {provider!r}. '
        f'Supported providers: openai, anthropic, google-genai, azure-openai, groq'
    )

TypeScript (interviewer.ts):

import { ChatOpenAI, AzureChatOpenAI } from '@langchain/openai'
import { ChatAnthropic } from '@langchain/anthropic'
import { ChatGoogleGenerativeAI } from '@langchain/google-genai'
import { ChatGroq } from '@langchain/groq'

// Parse provider and model from llmId
if (!llmId.includes(':')) {
  throw new Error(`LLM ID must have format "provider:model", got ${llmId}`)
}

const [provider, modelName] = llmId.split(':', 2)

// Instantiate appropriate chat model
if (provider === 'openai') {
  this.llm = new ChatOpenAI({
    modelName: modelName,
    temperature: temperature,
    apiKey: apiKey,
    configuration: { baseURL: baseUrl }
  })
} else if (provider === 'anthropic') {
  this.llm = new ChatAnthropic({
    model: modelName,
    temperature: temperature,
    apiKey: apiKey,
    clientOptions: { baseURL: baseUrl }
  })
} else if (provider === 'google-genai') {
  this.llm = new ChatGoogleGenerativeAI({
    model: modelName,
    temperature: temperature,
    apiKey: apiKey,
  })
} else if (provider === 'azure-openai') {
  this.llm = new AzureChatOpenAI({
    model: modelName,
    temperature: temperature,
    azureOpenAIApiKey: apiKey,
    azureOpenAIEndpoint: baseUrl,
  })
} else if (provider === 'groq') {
  this.llm = new ChatGroq({
    model: modelName,
    temperature: temperature,
    apiKey: apiKey,
  })
} else {
  throw new Error(
    `Unsupported LLM provider: ${provider}. ` +
    `Supported providers: openai, anthropic, google-genai, azure-openai, groq`
  )
}

2. Environment Variable Support

Add automatic API key detection per provider:

# Python
def get_api_key_for_provider(provider: str, explicit_key: Optional[str]) -> Optional[str]:
    if explicit_key:
        return explicit_key
    
    env_vars = {
        'openai': 'OPENAI_API_KEY',
        'anthropic': 'ANTHROPIC_API_KEY',
        'google-genai': 'GOOGLE_API_KEY',
        'azure-openai': 'AZURE_OPENAI_API_KEY',
        'groq': 'GROQ_API_KEY',
    }
    
    env_var = env_vars.get(provider)
    return os.environ.get(env_var) if env_var else None

3. Update Security Detection

Extend DANGEROUS_ENDPOINTS list:

DANGEROUS_ENDPOINTS = [
    'api.openai.com',
    'api.anthropic.com',
    'generativelanguage.googleapis.com',
    'groq.com',
]

4. Dependencies

Python (pyproject.toml):

dependencies = [
    "langchain-openai>=0.3.29",
    "langchain-anthropic>=0.3.4",
    "langchain-google-genai>=2.1.10",
    "langchain-groq>=0.2.5",
]

TypeScript (package.json):

{
  "dependencies": {
    "@langchain/openai": "^0.6.9",
    "@langchain/anthropic": "^0.4.7",
    "@langchain/google-genai": "^0.2.13",
    "@langchain/groq": "^0.2.11"
  }
}

Usage Examples

# Python
from chatfield import Interviewer, chatfield

interview = chatfield().field("name").build()

# Anthropic Claude
interviewer = Interviewer(interview, llm_id='anthropic:claude-3-5-sonnet-20241022')

# Google Gemini
interviewer = Interviewer(interview, llm_id='google-genai:gemini-2.0-flash-exp')

# Groq
interviewer = Interviewer(interview, llm_id='groq:llama-3.3-70b-versatile')

// TypeScript
import { Interviewer, chatfield } from '@chatfield/core'

const interview = chatfield().field('name').build()

// Anthropic Claude
const interviewer = new Interviewer(interview, { 
  llmId: 'anthropic:claude-3-5-sonnet-20241022' 
})

// Google Gemini
const interviewer = new Interviewer(interview, { 
  llmId: 'google-genai:gemini-2.0-flash-exp' 
})

// Groq
const interviewer = new Interviewer(interview, { 
  llmId: 'groq:llama-3.3-70b-versatile' 
})

Testing Strategy

1. Add provider-specific test fixtures with mock LLMs
2. Test provider detection and instantiation logic
3. Test environment variable fallback for each provider
4. Test error messages for unsupported providers
5. Add integration tests behind requires_api_key markers for each provider

Documentation Updates

• Update Api_Configuration.md with all provider examples
• Update README with multi-provider support
• Add troubleshooting guide for provider-specific issues
• Document model name formats per provider

Related Issues

• Add OpenRouter support for flexible LLM backend selection #40 (OpenRouter support - complementary gateway approach)
• Add open-source model support via LiteLLM and local hosting #58 (LiteLLM support - complementary proxy approach)

[jhs]

Additional Implementation Notes

Based on detailed analysis, here are some implementation considerations to add to the plan:

1. Provider-Aware Security Logic

The endpoint security detection should be provider-aware rather than blanket blocking all provider endpoints:

def _detect_dangerous_endpoint(self, llm, provider, mode):
    """Only flag endpoints as dangerous when using a DIFFERENT provider."""
    
    # Allow api.anthropic.com when using Anthropic provider
    if hostname == 'api.anthropic.com' and provider != 'anthropic':
        return on_dangerous_endpoint(...)
    
    # Allow api.openai.com when using OpenAI/Azure-OpenAI
    if hostname == 'api.openai.com' and provider not in ['openai', 'azure-openai']:
        return on_dangerous_endpoint(...)

This allows native provider endpoints when appropriate while maintaining security.

2. Temperature Defaults Per Provider

Different providers have different temperature defaults that should be respected:

def _get_default_temperature(self, provider):
    """Provider-specific temperature defaults."""
    defaults = {
        'openai': 0.0,
        'anthropic': 1.0,      # Anthropic defaults higher
        'google-genai': 0.9,   # Google also defaults higher
        'groq': 0.0,
        'azure-openai': 0.0,
    }
    return defaults.get(provider, 0.0)

3. Factory Pattern for Cleaner Code

Suggest using factory pattern with separate initialization methods per provider:

def _initialize_llm(self, llm_id, temperature, base_url, api_key):
    provider, model = llm_id.split(':', 1)
    
    initializers = {
        'openai': self._initialize_openai,
        'anthropic': self._initialize_anthropic,
        'google-genai': self._initialize_google_genai,
        'azure-openai': self._initialize_azure_openai,
        'groq': self._initialize_groq,
    }
    
    if provider not in initializers:
        raise ValueError(f'Unsupported provider: {provider}')
    
    return initializers[provider](model, temperature, base_url, api_key)

def _initialize_anthropic(self, model, temperature, base_url, api_key):
    """Anthropic-specific initialization logic."""
    # Handle Anthropic-specific config, defaults, etc.
    return ChatAnthropic(...)

This keeps the main __init__ cleaner and makes provider-specific logic easier to maintain.

4. Isomorphic Development Requirements

Critical for this project's dual-implementation approach:

• Identical "Isomorphic:" comments in both Python and TypeScript for any language-specific differences
• Exact test count parity - same number of tests with word-for-word identical descriptions
• Zero skipped tests - use no-op passing tests instead of .skip()
• Matching file structure - same method names (accounting for snake_case vs camelCase)

5. Technical Risks to Watch For

Provider-specific differences that may cause issues:

1. Streaming responses - Different providers handle streaming differently
2. Tool/function calling schemas - Not all providers support identical tool schemas
3. Context window sizes - Vary significantly (e.g., GPT-4: 128k, Claude: 200k, Gemini: 2M)
4. Token counting methods - Each provider counts tokens differently
5. Rate limiting behavior - Different retry strategies needed
6. Model name formats - Version strings differ per provider

6. Scope Estimation

• Code changes: ~200-300 lines across both implementations
• Documentation: ~100-150 lines
• Tests: ~100-200 lines (isomorphic)
• Files modified: ~15-20 files
• New dependencies: 8 packages (4 per language)

7. Testing Strategy Enhancement

Beyond the testing strategy already outlined:

• Test provider detection with invalid formats (no-colon, too:many:colons)
• Test environment variable precedence (explicit key > env var)
• Test provider-specific temperature defaults
• Test error messages for unsupported providers are clear and actionable
• Mock each provider's LLM for unit tests (avoid API calls)
• Integration tests behind requires_api_key marker per provider

8. Documentation Updates

In addition to Api_Configuration.md:

• Update all CLAUDE.md files (root, Python/, TypeScript/)
• Add provider comparison table showing capabilities
• Document model naming conventions per provider
• Add troubleshooting section for common provider-specific errors
• Update README badges/feature list to highlight multi-provider support