Skip to content

Add LLM model registry with OpenRouter auto-generation#1033

Closed
aureliensibiril wants to merge 12 commits intogetprobo:mainfrom
aureliensibiril:aureliensibiril/model-registry
Closed

Add LLM model registry with OpenRouter auto-generation#1033
aureliensibiril wants to merge 12 commits intogetprobo:mainfrom
aureliensibiril:aureliensibiril/model-registry

Conversation

@aureliensibiril
Copy link
Copy Markdown
Contributor

@aureliensibiril aureliensibiril commented Apr 13, 2026

Summary

  • Add a typed model registry (pkg/llm) so the agent framework knows what parameters each model supports — foundation for config validation
  • go generate ./pkg/llm/ fetches model data from OpenRouter (free, no API key) and writes registry_gen.go with 177 models across 9 providers
  • Registry.Lookup() accepts canonical IDs (anthropic/claude-opus-4.6), bare names (claude-opus-4.6), and normalized forms (claude-opus-4-6)
  • Generated file is committed — no API call needed at build time

Test plan

  • go test ./pkg/llm/ -race passes (9 tests: lookup variants, capability assertions, edge cases)
  • go build ./pkg/llm/ and go build ./internal/cmd/genmodels/ compile cleanly
  • go vet clean
  • gofmt clean

Summary by cubic

Adds a typed LLM model registry with OpenRouter-based auto-generation and a committed dataset for offline builds. Lookups are fast and accept canonical, bare, and normalized IDs.

  • New Features

    • Added pkg/llm registry with ModelDefinition, SupportedParameters, and Registry.Lookup(); NewRegistry() allows custom registries; DefaultRegistry() caches generated data.
    • Introduced internal/cmd/genmodels; added //go:generate in pkg/llm and updated make genmodels to run go generate ./pkg/llm to fetch OpenRouter models (no API key) and write pkg/llm/registry_gen.go with explicit true/false flags; covers 177 models across 9 providers; no build-time API calls; clean removes registry_gen.go.
  • Refactors

    • Generated registry now uses map[string]ModelDefinition for O(1) lookups and ID uniqueness.
    • Removed stored provider field; added ModelDefinition.Provider() to derive the provider from the model ID.
    • Tests use controlled inline model data instead of the generated dataset to avoid flakiness.

Written for commit 7aaff30. Summary will update on new commits.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="internal/cmd/genmodels/main.go">

<violation number="1" location="internal/cmd/genmodels/main.go:165">
P2: Generator can overwrite `registry_gen.go` with an empty registry because it never validates that any models were retained before writing.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

ModelDefinition, SupportedParameters, and Registry types with
multi-key lookup supporting canonical, bare, and normalized
model IDs. NewRegistry constructor accepts model definitions
for testability; DefaultRegistry caches the generated data.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Fetches model metadata from the OpenRouter API and generates
pkg/llm/registry_gen.go with typed ModelDefinition entries.
Covers 9 providers: Anthropic, OpenAI, Google, xAI, Perplexity,
Amazon, Meta, Mistral, DeepSeek.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
177 models across 9 providers fetched from OpenRouter.
This file is generated by go generate and committed to avoid
requiring an API call at build time.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Lookup by canonical ID, bare name, and normalized form.
Capability assertions for Claude, GPT-5.4, and o3. Edge
cases for empty string and provider-prefix-only inputs.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Always write both true and false values in the generated
registry so unsupported parameters are visible at a glance
instead of relying on Go zero-value defaults.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
@aureliensibiril aureliensibiril force-pushed the aureliensibiril/model-registry branch from 6af0c03 to 248601b Compare April 13, 2026 14:33
Replaces []ModelDefinition with map[string]ModelDefinition so
model ID uniqueness is enforced by the data structure itself
and lookups do not require a linear scan.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Build registries from inline map literals instead of relying
on generated OpenRouter data that can change between runs.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
The model registry fetches from an external API so it should
not run as part of the regular generate/build pipeline. Replace
the automatic generate dependency with a standalone genmodels
target.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
The provider is always the prefix before "/" in the model ID,
so storing it as a separate field is redundant. Replace the
field with a Provider() method.

Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
Signed-off-by: Aurélien Sibiril <81782+aureliensibiril@users.noreply.github.com>
@SachaProbo
Copy link
Copy Markdown
Contributor

Merged

@SachaProbo SachaProbo closed this Apr 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants