Skip to content

Pass renderer chat template kwargs from sampling#1447

Open
eligotts wants to merge 1 commit into
mainfrom
feat/chat-template-kwargs-renderers
Open

Pass renderer chat template kwargs from sampling#1447
eligotts wants to merge 1 commit into
mainfrom
feat/chat-template-kwargs-renderers

Conversation

@eligotts
Copy link
Copy Markdown
Contributor

@eligotts eligotts commented May 23, 2026

Summary

  • Teach RendererClient to consume sampling_args.extra_body.chat_template_kwargs.
  • Remove those kwargs from the /generate sampling params and pass them into renderer pool construction instead.
  • Include the kwargs in the shared renderer-pool cache key so different template settings do not collide.

We are extracting renderer template controls from chat_template_kwargs in sampling extra_body, which matches the hosted config path. Longer term, this should move toward typed pydantic config models with explicit renderer-specific template args; for now, renderers validate accepted kwargs and alert on unsupported keys.

Companion PRs

Validation

  • uv run ruff check verifiers/clients/renderer_client.py tests/test_renderer_client.py
  • PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 uv run pytest tests/test_renderer_client.py -k "renderer_client_honors_configured_renderer_name or renderer_client_uses_renderer_model_name_override or renderer_client_consumes_sampling_chat_template_kwargs"

Note

Pass chat_template_kwargs from sampling args to renderer pool in RendererClient

  • RendererClient.get_native_response now extracts chat_template_kwargs from extra_body in sampling args and passes them to _get_renderer_or_pool, rather than forwarding them downstream in sampling_params.
  • Distinct chat_template_kwargs values produce separate renderer pool cache entries, using a deterministic frozen representation as the cache key.
  • A _pop_chat_template_kwargs helper removes the key from sampling_params and raises ValueError if the value is present but not a mapping.

Macroscope summarized 3443fdf.

@macroscopeapp
Copy link
Copy Markdown

macroscopeapp Bot commented May 23, 2026

Approvability

Verdict: Needs human review

This change propagates chat_template_kwargs from sampling parameters to renderer pool creation, enabling new configuration options that affect how renderers process requests. Since this introduces a new runtime behavior path for template kwargs (like enable_thinking), human review is recommended.

You can customize Macroscope's approvability policy. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants