Pass renderer chat template kwargs from sampling#1447
Open
eligotts wants to merge 1 commit into
Open
Conversation
This was referenced May 23, 2026
ApprovabilityVerdict: Needs human review This change propagates chat_template_kwargs from sampling parameters to renderer pool creation, enabling new configuration options that affect how renderers process requests. Since this introduces a new runtime behavior path for template kwargs (like enable_thinking), human review is recommended. You can customize Macroscope's approvability policy. Learn more. |
willccbb
approved these changes
May 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RendererClientto consumesampling_args.extra_body.chat_template_kwargs./generatesampling params and pass them into renderer pool construction instead.We are extracting renderer template controls from
chat_template_kwargsin samplingextra_body, which matches the hosted config path. Longer term, this should move toward typed pydantic config models with explicit renderer-specific template args; for now, renderers validate accepted kwargs and alert on unsupported keys.Companion PRs
Validation
uv run ruff check verifiers/clients/renderer_client.py tests/test_renderer_client.pyPYTEST_DISABLE_PLUGIN_AUTOLOAD=1 uv run pytest tests/test_renderer_client.py -k "renderer_client_honors_configured_renderer_name or renderer_client_uses_renderer_model_name_override or renderer_client_consumes_sampling_chat_template_kwargs"Note
Pass
chat_template_kwargsfrom sampling args to renderer pool inRendererClientRendererClient.get_native_responsenow extractschat_template_kwargsfromextra_bodyin sampling args and passes them to_get_renderer_or_pool, rather than forwarding them downstream insampling_params.chat_template_kwargsvalues produce separate renderer pool cache entries, using a deterministic frozen representation as the cache key._pop_chat_template_kwargshelper removes the key fromsampling_paramsand raisesValueErrorif the value is present but not a mapping.Macroscope summarized 3443fdf.