Fix max_tokens parameter for GPT-5 models with Azure BYOK routing#436
Closed
saschabuehrle wants to merge 1 commit intoOpenRouterTeam:mainfrom
Closed
Fix max_tokens parameter for GPT-5 models with Azure BYOK routing#436saschabuehrle wants to merge 1 commit intoOpenRouterTeam:mainfrom
saschabuehrle wants to merge 1 commit intoOpenRouterTeam:mainfrom
Conversation
When routing to Azure BYOK with GPT-5 models, OpenRouter should translate max_tokens to max_completion_tokens because Azure's GPT-5 endpoint has deprecated max_tokens. This fix: - Detects GPT-5 model names (containing 'gpt-5') - Checks if provider routing is set to only use Azure - Uses max_completion_tokens instead of max_tokens when both conditions are met - Maintains backward compatibility for all other scenarios Fixes issue where GPT-5 requests fail with 'max_tokens' is not supported error when using Azure BYOK routing.
|
Thanks for the effort @saschabuehrle, but this fix belongs in the router (server-side), not in the SDK. The SDK sends requests to the OpenRouter API, which already accepts both We've applied the fix server-side in the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes issue #435 where GPT-5 models fail when routing to Azure BYOK due to the deprecated
max_tokensparameter.Problem
When using GPT-5 models (e.g.,
openai/gpt-5.2-chat) with Azure BYOK routing (provider: { only: ['azure'] }), requests fail with:This happens because Azure's GPT-5 endpoint has deprecated
max_tokensin favor ofmax_completion_tokens.Solution
This fix automatically detects when:
provider.onlyincludes 'azure')When both conditions are met, it uses
max_completion_tokensinstead ofmax_tokens.Changes
getArgs()method insrc/chat/index.tsto conditionally use the correct token parameterTesting
max_completion_tokensmax_tokensmax_tokensValidation
Tested with a reproduction script that confirms the fix resolves the original issue while maintaining compatibility.
Fixes #435
Greetings,
saschabuehrle