Skip to content

Fix max_tokens parameter for GPT-5 models with Azure BYOK routing#436

Closed
saschabuehrle wants to merge 1 commit intoOpenRouterTeam:mainfrom
saschabuehrle:fix-gpt5-azure-max-tokens
Closed

Fix max_tokens parameter for GPT-5 models with Azure BYOK routing#436
saschabuehrle wants to merge 1 commit intoOpenRouterTeam:mainfrom
saschabuehrle:fix-gpt5-azure-max-tokens

Conversation

@saschabuehrle
Copy link
Copy Markdown

This PR fixes issue #435 where GPT-5 models fail when routing to Azure BYOK due to the deprecated max_tokens parameter.

Problem

When using GPT-5 models (e.g., openai/gpt-5.2-chat) with Azure BYOK routing (provider: { only: ['azure'] }), requests fail with:

{
  "error": {
    "message": "Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.",
    "type": "invalid_request_error",
    "param": "max_tokens",
    "code": "unsupported_parameter"
  }
}

This happens because Azure's GPT-5 endpoint has deprecated max_tokens in favor of max_completion_tokens.

Solution

This fix automatically detects when:

  1. The model is a GPT-5 variant (model ID contains 'gpt-5')
  2. Provider routing is set to only use Azure (provider.only includes 'azure')

When both conditions are met, it uses max_completion_tokens instead of max_tokens.

Changes

  • Modified getArgs() method in src/chat/index.ts to conditionally use the correct token parameter
  • Added comprehensive unit tests to verify the fix works correctly
  • Maintains full backward compatibility for all other scenarios

Testing

  • ✅ All existing tests pass
  • ✅ New unit tests verify correct behavior for all scenarios:
    • GPT-5 + Azure only → uses max_completion_tokens
    • GPT-5 + general routing → uses max_tokens
    • Non-GPT-5 + Azure only → uses max_tokens
    • Various GPT-5 model name patterns

Validation

Tested with a reproduction script that confirms the fix resolves the original issue while maintaining compatibility.

Fixes #435

Greetings,
saschabuehrle

When routing to Azure BYOK with GPT-5 models, OpenRouter should translate
max_tokens to max_completion_tokens because Azure's GPT-5 endpoint has
deprecated max_tokens.

This fix:
- Detects GPT-5 model names (containing 'gpt-5')
- Checks if provider routing is set to only use Azure
- Uses max_completion_tokens instead of max_tokens when both conditions are met
- Maintains backward compatibility for all other scenarios

Fixes issue where GPT-5 requests fail with 'max_tokens' is not supported
error when using Azure BYOK routing.
@damjankuznar
Copy link
Copy Markdown

Thanks for the effort @saschabuehrle, but this fix belongs in the router (server-side), not in the SDK.

The SDK sends requests to the OpenRouter API, which already accepts both max_tokens and max_completion_tokens. The translation to the correct upstream parameter should happen in the router when it forwards the request to Azure. Fixing it in the SDK only works when the user explicitly sets provider.only: ['azure'] — it wouldn't help when Azure is selected by the router's load balancing or other routing logic.

We've applied the fix server-side in the AzureOpenAIAdapter, so GPT-5 models on Azure BYOK will now correctly receive max_completion_tokens. Closing this PR in favor of the server-side fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[gpt-5.*] 'max_tokens' is not supported with this model, when using azure BYOK

2 participants