Skip to content

Openaichat#437

Open
zichengorange wants to merge 2 commits intoGradientHQ:mainfrom
zichengorange:openaichat
Open

Openaichat#437
zichengorange wants to merge 2 commits intoGradientHQ:mainfrom
zichengorange:openaichat

Conversation

@zichengorange
Copy link
Copy Markdown

📋 PR Title Format

feat(api): OpenAI advanced sampling params (logprobs, logit_bias, penalties)

📝 Change Type

Please select the type of change this PR introduces (choose one or more):

  • [✅ ] feat: New feature.
  • fix: Bug fix.
  • docs: Documentation only changes.
  • refactor: A code change that neither fixes a bug nor adds a feature.
  • perf: Performance improvement.
  • test: Adding missing tests or correcting existing tests.
  • chore: Maintenance tasks (e.g., updating dependencies).

💡 Description

  • logprobs / top_logprobs: 在采样后返回选中 token 的 logprob 及 top-k logprobs,并通过 HTTP API 以 OpenAI 兼容格式返回(含流式)。
  • logit_bias: 支持对指定 token_id 加减 bias;bias=-100 时屏蔽该 token;超出 vocab 的 token_id 会忽略并打 Warning。
  • presence_penalty / frequency_penalty: 在 softmax 前按历史 token 应用惩罚,用向量化实现。
  • 容错: logprobs 计算异常时降级为不返回 logprobs,主线生成不受影响;API 层对异常 logprobs 做防御性处理。
  • 同步修改 SamplingParams、forward.proto、executor、sampler、http_server,并增加 E2E 测试与本地验证文档。

✅ Checklist

Please ensure the following points are addressed before merging:

  • [✅ ] I have performed a self-review of my own code.
  • [✅ ] I have added/updated tests that prove my fix or feature works (if applicable).
  • I have updated the documentation (if necessary).
  • My code follows the project's style guidelines.

cheng added 2 commits March 10, 2026 01:39
…penalties) onto upstream main

Made-with: Cursor
@zichengorange zichengorange requested a review from a team March 9, 2026 18:14
Copy link
Copy Markdown

@JiwaniZakir JiwaniZakir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HasField check on top_logprobs in proto_to_sampling_params (message_util.py) is likely broken: in protobuf3, HasField only works on message-type fields and oneof members — calling it on a scalar like top_logprobs (presumably an int32) will raise ValueError at runtime, so the lambda fallback lambda _: False silently swallows this and top_logprobs will never be populated. The correct approach for a scalar proto3 field is to check proto.top_logprobs != 0 or add it unconditionally with a sentinel default, depending on whether 0 is a valid value.

Additionally, docs/README.md contains a hardcoded personal path (/Users/zicheng/Downloads/parallax-main) throughout the "quick copy-paste" examples — this will be confusing or misleading for any other contributor and should use a placeholder like <project-root> or rely on the cd "$(dirname "$0")/.." pattern already used in compile_proto.sh.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants