Add Qwen3-vl Interactive Reference Code and Dataset#309
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
Victor49152
added
priority: P1
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new 8k sample variant of the Shopify product catalogue dataset, including documentation, example configurations, and comprehensive unit tests. The implementation refactors the existing Shopify dataset into a base class to support multiple variants. Additionally, the PR adds a tokenizer_name override to ModelParams to handle local model paths, improves the robustness of OpenAI response parsing by providing default values for optional fields, and updates the load generator to support multimodal prompt formats. Feedback suggests using None instead of magic values like 0 or "" for default values in the OpenAI response schema to improve consistency and data representation.
Victor49152
force-pushed
the
feat/qwen3_vl_interactive_ref
branch
from
b885cd4 to
ffa7b64
Compare
Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com>
Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com>
Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com>
Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com>
Victor49152
force-pushed
the
feat/qwen3_vl_interactive_ref
branch
from
ffa7b64 to
11ec3d6
Compare
Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com>
Victor49152
force-pushed
the
feat/qwen3_vl_interactive_ref
branch
from
b162b1d to
2221e2f
Compare
Victor49152
changed the title
Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com>
wangshangsam
left a comment
wangshangsam
left a comment
There was a problem hiding this comment.
Some nits, but LGTM overall!
|
|
||
| model_params: | ||
| name: "Qwen/Qwen3-VL-235B-A22B-Instruct" | ||
| # tokenizer_name: "Qwen/Qwen3-VL-235B-A22B-Instruct" # Set this if model name is a local/container path |
There was a problem hiding this comment.
| # tokenizer_name: "Qwen/Qwen3-VL-235B-A22B-Instruct" # Set this if model name is a local/container path |
This is no longer needed. Local checkpoint directory (with tokenizer files in the directory) can now run with the latest revision of @BolinSNLHM 's PR (already merged in).
There was a problem hiding this comment.
Yeah, this is missed. Should be deleted.
| @@ -14,6 +14,13 @@ model_params: | |||
| datasets: | |||
There was a problem hiding this comment.
Officially, were we calling this scenario "online" or "server"? (I vaguely recall that it's called "server")
If so, I would suggest to rename this file as server_qwen3_... for the sake of consistency.
There was a problem hiding this comment.
I was thinking about this naming, too. Since in regular mlperf, it's called server. But I saw other endpoints examples calling it online. Let me change to server since we are submitting to regular MLPERF anyway
2 participants
What does this PR do?
Type of change
Related issues
Testing
Checklist