Add ModelExpress to CUDA image#2621
Conversation
|
@codex review |
|
Codex Review: Didn't find any major issues. 🎉 ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ca96dcb. Configure here.
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 944ec179f3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc1954bab3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. Bravo. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Summary
Adds ModelExpress support to the standard CUDA image build path.
modelexpressoptional extra pinned tomodelexpress==0.3.0Dockerfile.cudaduring the existing lockeduv syncprotobuf<6uv.lockwithmodelexpressandprotobuf==5.29.6/opt/ucxso NIXL can use a CUDA-capable UCX backend instead of distrolibucx0Notes
This keeps the hosted-rl image workflow unchanged: it can build this
prime-rlbranch into the normalghcr.io/primeintellect-ai/hosted-rl/prime-rl:<tag>image.The protobuf override is needed because the current env stack pulls
prime-sandboxes, whose published metadata currently resolves protobuf 6.x. ModelExpress generated protobuf code is built for protobuf 5.x.The first Telus pilot image reached vLLM with
load_format: mxbut failed because NIXL could not load UCX (libucp.so.0missing). Adding distrolibucx0let NIXL discover UCX, but exposed that the distro UCX is 1.18.1 and lacks CUDA support. This branch now builds UCX 1.19.1 from source with CUDA and verbs support.Validation
uv lock --checkgit diff --checkNo local Docker image build was run.
Note
Medium Risk
Touches container dependency resolution (protobuf overrides) and adds a non-trivial UCX source build plus RDMA libs, which affects runtime networking/GPU transfer paths for NIXL-backed inference.
Overview
Adds ModelExpress to the CUDA container build: a new
modelexpressoptional extra (modelexpress==0.3.0), included in the lockeduv syncinDockerfile.cuda, withuv.lockupdated accordingly.Resolves a protobuf version clash by overriding ModelExpress’s published metadata so resolution stays on a compatible 5.x–6.x band while the rest of the stack (e.g.
prime-sandboxes) can still pull newer protobuf.Ships a CUDA-capable UCX 1.19.1 built in the builder stage (verbs + CUDA, installed under
/opt/ucx) and copies it into the runtime image withUCX_HOMEandLD_LIBRARY_PATH, plus RDMA-related build/runtime packages—so NIXL/ModelExpress can load UCX with GPU support instead of relying on distro UCX that lacked CUDA.Reviewed by Cursor Bugbot for commit 3fddc64. Bugbot is set up for automated code reviews on this repo. Configure here.