Can SpecForge train and export draft models for vLLM? #303
Unanswered
chenaoxuan
asked this question in
Q&A
Replies: 2 comments
-
|
Yep, kinda! You can train in SpecForge, but exporting straight to vLLM might need a bit of extra work. Not fully plug_and_play yet |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Seems that the draft model trained by specforge with sglang backend lead to lower acceptance length in vllm. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
If I train a model, can I easily export it for use with vLLM?
Beta Was this translation helpful? Give feedback.
All reactions