Implementation of "Fast Inference from Transformers via Speculative Decoding". Feel free to copy & modify this code for your needs.
GPTlike and T5 models are supported.
AIshutin/speculative-decoding
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Implementation of "Fast Inference from Transformers via Speculative Decoding". Feel free to copy & modify this code for your needs.
GPTlike and T5 models are supported.