The set checks when using hub kernels with GGUF are not traceable with torch.compile.
|
DEQUANT_TYPES = STANDARD_QUANT_TYPES | KQUANT_TYPES | IMATRIX_QUANT_TYPES |
|
MMVQ_QUANT_TYPES = STANDARD_QUANT_TYPES | KQUANT_TYPES | IMATRIX_QUANT_TYPES |
|
MMQ_QUANT_TYPES = STANDARD_QUANT_TYPES | KQUANT_TYPES |
The GGUF Kernels also need to be updated to support the latest version of torch.
Initial Solution has been proposed: #12556
The set checks when using hub kernels with GGUF are not traceable with torch.compile.
diffusers/src/diffusers/quantizers/gguf/utils.py
Lines 74 to 76 in b0dc51d
The GGUF Kernels also need to be updated to support the latest version of torch.
Initial Solution has been proposed: #12556