Qwen3.5 27B

[etemiz]

Tried mergekit-extract-lora and got this:

WARNING:mergekit.architecture:No JSON architecture found for Qwen3_5ForCausalLM

Any plans to support this beast?

[jeff4700]

Try to install transformers==4.57.6.

[etemiz]

Did that. Now I have another error:

transformers/models/auto/configuration_auto.py", line 1362, in from_pretrained
    raise ValueError(
ValueError: The checkpoint you are trying to load has model type `qwen3_5_text` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`

[David-AU-github]

You need Transformers 5.2.0 ; 4.57.6 does not have the new Qwen 3.5 in it.

The other issue : need a JSON template for the ARCH (Qwen3.5) in mergekit so mergekit knows the tensors and what to do with them. This is basically a tensor map ; which must also include "image" tensors and other goodies.

[etemiz]

qwen3_5_text.json

I just vibed this file. Worked for me. (Text only)

[jeff4700]

Hi @etemiz, that's great. I also informed the team to support new models. Currently it doesn't support Qwen3, Qwen3.5 models.