[Feature] Implement InternVL to llama.cpp

[James4Ever0]

Motivation

Many llama.cpp users are requesting this so far. Ollama is one of the interfaces of llama.cpp and it is quite popular. Implementing it will significantly accelerate InterVL adoption and recognition.

Related resources

ggml-org/llama.cpp#6803

Additional context

InternVL is based on LLaMA architecture. Currently text-only InternLM models have been ported to Ollama but not for multimodal ones.

[G-z-w]

Thank you for your suggestions. We will gradually push support for various frameworks, and we also welcome contributions from the community.

[James4Ever0]

Hey @czczup would you please clarify the reason closing this issue?

[sammcj]

Any update on internVL support with llama.cpp?

[cloudyuyuyu]

Any update on internVL support with llama.cpp?

[G-z-w]

Thank you for your attention. We are actively progressing on this work, and we also welcome contributions from the community.

[rampageservices]

Just curious why this issue was closed if you are actively progressing on this work?

[rampageservices]

Thanks for reopening this issue.

…

On Mon, Sep 30, 2024 at 2:54 PM Zhe Chen ***@***.***> wrote: Reopened #522 <#522>. — Reply to this email directly, view it on GitHub <#522 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE32PCTXAB3YAMM2F3GTQLLZZDYT7AVCNFSM6AAAAABM2CFCKSVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJUGQ2TAMRXGIZDAMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[cloudyuyuyu]

any update?

[Cartomex-MX]

November 12, any update? thank in advance

[BleedingDev]

Is there anything that could be helped with? :) InternVL2.5 is really important for the future. :)

[linxhome]

pay close attention to this, any update? pls

[James4Ever0]

Is it really that hard to do this? I suggest myself to implement it.

Current progress: ggml-org/llama.cpp#9403

Also ipex-llm has support for InternVL2.