Skip to content

Implemented generic multimodal chat handler.#125

Open
alcoftTAO wants to merge 2 commits intoJamePeng:mainfrom
TAO71-AI:mtmd
Open

Implemented generic multimodal chat handler.#125
alcoftTAO wants to merge 2 commits intoJamePeng:mainfrom
TAO71-AI:mtmd

Conversation

@alcoftTAO
Copy link
Copy Markdown

@alcoftTAO alcoftTAO commented May 4, 2026

  • Implemented a generic/global multimodal chat handler.

What does it do?

It automatically uses the model's chat template and replaces all of the model's multimodal tags with the media_marker tag.

This allows a much easier implementation for multimodal models, since the chat template doesn't need to be hard-coded for each model.

How to use it?

It is as simple as passing the clip_model_path parameter to the Llama class when created.

Note

Using the previous implementation (e.g. Qwen35ChatHandler) still works.

I'm also looking forward to implement more model architectures. Please, reply if you want me to implement any.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant