Commit dbededa
Anai Guo
fix(llama): release a vision chat handler's mtmd context on close (#2342)
`Llama.close()` tore down the model and its context but left the chat
handler's `_exit_stack` untouched, so the mtmd/clip context it built from
that model was never freed.
Handlers routinely outlive the `Llama` that initialized them -- callers
construct one handler and reuse it across loads -- and
`_init_mtmd_context()` returns early while `mtmd_ctx` is set. After the
first `close()` the handler therefore kept a context bound to an
already-freed model and handed it back on the next load, where it
surfaces as a null `mtmd_ctx` on the C++ side.
Register the handler's exit stack on the Llama's `_stack`. It unwinds
LIFO, so the mtmd context is released ahead of the model teardown
registered earlier in `__init__`, and `mtmd_free` resets `mtmd_ctx` to
`None` so the next load re-initializes cleanly. Duck-typed on
`_exit_stack`, which both `Llava15ChatHandler` and `MTMDChatHandler`
(and every subclass) expose.1 parent 3bda091 commit dbededa
1 file changed
Lines changed: 10 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
456 | 456 | | |
457 | 457 | | |
458 | 458 | | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
459 | 469 | | |
460 | 470 | | |
461 | 471 | | |
| |||
0 commit comments