input: separate input-method keyboard interception from routing grabs

[mikairyuu]

It's a pain to use libreoffice on systems with IME on COSMIC right now. GTK popups do not open at all if IME is running. It's been quite infuriating, so I'm pretty desperate to fix the situation.

Summary

Input-method v2 currently implements grab_keyboard as a Smithay KeyboardGrab. This makes it mutually exclusive with compositor routing grabs, such as popup grabs: installing one replaces the other.

This PR moves the input-method keyboard grab to a post-routing input interceptor instead. Routing grabs still decide where keyboard input goes, while the input method can intercept physical keyboard events after that decision.

This allows popup and input-method grabs to coexist without changing KeyboardGrab semantics.

Changes

• add a crate-internal post-routing keyboard input interceptor
• move zwp_input_method_v2.grab_keyboard to that interceptor
• keep KeyboardHandle::is_grabbed() describing Smithay routing grabs only
• preserve press/release ownership across interceptor changes and multiple keyboard sources
• clean up the interceptor when the input-method or keyboard-grab resource is destroyed
• keep the new infrastructure behind wayland_frontend
• add regression tests for grab ordering, focus changes, modifier synchronization, lifecycle cleanup, and press/release pairing

Issues

This addresses the overlapping popup/IME keyboard-grab limitation described in niri-wm/niri#3899, which can cause popups to stop working while an IME keyboard grab is active.

Testing

I have confirmed libreoffice popups work correctly with a running IME after the change. They also grab keyboard input and after closing return it properly.

Downstream note

Compositors that used KeyboardHandle::is_grabbed() to detect an input-method protocol grab should use InputMethodHandle::keyboard_grabbed() for that purpose instead.

In particular, I've found cosmic-comp's XWayland eavesdropping path to be requiring this, and this will need a companion PR.

niri already distinguishes the input-method protocol grab through InputMethodHandle::keyboard_grabbed(), but has a workaround that avoids installing popup keyboard grabs while an IME grab is active. That workaround can be removed once using this change.

AI disclosure

The code has been generated by ChatGPT, since Rust isn't my forte. But it's been thoroughly read, understood, and architecturally improved by me quite a few times before arriving here.

Checklist

• I agree to smithay's Developer Certificate of Origin.

[Drakulix]

Input-method v2 currently implements grab_keyboard as a Smithay KeyboardGrab. This makes it mutually exclusive with compositor routing grabs, such as popup grabs: installing one replaces the other.

Yes this is the same bug as #1845 trying to solve, just another symptom of it.

I really don't want to fix this by special-casing either IMEs or Popup-Grabs. Clearly the Grab-logic is insufficient and we need nested grabs.

A solution that correctly (imo) addresses this issue would refactor the whole grab logic to allow multiple grabs at once.

(Sorry, I am aware this is frustrating and you probably put some work into this.)

[Ph4ntomas]

I really don't want to fix this by special-casing either IMEs or Popup-Grabs. Clearly the Grab-logic is insufficient and we need nested grabs.

A solution that correctly (imo) addresses this issue would refactor the whole grab logic to allow multiple grabs at once.

(Sorry, I am aware this is frustrating and you probably put some work into this.)

So I've been thinking about that since I've worked on the touch refactor & tablet stuff. I'm not sure if we want multiple grabs, or if some kind of grab nesting would be preferred.

Its already possible to have some kind of nesting using with_grab but:

• you now need to probe the grab to know if it's grab-ed
• you need to implement interior mutability and upcast the grab because the API will only call your callback dyn *Grab

I wonder if we should add a with_grab_mut and a hook function to grabs so that the handles' has_grab defers to the grab itself to check the serial, instead of the last grab.

The reason I think nesting is better is mainly for touch input, since the outer grab can act as an event router once sub grab are bound to specific touch slots, and it's easier to reimplement multiple grabs via nested grabs than the other way around

[mikairyuu]

(Sorry, I am aware this is frustrating and you probably put some work into this.)

It's fine, I only hope to get this fixed somehow. It's a long-standing issue and it hasn't really moved forward over the last year despite multiple attempts.

Thanks, I see how 1845 is relevant.
There are a couple of things we might want to note here.

1. I've looked at weston/sway/kwin implementations. They all handle IME specially rather than through generic nested grab composition.

2. Even more importantly, there's an experimental xx-keyboard-filter-v1 in Wayland protocols that is aimed at IMEs specifically, and xx-input-method-v2 has ditched grab_keyboard completely.
   So the protocol design itself is also moving towards separating keyboard filtering from the input-method protocol.

So basically we have this choice, and, if we still intend to go the nested way, a choice between nested approaches - the composite grab approach of Ph4ntomas or a core-managed stack/chain approach.

What approach is the way to go?

[Ph4ntomas]

So basically we have this choice, and, if we still intend to go the nested way, a choice between nested approaches - the composite grab approach of Ph4ntomas or a core-managed stack/chain approach.

FWIW the two aren't necessarily mutually exclusive. Nested/composite grabs are flexible enough for Smithay to provide pre-made combinators.
OTOH it will require some amount of rewrite of the current API, as we don't each sub-grab to forward events individually since that would cause a bunch of duplicated events.
The good thing about that is we can re-design the API around nesting/composition, which would likely avoid the need for using with_grab and and upcast to get back to the proper type. For example, the API could assume grabs are nested by default, with non-nesting grabs simply returning a failure if the caller try to insert a new sub-grab. Alternatively, grabs could be split into container that deal with the nesting logic and may forward the input to the target, and hooks grabs, that performs other side-effects and may decide to mark the events as captured (which let the container know that it should stop propagating the event to other grab, or to the event target)

[Drakulix]

The reason I think nesting is better is mainly for touch input, since the outer grab can act as an event router once sub grab are bound to specific touch slots, and it's easier to reimplement multiple grabs via nested grabs than the other way around.

My main fear with this approach is the combination complexity.

So popup-grabs can nest a DnD grab, can they also nest an IME grab?
Or will an IME grab always be up top? What happens then if you invoke an IME inside a text field on a popup?
Do we need to list every type that is supported for nesting? If we allow arbitrary nesting are there combinations that break certain grabs if stacked in one order vs. the other?

I've looked at weston/sway/kwin implementations. They all handle IME specially rather than through generic nested grab composition.

Do you happen to know how they handle popup grabs? Given they conflict with various other things like IME or DnD? I assume the popup + dnd works on them as well.

So basically we have this choice, and, if we still intend to go the nested way, a choice between nested approaches - the composite grab approach of Ph4ntomas or a core-managed stack/chain approach.

I am not sure just because other compositors differ in how they do this necessarily implies much for smithay. Smithay's job is to built abstractions. They can be cumbersome at times causing bugs like this one, but this is also what makes smithay a powerful tool. Of the three you listed only weston even has a library and wlroots is much more opinionated in it's apis that smithay.

OTOH it will require some amount of rewrite of the current API, as we don't each sub-grab to forward events individually since that would cause a bunch of duplicated events.

Yes I fully expect that. Imo somebody just needs to sit down trying to come up with a design solving all of these issues. Preferably also building some clear test-clients for this specific behavior.

[Ph4ntomas]

My main fear with this approach is the combination complexity.

Yeah I fully agree on that. OTOH, apart from some basic use-case that Smithay could provide, most of the complexity would be on the compositor side.

For my example with touch input, it's only something I thought about because I'd like users to be able to scroll several window at the same time at some point, with each having its own set of TouchSlot. Regular "desktop-first" compositor don't really need to bother with that, so it's not necessarily something Smithay should bother with aside maybe providing a nicer interface.

Do we need to list every type that is supported for nesting? If we allow arbitrary nesting are there combinations that break certain grabs if stacked in one order vs. the other?

Assuming all grabs share the same trait, I think it would be best for some of these to be leaf only (they always refuse child grabs). It might be better to have a clear split between container and leaf grabs, but it just moves the issue one layer down.

Stacks / Queue would be easiest to deal with as only the 'top' grab would be called, but indeed the stacking/queuing order would matter.

Once we start dealing with several nested grab with each having their own side effect and/or policy wrt event propagation (to other grab or target) is where the complexity really appear.

Yes I fully expect that. Imo somebody just needs to sit down trying to come up with a design solving all of these issues. Preferably also building some clear test-clients for this specific behavior.

Agreed. It would help to have a clear feedback somewhere for the various cases we'd like to solve (IME, Popup/DnD, true multitouch support)