Description
poll_messages hardcodes Consumer::default() (foreign/python/src/client.rs:979), so the caller cannot name the consumer. That id resolves to numeric 0, so every Python process polling the same topic/partition shares one server-side offset slot and two independent consumers silently split the stream instead of each receiving it. It also makes consumer groups unreachable on this path, since ConsumerKind::ConsumerGroup cannot be selected at all.
Affected area / component
Python SDK
Proposed solution
Add a Consumer type to the binding, modelled as a complex enum like the other sum types there (PollingStrategy, AutoCommit), with the id accepting str | int:
pub enum Consumer {
Single { id: PyIdentifier },
Group { id: PyIdentifier },
}Make consumer a required argument on poll_messages, and make partition_id optional in the same change: under the vsr feature a consumer-group poll only resolves the member's assignment when no partition is sent, and an omitted partition reads partition 0 for a regular consumer.
await client.poll_messages(
stream="s", topic="t", partition_id=0,
polling_strategy=PollingStrategy.Next(), count=10, auto_commit=True,
consumer=Consumer.Single("my-app"),
)
await client.poll_messages(
stream="s", topic="t",
polling_strategy=PollingStrategy.Next(), count=10, auto_commit=True,
consumer=Consumer.Group("my-group"),
)Note: Requiring consumer breaks existing callers.
Alternatives considered
Keep consumer optional and default it to Consumer::new(Identifier::numeric(1)), matching the serde default the HTTP API already applies. That stays backwards compatible, but it silently moves an existing next() user's stored offset from slot 0 to slot 1 on upgrade, and Python would still be the only SDK picking a consumer for the caller.
To align with other SDKs, which all require a consumer, I lean toward the breaking option: it fails at the call site instead of quietly relocating an offset.
Contribution
Good first issue
Description
poll_messageshardcodesConsumer::default()(foreign/python/src/client.rs:979), so the caller cannot name the consumer. That id resolves to numeric 0, so every Python process polling the same topic/partition shares one server-side offset slot and two independent consumers silently split the stream instead of each receiving it. It also makes consumer groups unreachable on this path, sinceConsumerKind::ConsumerGroupcannot be selected at all.Affected area / component
Python SDK
Proposed solution
Add a
Consumertype to the binding, modelled as a complex enum like the other sum types there (PollingStrategy,AutoCommit), with the id acceptingstr | int:Make
consumera required argument onpoll_messages, and makepartition_idoptional in the same change: under the vsr feature a consumer-group poll only resolves the member's assignment when no partition is sent, and an omitted partition reads partition 0 for a regular consumer.Note: Requiring
consumerbreaks existing callers.Alternatives considered
Keep
consumeroptional and default it toConsumer::new(Identifier::numeric(1)), matching the serde default the HTTP API already applies. That stays backwards compatible, but it silently moves an existingnext()user's stored offset from slot 0 to slot 1 on upgrade, and Python would still be the only SDK picking a consumer for the caller.To align with other SDKs, which all require a consumer, I lean toward the breaking option: it fails at the call site instead of quietly relocating an offset.
Contribution
Good first issue