Skip to content

Generator/iterator request bodies are never consumed, even after a successful mocked call - #840

Open
juneja-varun wants to merge 2 commits into
getsentry:masterfrom
juneja-varun:fix-generator-body-not-consumed
Open

juneja-varun wants to merge 2 commits into
getsentry:masterfrom
juneja-varun:fix-generator-body-not-consumed

Conversation

@juneja-varun

Copy link
Copy Markdown

Fixes #713.

I upload large files in chunks via a generator (while f.tell() < file_size: requests.post(url, data=chunk_file(f, ...))), and under responses this loop never terminates - the file position never advances, because the generator passed as data= is never actually read. A real network send would consume it chunk by chunk while writing to the socket; responses just stores the generator object as-is in request.body and moves on.

This is the same class of gap as #719/#736 (file-like objects with .read() weren't being consumed either, fixed by reading them in _read_filelike_body) - just for the other kind of streaming body requests supports. Extended that same helper to drain a generic iterator/generator into bytes too.

(The original issue's title points at multipart_matcher, but that's not actually the site of the bug - files= uploads get fully read by requests itself during prepare_body(), before responses ever sees them. The real gap is in the general streamed-data= path, which is what the fix targets.)

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit da32f59. Configure here.

Comment thread responses/__init__.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

responses multipart_matcher does not consume the passed iterator; how to mock?

1 participant