Skip to content

Upload Files in threads, not processes; test on Python 3.14 - #58

Merged
tomasr78 merged 3 commits into
mainfrom
fix-parallel-uploads
Oct 20, 2026
Merged

tomasr78 merged 3 commits into
mainfrom
fix-parallel-uploads

Conversation

@tomasr78

Copy link
Copy Markdown
Contributor

Bug: convertapi.convert(..., {'Files': ['a.docx', 'b.docx']}) with local files fails with TypeError: can only concatenate str (not "NoneType") to str on macOS, and on Linux with Python 3.14+. The parallel uploads ran in a multiprocessing.Pool. Under the spawn start method (macOS default) and forkserver (Linux default since 3.14), each child process re-imports convertapi, so it loses api_credentials and anything else the caller configured. Linux on 3.13 and older forks, which is why CI never saw it. I reproduced it against staging on Python 3.14/macOS.

Fix: utils.map_in_parallel now uses a ThreadPoolExecutor. Uploads are network-bound, so threads fit, and they share the caller's configuration. max_parallel_uploads keeps its meaning.

Tests: a new mocked test uploads two local files with max_parallel_uploads 1 and 10 and checks that every upload carries the token. It fails on the old code and passes now. All 12 live tests pass against stag-v2. Python 3.14 is added to the CI matrix and the classifiers.

CHANGELOG: under 2.0.1 / Fixed.

🤖 Generated with Claude Code

Parallel uploads ran in a multiprocessing.Pool. Under the spawn (macOS) and forkserver (Linux,
Python 3.14+) start methods the child processes re-import convertapi and lose api_credentials, so
convert(..., {'Files': [local paths]}) failed with a TypeError. A ThreadPoolExecutor keeps the
caller's configuration and suits network-bound uploads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 20, 2026 07:53
…does not enforce

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused implementation preserves configuration, ordering, and upload limits, with regression coverage and no blocking issues identified.

No open issues 🎉

What changed in this PR

Fixes multi-file uploads losing caller configuration under multiprocessing by using threads instead.

Changes:

  • Replaces process-based uploads with threads while preserving ordering and concurrency limits.
  • Adds regression coverage for authenticated uploads with one and multiple workers.
  • Adds Python 3.14 to CI, package classifiers, and documentation.
File Description
tests/​test_client.py Tests authentication on multiple local-file uploads.
RELEASING.md Updates the documented CI Python range.
README.md Updates requirements and tested Python versions.
pyproject.toml Adds the Python 3.14 classifier.
convertapi/​utils.py Uses threads and returns materialized results.
CHANGELOG.md Documents the upload fix and Python 3.14 testing.
.github/​workflows/​build.yml Adds Python 3.14 to the test matrix.

Review effort: Balanced

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 20, 2026 07:55
@tomasr78
tomasr78 merged commit d653b60 into main Oct 20, 2026
7 checks passed
@tomasr78
tomasr78 deleted the fix-parallel-uploads branch October 20, 2026 07:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Concurrent reuse of an uninitialized UploadIO can consume the same stream twice, producing an empty or incomplete upload.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Medium severity Concurrent uploads race when the same uninitialized UploadIO is reused

convertapi/​utils.py:10

With max_parallel_uploads >= 2, passing the same uninitialized UploadIO twice in Files lets both workers see _result is None and upload its shared stream concurrently. A synchronized upload stub reproduces one complete body and one empty body, so conversion can fail or receive incomplete input. Synchronize lazy initialization per UploadIO, or resolve repeated references once before dispatch, and add a regression test confirming both entries reuse the same uploaded file ID.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants