perf(pdf): bound OCR upload and rendered-page memory (0.27.13)

[CyMule]

Summary

OCR-only PDF partitioning kept ten decoded page images in memory at once, and the OCR step copied the whole upload into one byte string for the entire OCR run. This PR renders one page at a time, copies uploads to temporary disk in 1 MiB blocks, and closes each rendered page image after OCR, including when OCR fails. Pixels, format, rotation metadata, page order, upload cursor behavior and caller-owned stream lifetime are preserved. Uploads may be any readable, seekable stream (duck-typed read/seek), not only BytesIO, SpooledTemporaryFile or buffered files.

Proposed library release: 0.27.13, together with the other pending library memory changes.

Validation

• New ownership, bounded-read, generator cancellation, OCR cleanup, native render parity and generic-stream tests pass locally; the original 13 also passed in the named SND runtime.
• Full PDF/OCR/utilities and new regression suite: 320 passed, 1 skipped locally with workspace Korean OCR data.
• Actual SND OCR-only partitioning of rotated and content-bearing multi-page fixtures: all 20 elements match baseline JSON hashes exactly.
• Actual SND 20-page, 1200-point square OCR-only PDF: baseline OOMKilled at 2 GiB; candidate succeeds at the same limit. Healthy baseline at 4 GiB peaks at 3,292 MiB RSS versus 954 MiB for candidate. Runtime is 94.06 versus 93.89 seconds and output hashes match.
• Repository Ruff checks/format and the actual Linux version-sync checker pass.

Operational impact

OCR rendering reads uploads from temporary disk instead of holding a full upload-sized copy in memory. Spooled uploads (SpooledTemporaryFile) are still copied in full, briefly, by spooled_to_bytes_io_if_needed for the text-extraction check before the OCR branch; the saving is that this copy is not held while OCR runs. There is no new upload size cap. Working render memory is bounded by one page; extracted output and memory retained by callers still scale with content. Public render-generator images remain usable after iteration and are owned by the caller.

Follow-up

_pdf_source_path and convert_to_bytes (#4419) both decide how to read a given upload type. Once #4419 lands, both should use one shared helper in unstructured/partition/common/common.py that copies a stream to disk in blocks.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[cubic-dev-ai]

All reported issues were addressed across 7 files

<sub>Shadow auto-approve: would not auto-approve because issues were found.

Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 1 file (changes from recent commits).

<sub>Shadow auto-approve: would not auto-approve. This PR does not meet the repository auto-approval settings.

Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 3 files (changes from recent commits).

<sub>Shadow auto-approve: would not auto-approve. This PR does not meet the repository auto-approval settings.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 3 files (changes from recent commits).

<sub>Shadow auto-approve: would not auto-approve because issues were found.
Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 3 files (changes from recent commits).

<sub>Shadow auto-approve: would not auto-approve. Auto-approval blocked by 1 unresolved issue from previous reviews.

Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 2 files (changes from recent commits).

<sub>Shadow auto-approve: would not auto-approve. This PR does not meet the repository auto-approval settings.

Re-trigger cubic</sub>

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[cubic-dev-ai]

No issues found across 7 files

<sub>Shadow auto-approve: would not auto-approve. This PR does not meet the repository auto-approval settings.

Re-trigger cubic</sub>