Skip to content

pdf-codec: render a page (or region) to pixels, for OCR and vision consumers #1198

Description

@fcbwilliams

What we need

A way to get pixels out of a PDF page (or a sub-region of one), so a caller can hand them to OCR or a vision model. Not the model call itself — see "what we are not asking for" below.

readPdf gives us positioned items and images-read.ts gives us embedded XObjects properly decoded, which covers a great deal. Two cases it cannot reach, and both are common in real client material:

1. A scanned document has no text layer to read

Measured on 2026-09-10, on five real product brochures rasterised to image-only PDFs with gs -sDEVICE=pdfimage24 -r150 (which is what a scanned document looks like to a parser): 102 pages, every one empty, 0 characters of text. The embedded XObject per page is extractable, so for a straightforwardly scanned page a caller can pull the page image out and OCR it — that part works today.

2. A chart drawn as vectors is not an image at all

This is the case with no workaround. The same five brochures, unrasterised, carry 291 image draw operations and 2,030 path operations across 102 pages. A chart exported from Excel or Illustrator into a PDF frequently arrives as ContentVector rects, lines and paths plus positioned text runs — there is no XObject to extract, the labels come out in an order that reads as scrambled, and no amount of text extraction recovers "revenue rose in EMEA and fell in APAC".

Rendering the region those vectors occupy is the only way to read it. That is true of any consumer of this library that wants to interpret a vector figure, not just us.

Shape we'd suggest

Something like renderPdfPage(pdfBytes, pageIndex, { scale | dpi, clipPt? }) returning PNG bytes, where clipPt is an optional region in the same point coordinates LayoutFrame already uses — so a caller that has located a figure region (via document-outline.js's segmentPdfRegions, which already classifies items into column/table/figure and attaches captions) can render just that figure rather than a whole page.

Two constraints we'd flag from our own environment, since they may shape where this lives:

Weight. pdf-codec is already ~13 MB unpacked, and the README is explicit that the root barrel's font payload is most of a Cloudflare Worker's 3 MB gzipped budget. A rasteriser is not small. The existing pdf-codec/read narrow entry with its build-failing static-graph test is exactly the right pattern — but rasterisation might warrant its own package (pdf-raster?) so that consumers who only extract never pay for it.

Runtime. This is the hard part and the reason we are asking rather than doing it ourselves. Our target is Cloudflare Workers, where there is no canvas and no OffscreenCanvas. The options we can see are a WASM rasteriser, or Cloudflare's Browser Rendering binding (which is per-session metered and slow), or keeping rasterisation client-side where the browser's canvas is free. Your existing workerd test suite makes you far better placed than us to judge which is viable — and if the answer is "Node and browser only, not workerd", saying so explicitly in the API's docs is itself useful, because it tells us where to put the call.

What we are NOT asking for

No OCR, no vision, no network calls in the codec. Keep that property — it is one of the best things about these packages, and we do not want a document library that reaches the internet. Hand us pixels and we will call the model ourselves; the results come back into the content model via the interpretation annotation requested separately.

Priority

Lower than the interpretation-model issue for us. We can extract embedded XObjects today with what pdf-codec already has, which covers scanned pages; the vector-figure case is the one with no path forward at all, and it is the one we will come back to.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions