Skip to content

A document longer than 150 windows is read only as far as the 150th #232

Description

@lemarier

A document longer than 150 windows is read only as far as the 150th. One extract message may ask for no more (MOST_WINDOWS in apps/worker/src/work.ts): three messages to a queue batch, each window a model call and an R2 write, must stay under the thousand subrequests a Worker invocation may make.

Since #231 such a reading says so, with unread set to the windows it left, and the figures pull keeps what the document gave before rather than taking it back. So nothing is lost from the records, but nothing past window 150 is read either.

Evidence

Four of wave 2's documents are over the cap once read from their pages:

Maker Document Windows Unread
Magnum MPXD extension kit manual 204 54
Energizer Solar Peak 3000 user manual 174 24
Energizer Solar Homepower installation manual 172 22
Victron Off-grid system booklet 164 14

Two of them, the Magnum and the Homepower manuals, are long because of #230 and should fall under the cap once that is fixed. The Peak 3000 manual and the Victron booklet are long in their own right.

Done when

A document past the cap is read to its end, split across messages that each stay inside the subrequest budget, and the reading is written once when the last part is in. Or, if that is not worth it for four documents, this is closed saying so.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions