Skip to content

History, an API key to read it with, and a version that is true - #26

Merged
cubehouse merged 3 commits into
mainfrom
feat/history
Sep 23, 2026
Merged

cubehouse merged 3 commits into
mainfrom
feat/history

Conversation

@cubehouse

@cubehouse cubehouse commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

What this adds

tp.entity(id).history, reached the same way as .live() and .schedule:

with ThemeParks(api_key=KEY) as tp:
    history = tp.entity(DISNEYLAND).history

    span = history.span()
    for entity_id, row in history.days(span.archive_from, span.retrievable_through):
        ...
Method Yields
span() archive_from, recorded_to, retrievable_through
days(start, end) (entity id, row), one summary row per park-local day
changes(date) (entity id, row), every recorded observation

days() and changes() follow the server's paging links to the end and yield
as they go, so a resort's five years never has to be in memory at once.

Why it is more than the generated models

Three things the spec is honest about but does not do for you.

span() flattens two shapes into one. A park's coverage document nests
the dates under summary; an entity's carries them at the top level under
different names. Every caller would write that branch before their first
question. retrievable_through is also the end date a backfill should use:
it is what the key may read, not what the archive holds, and those differ on
every plan below the top one. Bounding by the wrong one ends a long run in
403s.

Park id means park call. Both history endpoints answer every entity in a
park in one request. The same data fetched ride by ride is around a hundred
times more calls against the same budget. Pass a park id and you get the cheap
path without having to know the expensive one exists.

The budget is hourly. A spent one can be most of an hour from resetting,
and sleeping through that is indistinguishable from a hung process. Past
max_wait (120s by default) this raises BudgetExhaustedError, a
RateLimitError carrying retry_after, so a backfill can checkpoint and come
back instead of blocking.

examples/backfill.py does that end to end, with NDJSON or CSV output and
resume. It pulled Disneyland Resort's whole daily archive, 98,452 rows from
2021-07-03, in one run; both formats and the resume path were run against
production before this was raised.

Two pre-existing defects fixed on the way

The client could not send an API key. There was no api_key parameter
anywhere and the transport sent only user-agent and accept, so every
request this SDK made was anonymous whatever the caller had paid for. Both
clients now take api_key and send x-api-key. An empty string is treated as
no key: an unset environment variable arrives as "" far more often than as
None, and sending an empty key is a 401 rather than an anonymous request.

The generator's nullable patcher matched only classes without a docstring.
Its pattern could not cross the blank line that follows one, so every
documented class went unpatched behind a warning that nobody read. That
included next on all four history envelopes, which is null on the last page
of every paged response, so this SDK would have failed to parse the page that
ends a backfill. The pattern now spans blank lines, an unmatched patch is a
hard failure instead of a warning, and all 22 fields the spec marks both
required and nullable are listed.

Docs

  • README.md — a History section, and api_key in the client options table.
  • docs/api/history.md, in the reference nav.
  • Cookbook recipes 6-8: backfill to NDJSON, resume when the budget runs out,
    read a single day's changes.
  • CHANGELOG.md, version 3.1.0.

Verification

  • 172 unit tests pass, 20 of them new in tests/unit/test_history.py:
    paging follows the server's URL verbatim, park envelopes flatten to the same
    row stream as entity ones, BudgetExhaustedError fires on a long wait but
    not a short one, both coverage shapes produce an identical span, and the key
    reaches every endpoint rather than just history.
  • ruff check, ruff format --check, mypy --strict and
    mkdocs build --strict are all clean.
  • The example was run against production in both formats, and resumed from a
    planted checkpoint.

🤖 Generated with Claude Code

cubehouse and others added 2 commits September 23, 2026 20:11
`tp.entity(id).history` reads the archive and pages for you. `span()` gives
the three dates a backfill needs in one shape for parks and rides, `days()`
yields one summary row per park-local day, `changes()` yields every recorded
observation. Both iterate to the end of the server's paging links and yield
as they go, so a resort's five years never has to be in memory at once.

Given a park id, both use the park-level call, which answers every entity in
the park in one request. Fetching the same data ride by ride is around a
hundred times more calls against the same budget, and that is the difference
between a backfill that takes an afternoon and one that takes a week.

The history budget is hourly, so a spent one can be most of an hour from
resetting. Sleeping through that is indistinguishable from a hung process, so
past `max_wait` (120s) we raise `BudgetExhaustedError` carrying `retry_after`
and let the caller checkpoint. `examples/backfill.py` does exactly that, with
NDJSON or CSV output; it pulled Disneyland Resort's whole daily archive,
98,452 rows, in one run.

Two defects found on the way, both pre-existing:

- The client could not send an API key at all. No `api_key` parameter existed
  and the transport sent only `user-agent` and `accept`, so every request this
  SDK made was anonymous regardless of what the caller had paid for. Both
  clients now take `api_key` and send `x-api-key`; an empty string is treated
  as no key, since an unset environment variable arrives as "" more often than
  as None and an empty key is a 401 rather than an anonymous request.

- The generator's nullable patcher matched only classes with no docstring.
  Its pattern could not cross the blank line that follows one, so every
  documented class went unpatched behind a warning nobody read. That included
  `next` on all four history envelopes, which is null on the last page of
  every paged response, so the SDK would have failed to parse the page that
  ends a backfill. The pattern now spans blank lines, an unmatched patch is
  fatal rather than a warning, and all 22 fields the spec marks required and
  nullable are listed.

172 unit tests pass; ruff, mypy and `mkdocs build --strict` are clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The transport honoured any Retry-After up to max_retries times. For a REST
429, which asks for seconds, that is right. For a history 429 it is not: that
budget is hourly, so a spent one can ask for most of an hour, and three of
those is about two and a half hours of a process producing no output and no
error. Indistinguishable from a hang.

It also made BudgetExhaustedError unreachable. The whole point of that error
is to let a backfill checkpoint instead of blocking, and the transport rode
the wait out before the history layer ever saw the 429 - so the error only
arrived after the delay it exists to avoid.

RetryConfig gains max_retry_after, 120 seconds by default. Past it the client
does not sleep at all: it raises RateLimitError with retry_after set, and the
history layer turns that into BudgetExhaustedError. A REST 429 asking for a
few seconds is still ridden out exactly as before.

Nine tests, in both transports and end to end through the history layer with
the shipped retry config rather than a test-only one, since building the
client with retries off is precisely how this hid. Mutation-checked: reverting
the condition fails three of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cubehouse

Copy link
Copy Markdown
Member Author

Follow-up commit: BudgetExhaustedError was very nearly unreachable as first pushed.

The transport honoured any Retry-After up to max_retries times, so a spent history budget made the SDK sleep for most of an hour, three times over, before the history layer ever saw the 429. The error exists to let a backfill checkpoint rather than block, and it would only have arrived after roughly two and a half hours of exactly the blocking it is there to prevent.

RetryConfig now takes max_retry_after (120s by default). Past it the client does not sleep at all and raises RateLimitError with retry_after set. A REST 429 asking for a few seconds is still ridden out unchanged.

Nine tests across both transports and end to end through the history layer, the last of those using the shipped retry config rather than a test-only one, since building the client with retries off is how this hid in the first place. Reverting the condition fails three of them.

PACKAGE_VERSION was a literal saying 2.0.0 in a package at 3.1.0. Every request
this SDK has made since 3.0.0 announced a version two majors old, and nothing
anywhere failed, because a literal only stays right while someone remembers to
change it and across two releases nobody did.

It now comes from importlib.metadata, which cannot drift, with a "0+unknown"
fallback for a source tree with nothing installed. A gate test pins it to
pyproject.toml, to the installed metadata and to the User-Agent the transport
builds, and refuses the fallback value outright so it can never reach a
released artifact. Restoring the old literal fails three of the five.

The JavaScript sibling had the same bug, a major out rather than two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cubehouse cubehouse changed the title History methods, and an API key to send with them History, an API key to read it with, and a version that is true Sep 23, 2026
@cubehouse

Copy link
Copy Markdown
Member Author

Third commit, and a fourth pre-existing defect.

PACKAGE_VERSION was a literal reading 2.0.0 in a package at 3.1.0. Every request this SDK has made since 3.0.0 announced a version two majors old, and nothing anywhere failed, because a literal only stays right while someone remembers to change it and across two releases nobody did. The JavaScript sibling had the same bug, one major out rather than two, which is what made it worth fixing structurally rather than by hand.

It now comes from importlib.metadata, with a 0+unknown fallback for a source tree with nothing installed. Five gate tests pin it to pyproject.toml, to the installed metadata and to the User-Agent the transport builds, and refuse the fallback value outright so it can never reach a released artifact. Restoring the old literal fails three of them.

Releasing as 3.1.0 once this is green.

@cubehouse
cubehouse merged commit c46c280 into main Sep 23, 2026
6 checks passed
@cubehouse
cubehouse deleted the feat/history branch September 23, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant