fix(ingestion): stop duplicate instants reaching one INSERT

[jirhiker]

The first live materialization failed. Both raw assets succeeded;
san_acacia_observations failed three times (initial + 2 retries) and took the
run down.

ON CONFLICT DO UPDATE command cannot affect row a second time
HINT: Ensure that no rows proposed for insertion within the same command
      have duplicate constrained values.

Nothing was written — the transaction rolled back each time, so staging is
unchanged and holds only the 542,161 pre-existing AMPAPI rows.

Two causes, both mine

Overlapping windows. Diver-HUB ranges are inclusive at both ends — "from
start time up to and including end time" — but iter_windows made adjacent
windows share a boundary: [0, span] then [span, 2·span]. A reading logged
exactly on the boundary came back in both.

No deduplication in the loader. That pair then reached Postgres inside one
INSERT, which the constraint from #847 correctly refuses.

Fixes

Windows now leave exactly one second between them. Timestamps are
second-resolution, so nothing falls in the gap — there is a test for that.

The loader deduplicates within a batch, keeping the last occurrence, which
matches the upsert's own rule that a later value wins. That guard holds whatever
the source does, including a vendor logging one instant twice — so this does not
depend on the window fix being perfect.

Why the existing tests missed it

test_loading_the_same_window_twice_does_not_duplicate loads the same window in
two separate statements, which Postgres allows. The failure needs duplicates
in one statement. The new test does exactly that and asserts 6 records
become 3 rows with the later value winning.

Diagnosis note

RetryPolicy obscured this: the surfaced error was "Exceeded max_retries of 2",
and the real cause was two levels down under error.cause. Worth knowing when
reading a failed run.

125 ingestion tests plus 11 loader tests pass.

🤖 Generated with Claude Code

[github-actions]

Coverage

✅ 78.59% total — gate is 75%.

Coverage for the Python files changed in this PR

Name	Stmts	Miss	Cover	Missing
automated_ingestion/ocotillo/loader.py	60	14	77%	60, 142, 196-225
automated_ingestion/shared/windows.py	32	2	94%	54, 74
TOTAL	92	16	83%

[github-actions]

Your pull request is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
ocotillo-automated-ingestion		View in Cloud	Aug 19, 2026 at 08:21 PM (UTC)