[IAmJSD]

Summary

The ladder compared an on-demand-equivalent (OD) target with existing Savings Plan coverage measured in commitment dollars. This PR converts existing and expiring coverage to OD dollars before Allocate. Following the maintainer's note on the issue, it also makes legacy NULL-basis and on_demand_equivalent tranches impossible to claim.

Closes #946

Units

• Commitment $/hr: what is bought and stored. The AWS provider's LayerState.ExistingUSDPerHour and ExpiringUSDPerHour are in this unit: SavingsPlan.Commitment for the SP layers and amortized RI cost for the RI layer.
• OD $/hr: on-demand-equivalent spend covered. Baseline, target, gap and the netting totals are in this unit.
• Rate r is in (0, 1] and means commitment dollars per OD dollar. od = commitment / r and commitment = od * r. This is the same rule go's allocate_units.go uses for tranches.

Root cause

executeLadderRun passed GetLayerStates straight to Allocate and assembleLadderPlan. go's LayerState doc says callers must convert these values to OD first. Existing coverage was therefore understated by a factor of r and the gap overstated. Example: a 2.5 commitment $/hr EC2 Instance SP at r = 0.5 really covers 5 OD $/hr. It was netted as 2.5, so against a target of 10 the run placed 7.5 OD (3.75 commitment) instead of 5 OD (2.5 commitment).

Fix

1. internal/server/ladder_existing_units.go (new): ladderLayerStatesOnDemand(states, rates) returns a converted copy of the states and never mutates the provider's map.
   • It converts each layer with the rate that executeLadderRun already fetched from GetCommitmentRates for the config's term and payment option. This is option 2 on the issue. DescribeSavingsPlans does not expose each plan's own rate, so plans bought at a different discount are approximated by today's rate.
   • The math is exact (big.Rat). When the quotient is not a float64, existing rounds up and expiring rounds down. Net coverage (existing - expiring) is therefore never understated, which errs toward under-buying.
   • A layer with no commitment (nil or 0) needs no rate. Nil stays nil, and Allocate still rejects it.
   • A layer with non-zero existing or expiring commitment and no rate fails the run (Errored, nothing persisted). Netting it 1:1 would reproduce this bug, and no value is a safe upper bound. Invalid rates (outside (0, 1], NaN, Inf) and negative, NaN or overflowing amounts also fail.
2. executeLadderRun calls it right after the rate fetch. The converted states feed Allocate and the plan, so ladder_runs.existing_usd_hr and plan JSON existing are now OD. That matches go's LadderPlan.ExistingUSDPerHour doc and the amend ceiling (LEAST(target, baseline*coverage) - existing), which already assumed OD. The ceiling therefore gets stricter, never looser.
3. Maintainer comment / known issue: ClaimDueLadderTranches (selectDueLadderTranchesTx) now selects only amount_basis = 'commitment' rows. The validated ladder_tranche_amount_provenance constraint guarantees those rows have a commitment_rate.
   • Legacy NULL-basis and on_demand_equivalent rows are never claimed. They stay netted while within LadderFireGrace. After that, the existing overdue sweep cancels them and the next run re-plans their gap in commitment dollars ("cancelled or re-planned", with no migration).
   • The filter is in SQL, before LIMIT ... FOR UPDATE SKIP LOCKED. Older legacy rows therefore cannot use up a claim's limit and starve a due commitment row.
   • Basis is never updated after insert, and the claim still runs under the per-config ladder lock, so it adds no new race.
   • The known_issues entry is marked resolved.

Visible behavior changes

• AWS accounts with active Compute SPs or Convertible RIs now get an Errored ladder run (logged as existing coverage: layer compute-sp: no commitment rate ...). Today the AWS provider supplies a rate only for EC2 Instance SP (fix(ladder): bump cloud-commitments-go and plan tranches in commitment dollars #976 already holds those layers for new purchases). Before this PR, such runs silently produced over-sized plans; now they produce no plan. Once go's provider returns Compute SP and RI rates, or option 3 (Cost Explorer covered OD cost) lands, these runs plan again. A softer alternative would be a persisted Hold-only run with this reason. I kept fail-loud to match fix(ladder): bump cloud-commitments-go and plan tranches in commitment dollars #976's "no capability → Errored" precedent, and I'm happy to switch if you prefer.
• existing_usd_hr on new runs is OD. Historical ladder_runs rows keep their commitment-dollar snapshot because no rate was stored to backfill from. If the latest run predates this change, the amend ceiling uses that older, smaller existing value until the next run writes an OD one.
• No money moves. The ladder executor still has no production caller.

Verification

All evidence comes from unit, fake-capability and local-PostgreSQL tests. Nothing ran against a real AWS account. Tests ran with AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null because this machine's default AWS profile is invalid.

Fail-before (on origin/main c8bdd46, with the new tests and no fix):

• go test ./internal/server/ -run 'LadderLayerStatesOnDemand|TestExecuteLadderRun_(Existing|Expiring)':
  • TestExecuteLadderRun_ExistingSavingsPlanCoverageIsOnDemandEquivalent: existing 2.5 (want 5), gap 7.5 OD (want 5), total_hourly_commit 3.75 (want 2.5), plan JSON existing 2.5 (want 5).
  • TestExecuteLadderRun_ExpiringCoverageIsOnDemandEquivalent: gap 8 (want 6), commit 4 (want 3).
  • TestExecuteLadderRun_ExistingCoverageWithoutRateIsErrored: no error (the run was netted 1:1).
• go test -tags integration ./internal/config/ -run TestClaimDueLadderTranches_NeverClaimsOnDemandOrUnknownBasisRows: the limit-1 claim took the older NULL row, 3 execution rows were created (want 1), and the NULL and OD rows were fired (want scheduled, then cancelled after grace).

Pass-after (commit 6af6413):

• Tests that pin the conversion: TestLadderLayerStatesOnDemand_PinsCommitmentToOnDemandConversion covers 5 @ 0.5 → 10, 3 @ 0.25 → 12, 7 @ 1 → 7, and expiring by the same rate. ..._RoundsTowardUnderBuying checks that at 1/0.3, existing is the nearest float at or above the exact value and expiring the nearest at or below. ..._DoesNotMutateInput, ..._EmptyLayersNeedNoRate and ..._RejectsUnconvertibleCoverage (8 cases) cover the rest, plus the three executeLadderRun tests above.
• Mutation probes, all killed by the tests above:
  • expiring rounded up;
  • existing rounded down;
  • a missing rate treated as 1:1;
  • multiply instead of divide;
  • expiring left unconverted;
  • dropping the claim filter (the fail-before run above).
• go build ./..., go vet ./..., go vet -tags integration on config/purchase/api/server, and make tidy-check are clean.
• go test -count=1 passes for ./internal/server/ ./internal/config/ ./internal/purchase/ ./internal/api/ ./internal/mocks/, and go test -race -run Ladder ./internal/server/ passes.
• go test -tags integration -count=1 ./internal/config/ (whole package) passes, as do -run Ladder ./internal/purchase/ and -run Ladder ./internal/api/. These ran on the repo's testcontainers PostgreSQL via Docker, with no skips.
• golangci-lint v2.10.1 (--allow-parallel-runners, with and without --build-tags integration) reports 0 issues.
• gocyclo -over 10 -ignore "_test\.go" on the changed non-test files is silent.
• scripts/gosec-hook.sh on the changed packages and markdownlint-cli 0.47.0 on the known_issues file are clean.
• git-secrets ran in a fresh temp repo with an isolated HOME and every changed file tracked. A planted AKIA key was flagged (rc 1); the clean scan returned rc 0. scripts/test-git-secrets-allowlist.sh reports 137 PASS, 0 FAIL.

Test fixture changes: the existing claim tests now seed commitment-basis rows at rate 1 through a new claimableTranche helper (or explicit basis fields in the purchase and api rigs), so their netting values are unchanged. TestLadderTrancheProvenance_InsertClaimAndReconcile now asserts that only the commitment row is claimed and fires the NULL and OD rows directly, so the fired-row reader still covers every basis. TestLadderAmendCeiling_CancelsOverdueBeforeCounting marks its overdue row as commitment, so the cancel, not the basis filter, is what keeps it unclaimed.

Out of scope / follow-ups

• Per-plan rates (option 1) or covered-OD from Cost Explorer (option 3) would remove the today's-rate approximation. Both need provider work in cloud-commitments-go.
• Compute SP and Convertible RI rates from the AWS provider: without them, accounts holding those commitments error (see above).
• go's sumSPHourlyCost sums EC2 Instance SPs from every region, while the EC2 Instance SP rate is regional. This is pre-existing and noted here only.
• Historical ladder_runs.existing_usd_hr rows are not backfilled.

Frontend follow-up for the UI owner

No change is required. The timeline chart already treats existing_usd_hr as the OD base. New runs now match that label, so only runs from before this PR show a commitment-dollar base. Optionally, the run snapshot text ("existing $X") could say "on-demand-equivalent".

Maintainer: please mirror the issue's triage labels on this PR: priority/p2, severity/medium, urgency/this-quarter, impact/few, effort/m, type/bug, triaged.

🤖 Generated with Claude Code

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}