Conversation
CheckMessageSend read usage_summaries, which only gets written when a send
reaches its terminal outcome in meterSentTx, so every accepted-but-unsent
message was invisible to the cap. Reproduced against real Postgres on main:
10 concurrent POST /v1/agents/{email}/messages with max_messages_month=3 all
returned 202, and three sequential sends with the cap at 2 all passed.
The accepted message row is now the reservation. MessagesThisMonth and
MessagesToday count outbound rows in accepted/sending alongside the metered
total, and the accept transaction takes a per-user advisory lock (keyspace 3)
and re-reads the flow caps on its own connection before inserting, the same
shape as tokencanopy#942 and tokencanopy#901. Scheduled and review-held sends are excluded because
the worker's existing fire-time gate judges them against the month they fire
in.
Fixes tokencanopy#1005
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CheckMessageSendreads the month's outbound total fromusage_summaries, but that total is only written at the terminal outcome (meterSentTx), so an accepted-but-unsent message was invisible to the cap and a burst of immediate sends could all pass against the same pre-increment count. I reproduced it against real Postgres onmain: 10 concurrentPOST /v1/agents/{email}/messageswithmax_messages_month=3all returned 202, and three sequential sends with the cap at 2 all passed.The accepted message row is now the reservation.
usage.Store.MessagesThisMonthandMessagesTodayadd the units of outbound rows inaccepted/sendingto the metered total, so a send consumes budget the moment it is accepted, andlimits.DBEnforcer.ReserveMessageSendTxtakes a per-user advisory lock (keyspace 3, distinct from the domain-name/domain-owner/agent keyspaces) and re-reads the flow caps on the accept transaction's own connection before the insert — the same shape as #942's max_agents fix and #901's max_domains fix.DeliverOutboundcalls it for immediate sends, and the platform test send (acceptPlatformSend) calls it too. A reservation that would cross the cap rolls the whole accept transaction back and answers 402limit_exceededwith the same details the pre-check already produces.Scheduled and review-held sends are deliberately excluded from the reservation: the worker's fire-time gate judges those against the month they actually fire in, which is the month the cap should apply to. The checks that reject them at fire time now see in-flight immediate sends too, which is what that gate was already reaching for.
Operational risk
No schema change and no change to the
/v1request or response shape. A send that used to slip past the cap under a burst now gets the same402 limit_exceededthe endpoint already returns for an over-cap request.Two behavior notes worth a look. The account usage view (
GetUsage→usage.messages_month) now includes accepted-but-unsent sends, because it reads the same counter; billing reconciliation readsusage_summariesdirectly and this change leaves that table alone. And a send that fails before the terminal meter frees its reservation, because failed sends were already unmetered — I didn't change that refund behavior, the reservation just follows it.The loopback self-send path is not covered. It meters synchronously in the same request rather than through the queue-first accept transaction, so it has no accept-to-send window to close. Closing it would need a reservation inside the loopback transaction; say the word if you want that here.
Client surface checklist
Not applicable: internal concurrency fix, the request/response shape of the send endpoints is unchanged.
Test plan
internal/e2e/messages_send_quota_race_e2e_test.go,-tags integration), against a real Postgres 16 container:TestConcurrentImmediateSendsRespectMonthlyQuota: 10 concurrent immediate sends with the cap at 3. Onmainall 10 get 202; on this branch exactly 3 get 202 (3acceptedrows) and 7 get 402 withresource=messages_month, limit=3, current=3.TestImmediateSendConsumesMonthlyQuotaAtAccept: cap 2, three sequential sends. Onmainall three pass; on this branch the third is 402 and the first two each consume budget.TestImmediateSendConsumesDailyQuotaAtAccept: the optionalmax_messages_daycap takes the same accept-time reservation.internal/usage/pending_quota_test.go): accepted sends count, scheduled sends and review holds do not, and a terminal send moves out of the pending count intousage_summarieswithout double counting.go test -tags integration -race ./internal/e2e/ -run 'QuotaAtAccept|RespectMonthlyQuota'green.go test ./internal/usage/ ./internal/limits/ ./internal/agent/ ./internal/httpapi/green, andgo test ./...green../internal/e2e/suite green exceptTestWebhooksE2E_RotationGrace_DualSigandTestWebhooksE2E_DisabledWebhookNoFire, which need Mailpit on127.0.0.1:1025; both fail the same way onmainhere.scripts/check-repository-text-integrity.shgreen.make fmt-checkreports the same three pre-existing gofmt-drift files asmainunder Go 1.27 (CI pins 1.26); my files are clean.Fixes #1005