feat(webapp): server-side agent message quota - #4552
Conversation
Enforce the Free plan's agent-message allowance on the server, not just as a client hint. A per-(organizationId, period) counter lives in its own table, not joined to chats, so deleting a chat can no longer free quota inside the period. The create path and the .in append path each count one user message and refuse over the cap with a typed 403 the client renders as the upgrade block; wakes (action turns) never count. Fails open: an absent limit (self-hosted, or before the cloud side ships) or a counter read that throws means no cap. TRI-12863.
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughAdds monthly, organization-level agent message quotas with UTC billing periods. Stores usage in a new database table with atomic increment queries. Applies quota checks during chat creation and message forwarding, and records counted messages. Updates dashboard agent chat to process quota refusals, refresh usage, and display an upgrade block with the applicable limit. Adds PostgreSQL-backed tests for threshold, period, persistence, resolver, and error-handling behavior. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…1166' into feat/agent-message-quota-tri-12863
@trigger.dev/build
trigger.dev
@trigger.dev/core
@trigger.dev/python
@trigger.dev/react-hooks
@trigger.dev/redis-worker
@trigger.dev/rsc
@trigger.dev/schema-to-json
@trigger.dev/sdk
commit: |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
apps/webapp/test/dashboardAgentQuota.test.ts (1)
162-174: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider covering a failing
readLimitas well.The fail-open test covers only a counter read that throws. The limit lookup is the other half of
Promise.alland can also reject. Add a case wherereadLimitrejects and assertundefined.♻️ Suggested extra case
it("fails open when the counter read throws", async () => {Add after that test:
it("fails open when the limit lookup throws", async () => { const result = await resolveAgentMessageQuota({} as unknown as DashboardAgentDb, { organizationId: ORG, readLimit: async () => { throw new Error("limit cache down"); }, }); expect(result).toBeUndefined(); });apps/webapp/app/services/dashboardAgentQuota.server.ts (1)
13-15: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueKeep the cast only while
agentMessagesis outsideLimits.This is the only non-node_modules reference to
AGENT_MESSAGE_LIMIT_KEY, andagentMessagesis not declared in the repo yet. Add a follow-up to remove the cast once@trigger.dev/platformdeclares it, or keep the currentLimitslookup via theagentMessageskey directly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: fc824e93-d916-42f2-a87d-9e9793e73c9c
📒 Files selected for processing (12)
.server-changes/agent-message-quota.mdapps/webapp/app/components/dashboard-agent/DashboardAgentChat.tsxapps/webapp/app/components/dashboard-agent/useAgentMessageQuota.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.dashboard-agent.in.$.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.dashboard-agent.tsapps/webapp/app/services/dashboardAgentQuota.server.tsapps/webapp/test/dashboardAgentQuota.test.tsinternal-packages/dashboard-agent-db/drizzle/0004_stale_corsair.sqlinternal-packages/dashboard-agent-db/drizzle/meta/0004_snapshot.jsoninternal-packages/dashboard-agent-db/drizzle/meta/_journal.jsoninternal-packages/dashboard-agent-db/src/queries.tsinternal-packages/dashboard-agent-db/src/schema.ts
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
A failed upstream send (5xx/502) or a non-2xx response burned a quota message that never reached the agent. Record only after upstream.ok.
Draft submit and chat retry now bail when the message cap is reached, so a suggested prompt or retry over the cap no longer fires a silent 403. The capped draft keeps any open watch card. The quota re-reads when a turn settles instead of on optimistic append, so the count and cap no longer lag by one message.
| export function agentTurnCountsAgainstQuota( | ||
| turn: { kind?: string; payload?: { trigger?: string } } | undefined | ||
| ): boolean { | ||
| return turn?.kind === "message" && turn.payload?.trigger !== "action"; | ||
| } |
There was a problem hiding this comment.
🟡 Retrying a failed answer uses up one of the user's monthly messages
A regenerate/retry turn is treated as a chargeable user message (agentTurnCountsAgainstQuota at apps/webapp/app/services/dashboardAgentQuota.server.ts:96-100), so every retry silently eats another message from the monthly allowance even though the person typed nothing new.
Impact: Users burn their free allowance faster than the messages they actually sent, contradicting the release note that only delivered messages are counted.
How a regenerate reaches the counting path
The SDK transport sends both submit-message and regenerate-message turns as { kind: "message", payload: { trigger } } to /in/append (packages/trigger-sdk/src/v3/chat.ts:776-851), which the webapp proxies at apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.dashboard-agent.in.$.ts:139-148 and then increments on success at lines 190-194.
agentTurnCountsAgainstQuota only excludes trigger === "action", so regenerate-message returns true. The retry button in DashboardAgentChat (retryAction → regenerate()) therefore charges a message for re-running a turn the user already paid for — the exact case the failed-send fix was meant to cover. Steering sends (sendPendingMessage, packages/trigger-sdk/src/v3/chat.ts:1118-1126) and tool-approval continuations also arrive as submit-message and are charged the same way.
The rule should key on the turn actually carrying a new user message (e.g. payload.trigger === "submit-message" with a message present), not merely "not an action".
Prompt for agents
agentTurnCountsAgainstQuota in apps/webapp/app/services/dashboardAgentQuota.server.ts currently counts any `kind: "message"` turn whose payload trigger is not "action". The SDK chat transport (packages/trigger-sdk/src/v3/chat.ts, sendMessages) emits `kind: "message"` for BOTH `submit-message` and `regenerate-message`, and regenerate deliberately omits the `message` field because the agent re-slices its own history. As a result, pressing Retry after a failed/unsatisfying turn charges another message against the org's monthly allowance even though no new user message was sent. Tool-approval continuations and steering sends also arrive as `submit-message` and get charged. Consider narrowing the rule to turns that carry a new user message (trigger === "submit-message" AND payload.message present with role "user"), and add a test that a regenerate turn does not count.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
What & why
The Free-plan agent-message allowance was only a client-side hint (
useIsFreePlan()was hardcoded toundefined, so nothing was enforced) and the running count was a liveCOUNT(*)overchat_messages— which meant deleting a chat silently freed quota. This PR makes the allowance a real server-side limit with a durable counter, so the cap holds regardless of the client and a deleted chat can't reclaim messages within the period.The cloud side that fills the actual per-plan number is a separate PR (TRI-12863 P0). Until it deploys the
agentMessageslimit key is absent, which resolves to the repo's unlimited sentinel — the correct fail-open default, and why this ships independently.What's inside
trigger_dashboard_agent.agent_message_usage, keyed(organization_id, period)whereperiodis a UTC calendar month"YYYY-MM". FK-free (pgSchemaconvention), plus a drizzle migration. Deliberately not joined to chats — that closes the delete-a-chat hole. Distinct from TRI-13068'sAiUsageEvent; neither is derived from the other.dashboardAgentQuota.server.ts: a purecheckAgentMessageQuota({ used, limit })(so the MCP send path can reuse the rule later) plus an org-scoped resolver that reads the period counter and the cached plan limit, and arecordAgentMessageSentincrement..inappend path each increment one user message. The append path counts only after the existingtrigger === "action"403, so wakes never count.403 { error: "message_quota_reached", limit }, which the client renders asAgentUpgradeBlockrather than a generic failure. Never a silent drop.?quota=1now reads the period counter;useIsFreePlan()is a real read gated on billing presence (no subscription → self-hosted → no cap, no upgrade UI), not on the plan value.Key decisions
100_000_000(neverInfinity— that serializes tonullin the Redis limit cache), and a counter read that throws returns "no cap". Self-hosted needs zero extra branching — it falls out of the fallback, with a test to prove it.(org, period)table closes the delete hole. A standalone counter, not a count over chat rows, so deleting a chat can't free quota inside the period.Testing
apps/webapp/test/dashboardAgentQuota.test.ts(testcontainers, no mocks):checkAgentMessageQuotaunder/at/over/unlimited (control-breaks the>=);agentTurnCountsAgainstQuotacounts a message but not a wake (control-breaks the wake exclusion);TRI-12863