feat(webapp): run the dashboard agent through AWS Bedrock behind an env switch - #4609
feat(webapp): run the dashboard agent through AWS Bedrock behind an env switch#4609kathiekiwi wants to merge 19 commits into
Conversation
…switch DASHBOARD_AGENT_MODEL_PROVIDER=bedrock routes the agent's model calls through AWS Bedrock; the default stays direct Anthropic. Managed prompts keep canonical "anthropic:<model-id>" strings, resolved per provider in one place alongside the provider-shaped prompt-cache options.
Bedrock reports its cache write on the call's provider metadata and its cache read only on the call's usage, so the per-step cache telemetry reads whichever the active provider populates.
…ed fallback Confirm both Bedrock profiles against the SDK model-id union, throw on an unmapped id instead of guessing an unqualified profile, and add a structural shape test.
authenticateUserActor now applies the same membership floor as authenticateSession: a delegated token for a user who is not a member of the scoped org/project is denied, instead of being handed a usable (read:all-by-default) ability. Platform admins stay exempt.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (4)
📜 Recent review details⏰ Context from checks skipped due to timeout. (14)
WalkthroughThe dashboard agent now supports Anthropic and Bedrock model providers. Environment validation accepts provider-specific AWS configuration. Shared model resolution and cache-breakpoint helpers replace Anthropic-specific runtime logic. Cache telemetry reads provider metadata and usage fallbacks. Delegated user-actor authentication now validates token subjects and enforces organization and project membership, with integration test coverage. Mergeability Score: 🔵 Low · up to The PR adds an opt-in Bedrock provider while preserving the Anthropic default; the remaining merge-readiness risk is a test cleanup path in step-cache.test.ts that can remove a pre-existing provider setting and make provider selection order-dependent. This is a bounded test reliability issue requiring owner awareness, not evidence of a production behavior regression. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Only load the user and run the membership floor when organizationId or projectId is present, restoring pre-P5 behavior for unscoped tokens (no user query, no denial). The user lookup falls back to the primary before failing closed, so replica lag can't spuriously 401 a just-created member.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…er switch The sampled-turn eval task resolved its judge with Anthropic directly, so under the Bedrock switch with no Anthropic key every sampled eval failed. It now follows the same provider seam as the agent.
…versations On Bedrock the turn-wide prefix marker and the per-step marker were byte-identical, so the step-strip pass removed the prefix on short chats, leaving the history uncached every step. The step marker now carries an explicit ttl so the prefix survives, matching the Anthropic path.
Use a nonexistent userId for the unscoped case so success can only hold if the lookup is skipped.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…EFAULT_REGION The Bedrock SDK reads only AWS_REGION; pass region explicitly so AWS_DEFAULT_REGION (common in AWS deploys) works too. Credentials stay on the SDK's own chain.
…ing the webapp gate
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
The Bedrock step marker discriminated itself with a ttl inside cachePoint, which the SDK copies verbatim into the Converse request. Move the prefix/step discriminator to a non-serialized top-level providerOptions key so the wire cachePoint is a plain {type:"default"}. The key's value is an object because the AI SDK validates message providerOptions as records.
There was a problem hiding this comment.
Pull request overview
This PR adds a provider seam to the dashboard agent so model calls can be switched between direct Anthropic and AWS Bedrock via an env var (defaulting to Anthropic), updates prompt-caching breakpoints/telemetry to work across providers, and hardens RBAC delegated user-actor tokens by enforcing org/project membership in scoped contexts.
Changes:
- Introduces
model-provider.tsto resolve canonical"anthropic:<id>"prompt model strings against the active provider (Anthropic vs Bedrock) and to standardize cache-breakpoint option shapes. - Updates dashboard-agent and webapp head-start callsites to route model selection + cache breakpoint configuration through the new seam.
- Adds RBAC tenant-floor enforcement for scoped delegated user-actor tokens, plus Postgres-backed tests for the new behavior.
Reviewed changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pnpm-lock.yaml | Adds Bedrock AI SDK dependency and related transitive lock updates. |
| internal-packages/rbac/tsconfig.json | Updates TS config (currently excludes a new test from tsc). |
| internal-packages/rbac/src/fallback.userActor.test.ts | Adds Postgres-backed tests covering scoped/unscoped user-actor token behavior. |
| internal-packages/rbac/src/fallback.ts | Enforces membership floor for scoped delegated user-actor tokens (admin exempt). |
| internal-packages/rbac/package.json | Adds @internal/testcontainers devDependency for RBAC tests. |
| internal-packages/dashboard-agent/src/watch-actions.ts | Routes model selection through resolveDashboardAgentModel() for provider switching. |
| internal-packages/dashboard-agent/src/step-cache.ts | Centralizes cache-breakpoint + cache-usage parsing via the provider seam. |
| internal-packages/dashboard-agent/src/step-cache.test.ts | Extends tests for Bedrock breakpoint shape and Bedrock cache read/write telemetry behavior. |
| internal-packages/dashboard-agent/src/model-provider.ts | New provider registry + canonical model resolver + provider-specific cache breakpoint helpers. |
| internal-packages/dashboard-agent/src/model-provider.test.ts | Adds unit tests for provider switching, model mapping, and breakpoint strip/classification. |
| internal-packages/dashboard-agent/src/eval-turn.ts | Routes eval/judge model selection through the provider resolver. |
| internal-packages/dashboard-agent/src/dashboard-agent.ts | Updates main agent flow to use provider resolver and standardized breakpoint injection. |
| internal-packages/dashboard-agent/src/compaction.ts | Routes summary model selection through provider resolver. |
| internal-packages/dashboard-agent/src/cache-breakpoint.test.ts | Updates expectations to include the new breakpoint discriminator tag. |
| internal-packages/dashboard-agent/src/agent-runtime.ts | Exports provider seam and updates breakpoint injection to use it. |
| internal-packages/dashboard-agent/package.json | Exports ./model-provider and adds Bedrock provider dependency. |
| apps/webapp/app/services/dashboardAgentHeadStart.server.ts | Head-start now uses provider resolver + standardized prefix breakpoint injection. |
| apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.dashboard-agent.ts | Makes head-start eligibility gate provider-aware (AWS region vs Anthropic key). |
| apps/webapp/app/env.server.ts | Adds optional AWS env vars and validates DASHBOARD_AGENT_MODEL_PROVIDER. |
Files not reviewed (1)
- pnpm-lock.yaml: Generated file
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| function breakpointKind(providerOptions: ProviderOptions): CacheBreakpoint | undefined { | ||
| return providerOptions?.[CACHE_BREAKPOINT_KEY]?.kind; | ||
| } |
| // Excluded from tsc: it imports @internal/testcontainers, whose source | ||
| // (pulled in via customConditions) doesn't type-check under this config. | ||
| // vitest still runs and transpiles it. | ||
| "exclude": ["node_modules", "dist", "src/fallback.userActor.test.ts"] |
What & why
The dashboard agent can now run its model calls through AWS Bedrock instead of the direct Anthropic API, chosen by a single env switch. It's off by default (
DASHBOARD_AGENT_MODEL_PROVIDERunset ⇒anthropic), so merging changes nothing at runtime — the Bedrock path is a dormant branch until an operator sets the switch and AWS config. The default Anthropic path is byte-for-byte unchanged.This also carries a related tenant-isolation hardening for the agent's delegated token (kept together deliberately — both land the agent on Bedrock for HIPAA readiness). Refs: TRI-13251, TRI-11032.
What's inside
Provider seam —
internal-packages/dashboard-agent/src/model-provider.ts: the registry now holds bothanthropicandbedrock;resolveDashboardAgentModel()maps the canonical"anthropic:<id>"strings the managed prompts carry to the active provider, and the cache-breakpoint helpers emit the active provider's shape — AnthropiccacheControlvs BedrockcachePoint. Managed prompt strings stay canonical, so stored prompts don't change meaning. Unmapped model ids throw rather than shipping a guaranteed-404 profile. All agent, watch, compaction and title callsites route through the resolver; thedashboardAgentModelKeylocals override (test mock injection) is preserved.Cache telemetry —
step-cache.ts: cache token usage is read from the active provider (Anthropic reports it on provider metadata; Bedrock reports the write on metadata and the read via standard usage), sogen_ai.usage.cache_*is populated on both. This also fixes a latent ordering bug where step attributes could null-overwrite the prompt-cache read count.Webapp callsites —
dashboardAgentHeadStart.server.tsand the head-start route resolve the model and the cache breakpoint through the same seam, so the warm-up prefix and the following turn share one provider. The head-start firing gate is provider-aware: on Bedrock it gates onAWS_REGIONand lets the SDK resolve credentials (IAM role / static keys / session token / bearer), so a role-based deploy still warms; on Anthropic it staysBoolean(ANTHROPIC_API_KEY).app/env.server.tsgains the optional AWS vars and validatesDASHBOARD_AGENT_MODEL_PROVIDER.ANTHROPIC_API_KEYis untouched and not required on a Bedrock deploy.Tenant-isolation hardening —
internal-packages/rbac/src/fallback.ts: for a scoped context, the OSSauthenticateUserActornow applies the same membership floor as the session path — a delegated user-actor token whose user is not a member of the scoped org/project is denied (403). Unscoped tokens keep their prior behavior (no tenant claim, no lookup). The user lookup falls back replica→primary so replication lag can't spuriously 401 a just-joined member. Members and admins are unaffected. Previously this invariant held only through per-route discipline; this makes it structural.Enabling Bedrock (later, ops)
DASHBOARD_AGENT_MODEL_PROVIDER=bedrockidentically in both the webapp and the agent task container — the webapp warms the cache prefix and the task reads it, so a split would silently miss the cache.AWS_REGIONand provide credentials the Bedrock SDK can resolve (IAM role preferred). For v1 this runs without an Anthropic API key. Note: with no Anthropic key set, rollback is "turn the agent off", not "unset the switch" (unsetting falls back to the Anthropic provider, which then has no key).Testing
Unit tests cover both provider paths: the provider switch and per-provider cache shapes, a structural regex asserting Bedrock ids are real inference profiles (not an echo of the table), the split-metadata cache telemetry, and real-Postgres RBAC tests — member allowed, scoped non-member denied (org-only and project-only), missing user → 401, admin non-member exempt, unscoped success.
typecheck --filter webappand the dashboard-agent + rbac suites pass.