chore(release): staging to production - 2026.05.04 - #1061
Merged
Conversation
## Summary Implements configurable Fiddler guardrails **failure posture** (fail-open vs fail-closed), **observability-only (shadow)** mode, honest **degraded** health when the API cannot be evaluated, and full **config hierarchy** (environment → organization → chatflow). Adds **plan-tier awareness** so freemium Fiddler accounts no longer surface phantom 404 banners, plus a comprehensive UI polish pass for both light and dark themes. Closes #1059 ## What changed ### Runtime - New typed errors (`FiddlerAuthError`, `FiddlerUpstreamError`, `FiddlerTimeoutError`, `FiddlerNetworkError`, `FiddlerCircuitOpenError`, **`FiddlerUnsupportedError`**) replace silent fake-safe fallbacks. - Shared `runStage` helper centralizes fail-open / fail-closed branching across all 5 input/output call sites. - `FiddlerUnsupportedError` (HTTP 404 + "not supported by the freemium guardrails API") is treated as `ok: true, degraded: false` — does not block requests, does not show degraded banners, does not trip the circuit breaker. - Process-wide cache of unsupported endpoints keyed by `(sha256(apiKey), endpoint)` — freemium accounts pay the round-trip once per endpoint, and rotating to a paid key invalidates automatically. - Faithfulness payload corrected to Fiddler's spec: `{data: {prompt, response, context}}` (was `{data: {input, context}}` → 400). User prompt is plumbed through `runOutputStage` from `buildChatflow.ts` and `buildAgentflow.ts`; faithfulness is skipped (not thrown) when no prompt is available. - Org / Platform credential resolution: the runtime no longer filters strictly by `workspaceId`, so org-visible credentials reach sibling-workspace chatflows. - AgentFlow V2 output validation parity (was missing entirely; previously only chain and legacy multi-agent flows ran output validation). - Structured `logger.*` everywhere — failures land in Datadog / log aggregation. ### API - `GET /api/v1/guardrails/selftest` for operators — resolved config, credential source, live `healthCheck` latency/status, circuit-breaker state. - `FiddlerGuardrailsService.capabilityMatrix()` probes safety / PII / faithfulness independently. Selftest now returns a `capabilities` block plus human-readable `notes` when PII or faithfulness aren't on the connected plan. ### Admin UI (Guardrails Settings) - MasterConfig: failure-mode toggle, observability-only switch, **Plan capabilities** chip row (Safety / PII / Faithfulness — green available, amber not-included, red degraded) with a Recheck button. - Credential change (select-from-dropdown **and** add-new) is now async-aware — `onSave` is awaited before the capabilities `useEffect` re-fetches, so the chip row reflects the new key immediately instead of one click later. Optimistic local state with rollback on save failure. - SimpleMode: Strict / Balanced / Lenient presets now set `failureMode` and `observabilityOnly` so degraded behavior matches preset intent. - AdvancedMode: per-stage Enable switches, sliders, sub-tabs, and accordion summaries restyled for theme parity via shared `sx` constants. - Comprehensive light / dark theme pass: - Outlined action buttons (Edit / Change / Recheck) anchor on `text.primary` + `divider` instead of translucent `primary.main` (were nearly invisible in dark mode). - Page-level switches (Enable Guardrails, Observability-only) and section switches (Safety, PII, Faithfulness) use `info.main` (Material blue) when on — same look in both themes. - Selected toggle button (Fail open / Fail closed) uses `action.selected` + `text.primary` for legible contrast in dark mode. - Selected preset cards use `action.selected` + theme-aware border (was a malformed rgba opacity suffix). - Sliders, inner tabs, accordion summaries restyled with shared `sx` constants for consistent rhythm. - Save Configuration buttons now show a solid disabled state (the global `contained` gradient was bleeding through the disabled state). ### Chatflow UI - Per-chatflow `failureMode` override in chatflow guardrails dialog. - Per-chatflow **Observability-only (override)** switch mirrors org admin controls. - Saves merge partial updates and re-read `chatbotConfig` from the save response (no more dropped fields on partial saves). ### Chat message UI - Amber degraded-state banner renders only on real outages (auth, timeout, upstream) — never on plan-tier `unsupported`. - In-accordion Health detail block (mode, per-stage reason, HTTP status, latency). - "✓ Clean" rendering for Input / Output Validation sections when no violations / redactions / faithfulness signals exist (previously left an empty section under the header). - `reason: "unsupported"` rendered as **"Not included in Fiddler plan (skipped silently)"** instead of generic `api_error`. ### Docs - `packages/docs/docs/integrations/fiddler.mdx` — new "Plan Tiers (Freemium vs Paid)" section, updated `selftest` example with `capabilities` and `notes`, refreshed troubleshooting FAQ. - `CHANGELOG.md` — full Features / Bug Fixes / Internal / Plan-Tier Awareness entries under `[Unreleased]`. - `.alphaAgent/spec/env-vars.json` — new `FIDDLER_FAILURE_MODE`, `FIDDLER_OBSERVABILITY_ONLY` registered. - `apps/web/e2e/tests/GUARDRAILS_TESTS.md` — E2E coverage notes. ## Schema impact - New `health` field on `chat_message.guardrails_metadata` JSON column — **no DB migration** (reuses existing TEXT column). ## Tests - New `packages/server/test/services/guardrails/index.test.ts` — 8 passing unit tests covering `FiddlerUnsupportedError` handling, the unsupported-endpoint cache, circuit-breaker interaction, and the corrected faithfulness payload. ## Testing checklist - [ ] `/api/v1/guardrails/selftest` returns a `capabilities` block reflecting connected plan tier - [ ] Freemium key: chat works; no amber banner; admin chip row shows amber on PII / Faithfulness - [ ] Paid key: all three capability chips green; safety / PII / faithfulness all run end-to-end - [ ] Failure mode = `closed` + bad API key → chat blocked with clear message - [ ] Failure mode = `open` + bad API key → chat delivered with amber degraded banner - [ ] Observability-only ON → real violations recorded in `guardrailsMetadata`, request not blocked - [ ] Per-chatflow override beats org default for both `failureMode` and `observabilityOnly` - [ ] Change credential in admin → Plan capabilities row refreshes without manual Recheck - [ ] Light + dark theme: outlined buttons, switches, sliders, preset cards, toggle group all legible --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…status (#1062) ## Summary - **Bug fix — `generateCsv` dropped AI-introduced columns.** `convertToCSV` was building the header from `Object.keys(data[0])`. If row 0's `generatedData` was empty, errored, or shaped differently from later rows, every column that appeared only later was silently dropped from the export. Replaced with a stable union of keys across all rows (first-seen order); fields containing newlines/CR are now quoted as well. Mirror fix applied in the `packages-answers/scripts/generateCsv.ts` Prisma copy. - **CSV Transformer reachable from the sidebar.** Added a "CSV Transformer" entry under Agent Studio (and the public-org Answer Studio submenu) in `AppDrawer.tsx`, linking to `/sidekick-studio/csv-transformer`. Always visible (no env gate). - **Cron-status banner on the page.** The Next.js page is now a server component that reads `ENABLE_CSV_RUN_CRON` and passes a `cronEnabled` prop into the client. Renders a green `success` banner when enabled, and an orange `warning` banner when disabled (explaining that runs will be created but not processed until the env is set + worker restarted). No `NEXT_PUBLIC_` duplication required. The original `dynamic(..., { ssr: false })` behavior is preserved via a thin `CsvTransformerClient.tsx` wrapper. ## Why - Users were submitting CSV runs and getting back files with the four enrichment columns empty even though the chatbot was returning data — root cause was a combination of (a) an output-parser schema mismatch on the chatflow side and (b) the `convertToCSV` first-row-only header bug. This PR fixes (b). - The CSV Transformer page existed at `/sidekick-studio/csv-transformer` but was unreachable from the main sidebar navigation in most environments. - It was not obvious whether a given environment had the CSV cron worker turned on, so users could submit runs that would never be processed and have no visible indication why. ## Test plan - [ ] Build succeeds (`pnpm build`). - [ ] Sidebar: open Agent Studio → "CSV Transformer" appears below Credentials and links to `/sidekick-studio/csv-transformer`. - [ ] CSV Transformer page renders the **warning** banner when `ENABLE_CSV_RUN_CRON` is unset or not `true`. - [ ] Set `ENABLE_CSV_RUN_CRON=true`, restart server, and confirm the page renders the **success** banner. - [ ] Submit a small CSV through a CSV-tagged chatflow whose output schema returns a key that the original CSV does not have. Confirm that the processed CSV download contains the new column header and values for all rows (regression test for the dropped-column bug). - [ ] Existing CSV Transformer flow (Process tab, History tab, download) still works.
maxtechera
temporarily deployed
to
staging - aai-unified2-flowise-moonstruck
May 4, 2026 19:24 — with 
Render
Inactive
Render
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🚀 Release: Staging to Production
Release Date: 2026-05-04
Changes in this release
This PR is automatically created/updated when commits are pushed to staging.
Merging this PR will trigger the release workflow to create a new GitHub release.