feat: Lago billing, i18n (ru/de), and universal-integrator tool generator#5764
Closed
AAChibilyaev wants to merge 4904 commits into
Closed
feat: Lago billing, i18n (ru/de), and universal-integrator tool generator#5764AAChibilyaev wants to merge 4904 commits into
AAChibilyaev wants to merge 4904 commits into
Conversation
…ow runs (simstudioai#5062)" (simstudioai#5065) This reverts commit 16e7456.
…nges, code hygiene
…imstudioai#5060) * fix(providers): pin Azure OpenAI/Anthropic endpoints to validated IP (TOCTOU SSRF) * fix(providers): fail closed when Azure endpoint validates without a pinnable IP * refactor(security): drop pinned-fetch agent pool, keep per-call dispatcher * refactor(security): make createPinnedFetch the single pinned-fetch source * refactor(security): drop pinned-fetch agent pool; keep per-call dispatcher
…cern modules (simstudioai#5069) * refactor(table): extract row ordering, executions, and tx helpers from service.ts Move the row position/fractional-ordering internals to rows/ordering.ts, the row-execution (workflow-group result) internals to rows/executions.ts, and the shared tx-timeout helpers to tx.ts. Pure code-motion — verbatim bodies, identical behavior. service.ts: 5324 -> 4442 lines. * refactor(table): extract row CRUD and query into rows/service.ts Move insert/update/upsert/delete/replace/batch row writes, queryRows/ getRowById reads, and findRowMatches into rows/service.ts. Verbatim bodies; consumers repointed; @/lib/table barrel re-exports the new module so callers are unchanged. service.ts: 4442 -> 2788 lines. * refactor(table): extract column/schema management into columns/service.ts Move add/rename/delete column ops and column-type/constraint updates into columns/service.ts. addTableColumnsWithTx stays in service.ts (table-creation primitive) to avoid a cycle. Verbatim bodies; barrel re-exports the module. service.ts: 2788 -> 2149 lines. * refactor(table): extract job state machine and export jobs into jobs/service.ts Move tableJobs reads/mapping, the job lifecycle state machine, and export-job queries into jobs/service.ts. service.ts imports latestJob* one-way for table metadata enrichment (no cycle). Import-data orchestration helpers stay in service.ts for now. Verbatim bodies. service.ts: 2149 -> 1791 lines. * refactor(table): extract workflow-group management into workflow-groups/service.ts Move add/update/delete workflow groups + outputs and pruneStale into workflow-groups/service.ts, preserving the dynamic backfill-runner import (cycle-breaker). Verbatim bodies; no cycle. service.ts: 1791 -> 851 lines. * refactor(table): extract import-job data ops into import-data.ts Move bulk insert, schema setup, and append/replace import operations into import-data.ts (consumed by import-runner + import route). service.ts is now the pure table-entity module (root CRUD + shared lock/column primitives). Verbatim bodies; no cycle. service.ts: 851 -> 664 lines (5324 at start). * refactor(table): restore private visibility and drop dead helpers post-split Un-export DerivedJobFields/JOB_PROJECTION/mapJobRow (file-local in jobs/service.ts, were private pre-split) so they no longer leak into the @/lib/table barrel. Remove dead code carried into the new modules: countTables (createTable does its own inline count check) and the unused buildOrderedRowValues/OrderedRowValue pair. * refactor(table): use absolute imports throughout and fix an orphaned TSDoc Normalize all relative imports under lib/table to absolute @/lib/table/... per the project import rule (the split modules and a few pre-existing holdouts), and relocate the getTableById TSDoc that had drifted above applyColumnOrderToSchema. Comment/import-only; zero behavior change.
…imstudioai#5074) GET /api/credential-sets/invitations returned every pending, unexpired link-only (null-email) invitation across all organizations, including the bearer token. Any authenticated user could enumerate and accept another org's invitation, joining its credential set (cross-tenant access). Scope the listing strictly to invitations addressed to the caller's own email. Open-link invites remain redeemable only via the out-of-band /credential-account/[token] URL.
…ment (simstudioai#5072) * feat(jsm): add Atlassian Assets (Insight/CMDB) tools for asset management Add nine JSM Assets tools so workflows can read and write Atlassian Assets (Insight/CMDB) objects — the foundation for keeping JSM asset tables in sync for software/hardware asset management. Tools (wired into the Jira Service Management block): - jsm_list_object_schemas, jsm_get_object_schema - jsm_list_object_types, jsm_get_object_type_attributes - jsm_search_objects_aql (AQL search with pagination) - jsm_get_object, jsm_create_object, jsm_update_object, jsm_delete_object Each tool proxies through an internal route that resolves the Jira cloudId and the Assets workspaceId, then calls the Assets API via the OAuth 2.0 (3LO) gateway form (/ex/jira/{cloudId}/jsm/assets/workspace/{workspaceId}/v1). Adds the CMDB OAuth scopes to the jira provider (read/write/delete cmdb-object, read cmdb-schema/type/attribute) with descriptions, contract schemas for each route, and block operations/subBlocks/outputs. Bumps the API-validation route baseline for the nine new routes. * refactor(jsm): harden Assets param coercion and response typing - Add toOptionalInt helper so non-numeric pagination inputs never emit NaN into the Assets query string (startAt/maxResults/page/resultsPerPage) - Replace Record<string, any> in mapAssetObject with typed Raw* interfaces * fix(jsm): validate Assets workspaceId and honor `last` pagination flag Address review findings on the Assets tools: - Add validateAssetsWorkspaceId and guard the workspaceId in every Assets route before it is interpolated into the API path (mirrors the existing cloudId guard) — prevents a crafted workspaceId from escaping the workspace-scoped path - Object schema list now falls back to the `last` flag when `isLast` is absent, so pagination doesn't stop early * feat(jsm): allow overriding the auto-resolved Assets workspace Atlassian provisions one Assets workspace per site, so workspace discovery uses values[0] by design. For the rare multi-workspace site, expose an advanced "Assets Workspace ID" override on the block that flows through to every Assets operation, and document the single-workspace assumption. * refactor(jsm): include Assets responses in the JsmResponse union Append the nine Assets tool response types to JsmResponse for completeness and consistency with the rest of the JSM tool surface.
…ai#5049) * fix(uploads): authorize internal file URLs before download downloadFileFromUrl treated any URL containing /api/files/serve/ as trusted-internal and read the object straight from storage by key with no access check, while every other resolution path in the file calls verifyFileAccess. Reachable during workflow execution via file[] inputs (type: 'url'), letting an authenticated user read arbitrary storage objects across tenants by supplying a storage key. Thread the caller's userId into downloadFileFromUrl and run verifyFileAccess(key, userId, undefined, context, false) on the resolved key before downloadFile; fail closed when no userId is present. Update all callers (execution file inputs, tool file outputs, KB ingestion); webhook and chat inputs already thread userId via processExecutionFiles. * chore(uploads): log denied internal file downloads for rollout telemetry * fix(uploads): derive internal file context from key, not query param Cursor Bugbot flagged a context-spoofing bypass: downloadFileFromUrl resolved context via parseInternalFileUrl, which honors a caller-controlled ?context= query param. An attacker could label a private storage key with a world-readable context (profile-pictures/og-images/workspace-logos) so verifyFileAccess short-circuits to granted while downloadFile still reads the private object. Infer context from the key only (inferContextFromKey), mirroring how /api/files/serve resolves it; ignore the query param. Also move the userId guard ahead of key resolution so auth failures surface first. * docs(uploads): move context-derivation rationale into TSDoc * fix(uploads): match internal file marker in URL path only isInternalFileUrl matched the /api/files/serve/ substring anywhere in the string, so a crafted URL could carry it in a query string or fragment and skip DNS/SSRF validation. Match it in the path component only. The raw path is checked without URL normalization on purpose: the files parse route relies on traversal sequences surviving this check (an absolute https://host/api/files/serve/../.. URL must classify as internal so the '..' check rejects it, rather than being normalized to /etc/... and waved through as external). Host is intentionally not gated — cross-tenant reads are prevented at the storage sink by verifyFileAccess, and host-allowlisting would break self-hosted/multi-domain deployments. Adds unit tests. * consolidate access, billing principals --------- Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
* feat(auth): OAuth-only signup with Microsoft provider - Remove email/password form from /signup — Google, Microsoft, GitHub OAuth only - Add Microsoft as a social provider (MICROSOFT_CLIENT_ID / MICROSOFT_CLIENT_SECRET / DISABLE_MICROSOFT_AUTH) - Wire microsoftAvailable through provider checker, API contract, providers route, and all auth UI - Hide "Continue with email" in auth modal signup view; login view unchanged - Fix MicrosoftIcon SVG to use official brand colors and proportions * fix(auth): remove unused useSession, guard invalid-callback warn with ref * feat(auth): gate email signup via DISABLE_EMAIL_SIGNUP flag * feat(auth): restore signup email form gated by NEXT_PUBLIC_DISABLE_EMAIL_SIGNUP * refactor(auth): single DISABLE_EMAIL_SIGNUP env var controls both ui and backend * fix(config): restore isHosted hostname check
…studioai#5041) * feat(db): zero-downtime migration safety lint + db-migrate skill Add scripts/check-migrations-safety.ts (check:migrations), a CI gate that classifies statements in newly-added migrations into hard errors (rewrite), annotate-to-acknowledge contract ops (`-- migration-safe: <reason>`), and backfill warnings. Wire it into test-build.yml. Add the /db-migrate skill as the judgment half (expand/contract phasing, app-code cross-ref, annotation authoring). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(skills): run cleanup and db-migrate safety checks in /ship * fix(db): address review — DROP INDEX lock symmetry, RENAME CONSTRAINT false-positive, alter-type literal match - Non-concurrent DROP INDEX is now a hard error (ACCESS EXCLUSIVE lock), symmetric with CREATE INDEX; DROP INDEX CONCURRENTLY after a COMMIT passes clean. Removes the false-confidence annotate path. - RENAME rule narrowed to RENAME COLUMN / table RENAME TO; RENAME CONSTRAINT and ALTER INDEX ... RENAME (metadata-only) no longer flagged. - alter-type regex now requires TYPE to follow the column identifier, so it no longer matches TYPE inside a string default. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(db): enforce IF EXISTS on DROP INDEX CONCURRENTLY for replay idempotency Symmetric with the CREATE INDEX CONCURRENTLY rule: a post-COMMIT DROP INDEX CONCURRENTLY replays from the top on failure, so without IF EXISTS it aborts re-dropping an already-gone index. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * improvement(skills): gate /ship cleanup on UI changes; default migration base to staging - /ship runs /cleanup only when the diff touches UI code (.tsx or apps/sim/components|hooks|stores); the six passes are React-only. - /ship runs check:migrations against origin/staging (the PR base). - check:migrations default baseRef is now origin/staging instead of origin/main. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(db-migrate): add contract-pending TODO convention for deferred drops Establishes a durable, greppable marker (`contract-pending(<precondition>): ...`) left on the legacy column in schema.ts when an expand defers a drop, so the contract phase doesn't rot. The outstanding-work list is `grep -rn contract-pending`; the contract PR's `-- migration-safe:` annotation references the expand and deletes the marker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…studioai#5075) * fix(webhooks): cap request body size on public webhook receivers Public, unauthenticated webhook endpoints read the entire request body into memory before any lookup or signature verification, letting a caller exhaust pod memory with arbitrarily large bodies. Bound the body via the existing size-limited stream reader (content-length guard + streamed cap) and return 413 on oversize. Applies to parseWebhookBody (trigger receiver) and the agentmail route. Cap defaults to 10 MB, overridable via WEBHOOK_MAX_REQUEST_BYTES. * refactor(webhooks): extract shared body-size cap to constants module Address review feedback: hoist WEBHOOK_MAX_BODY_BYTES into a single lib/webhooks/constants.ts so the trigger receiver and AgentMail route share one source of truth instead of recomputing the env-derived cap (prevents drift). Also drop the redundant request clone when the body stream is null. * refactor(webhooks): drop redundant null-body branch in capped readers Both capped body readers had an `if (!stream)` fallback to an uncapped `.text()`/empty string. `readStreamToBufferWithLimit` already returns an empty buffer for a null stream, so the branch is redundant and the `.text()` fallback was a theoretical bypass (chunked request, no content-length, null body). Collapse both to a single capped read. * chore(webhooks): drop inline comments from capped body readers
…ai#5077) Validate the user-supplied vLLM endpoint (request.azureEndpoint) against the central SSRF guard and pin the connection to the resolved IP before issuing any request, mirroring the Azure OpenAI/Anthropic providers. The operator-configured VLLM_BASE_URL stays trusted and unvalidated.
…i#5078) Pass allowHttp to validateUrlWithDNS so plain-HTTP self-hosted vLLM endpoints are permitted. This only relaxes the protocol check; the private/reserved-IP blocklist and blocked-port checks still apply, so SSRF protection is unchanged.
…i#5059) * feat(feature-flags): AppConfig-backed gated feature flags * fix(ci): repoint 'Validate feature flags' step to env-flags.ts after rename * improvement(feature-flags): drop in-code defaults; fallback resolves a per-flag secret, gating is AppConfig-only * improvement(feature-flags): make flag names a closed set so every flag requires a fallback secret * improvement(feature-flags): single FEATURE_FLAGS registry — each entry defines name, description, and fallback in one place * improvement(feature-flags): fallback is the env secret key (keyof typeof env), resolved to a boolean
…ct-point tools (simstudioai#5082) * feat(grafana): validate integration and add folder, health, and contact-point tools - Require alert-rule title/ruleGroup/data in the block (create would 400 without them) - Trim UID path params across dashboard and alert-rule tools to avoid copy-paste 404s - Use Grafana brand color for the block background - Surface previously-unsettable list params (limit, starred, annotation type) - Add get/update/delete folder, check data source health, get health, and create contact point tools - Strip non-TSDoc comments; regenerate docs and integrations.json * fix(grafana): scope block param remaps per operation to prevent cross-operation leaks
…tudioai#5076) * refactor(connectors): split client metadata from server runtime + cover node:net in client bundle The browser build broke with `Cannot find module 'node:net'`. Server-only SSRF code in `input-validation.server.ts` (`dns/promises`, and since PR simstudioai#5060 `undici` → `node:net`/`node:tls`) is statically reachable from the client bundle via the tool/connector registries, which the workflow editor imports for metadata. Node networking builtins have no browser shim, so Turbopack cannot compile them for the client. Two changes: 1. Split each connector's client-safe declarative metadata into a sibling `meta.ts` (`<name>ConnectorMeta`), mirroring the `XBlockMeta` / `BLOCK_META_REGISTRY` pattern. `connectors/registry.ts` is now the client-safe `CONNECTOR_META_REGISTRY` (+ `getConnectorMeta` / `getAllConnectorMeta`); the full registry with runtime fns moves to `connectors/registry.server.ts`. Client components consume the meta registry; the sync engine and knowledge API routes consume the server registry. This removes connectors from the client's server-only graph. Connector metadata is byte-for-byte identical before/after; runtime fns are untouched. 2. Extend the existing simstudioai#4899 `turbopack.resolveAlias` browser stub — which already mapped `dns`/`dns/promises` to an empty module for the browser — to also cover `net`/`tls` (+ `node:` variants), since `undici` now pulls those in. The remaining tool/provider definitions still reach `input-validation.server` server-side; the browser-only stub keeps those Node builtins out of the client bundle while the real modules stay on the server, so SSRF validation and IP pinning are unaffected. Connector authoring/validation skills updated to teach the meta.ts split. * fix(icons): use Square logo glyph only, drop wordmark * fix(connectors): share Discord max-messages default across meta and runtime Discord defined DEFAULT_MAX_MESSAGES separately in meta.ts (config placeholder) and discord.ts (sync behavior), which could drift. Export it from meta.ts and import it in the runtime, matching the single-source pattern used by the other connectors (e.g. gmail, intercom). * refactor(tools): route grafana/agiloft egress server-side, drop SSRF browser shim Move the server-only SSRF-pinned fetch out of the grafana (update_dashboard, update_alert_rule) and agiloft (11 record/search tools) definitions and into internal API routes, the same pattern the rest of the server-side tools (and agiloft's own attach/retrieve) already use. The tool definitions are now purely declarative (request → internal route), so they no longer import `input-validation.server` and the tools registry is fully client-safe. With connectors (meta split) and these tools no longer reaching server-only code from the client bundle, the browser no longer pulls in `dns`/`net`/`tls`: - Add `import 'server-only'` to `input-validation.server.ts` so any future client import fails loudly at build time instead of silently bloating the bundle. - Remove the `turbopack.resolveAlias` browser stub and delete `empty-node-fallback.browser.ts` — the root cause is fixed, the shim is gone. Behavior is unchanged: each route runs the exact merge/validation/fetch logic the tool ran before (every header, param branch, JSON-parse guard, error string, and SSRF pinning preserved); only the location of execution moved from the client- bundled definition to a server route. * fix(connectors): move onedrive tagDefinitions into meta; drop server-only guard - onedrive's tagDefinitions lived in the runtime file, so the client meta registry returned undefined for it and the add-connector tag opt-out section stopped rendering for onedrive. Move it into meta.ts like the other connectors so client and server see identical metadata (verified across all 50). - Remove the 'server-only' import from input-validation.server.ts: the meta/route split already keeps it out of the client bundle, and blocks/tools registries don't use the guard either. * fix(grafana): surface upstream error when the prefetch GET fails Check response.ok on the existing-resource GET in both update routes and return the upstream status/body, matching how the tool framework surfaced GET errors before the move to internal routes (the framework checks response.ok before transformResponse). Without this, a failed prefetch produced a generic 'Failed to fetch existing ...' message and dropped Grafana's error detail. * fix(grafana): reject invalid panels JSON instead of silently ignoring it Grafana's dashboard API treats panels as a required array and returns 400 on invalid JSON; this route already errors on every other JSON param. Return 'Invalid JSON for panels parameter' instead of swallowing the parse error and proceeding with a misleading success. * fix(grafana): trim dashboard/alert-rule UID in route URLs (carry over simstudioai#5082) PR simstudioai#5082 added .trim() on dashboardUid/alertRuleUid in the original tool URL builders. Those tools now build their URLs in the internal routes, so apply the same trim there to preserve that behavior. * fix(grafana): route update_folder egress server-side (carry over simstudioai#5082) simstudioai#5082 added a grafana update_folder tool that does SSRF-pinned fetch in its postProcess, re-introducing the client-bundle leak. Convert it to the internal API route pattern like the other update tools so the def is declarative and input-validation.server stays out of the client bundle. * fix(grafana): surface route failures in transformResponse instead of masking them The grafana update tools' transformResponse hardcoded success: true and dropped the route's error, so an upstream/validation failure (HTTP 200 with { success: false, error }) was reported to the workflow as a success. Forward data.success and data.error (matching the agiloft tools) so failures propagate as before the move to internal routes.
…flags (simstudioai#5086) * feat(feature-flags): migrate 3 env-flags to AppConfig-backed runtime flags * fix(feature-flags): hardcode workflow-columns on, fix feature-flags tests * chore(feature-flags): document mothership-beta userId targeting limitation
…lendar + sharing tools (simstudioai#5084) * feat(google-calendar): wire freebusy, align tools with API v3, add calendar + sharing tools * fix(google-calendar): address review — trust offset timezones, make list_acl showDeleted usable, harden unshare error parse, clarify update attendees * fix(google-calendar): wire list q/pageToken into block, harden invite PUT error parse * fix(google-calendar): make list orderBy user-selectable, clarify update timeZone applies to start/end * fix(google-calendar): require timeZone for recurring timed events, clarify recurrence replace semantics * fix(google-calendar): validate grantee before building share ACL body Throw a clear error when scopeType is user/group/domain but scopeValue is missing or blank, instead of POSTing a scope-type-only body that the Calendar ACL API rejects with an opaque error.
simstudioai#5070) * improvement(perm-groups): allow workspace filter for permission groups * show errors correctly * address comments * address concurrent edit concern * address locks * address comments" * index migration safety * address at route level
…oai#5091) * fix(scheduled-tasks): fix schema rejection * fix(db): fix duplicate db query
…lassian assets and google calendar tools
…ing streaming (simstudioai#5093) * fix(chat): keep autoscroll pinned when the virtualizer re-scrolls during streaming The sticky-scroll detach heuristic (scrollTop drops while scrollHeight doesn't grow) could not distinguish a user scrollbar drag from a programmatic scroll. react-virtual re-pins content by moving scrollTop whenever a measured row's size changes — including the transient height shrinks streamdown emits as it re-parses each streaming token — so the hook misread those upward programmatic scrolls as the user scrolling away and detached mid-stream. Gate the scroll-delta detach branch behind a genuine recent user gesture (pointerdown/up tracking + wheel/touch/keydown stamp). Programmatic scrolls have no preceding gesture, so they no longer detach; scrollbar drag, wheel, and keyboard detach are preserved. * fix(chat): address review — reset pointer ref on teardown, stop wheel/touch opening detach window - Reset pointerDownRef in effect cleanup so a pointer held through teardown (e.g. dragging the scrollbar as a stream finishes) can't leak a stuck-true ref into the next session and detach on the first programmatic re-pin. - Wheel-up and touch-drag already detach directly, so the onScroll delta heuristic only needs to authorize scrollbar drag (pointerDownRef) and keyboard. Stop stamping the gesture window on wheel/touch, which otherwise let a harmless downward wheel open a 250ms window where a virtualizer shrink could falsely detach. * fix(chat): scope detach authorization to real scroll gestures; TSDoc comments - onPointerDown only marks an active drag when the press targets the scroll container itself (the scrollbar), not its content, so a text-selection drag on a message can't authorize a detach during a programmatic re-pin. - Reset lastUserGestureAtRef on teardown alongside pointerDownRef so neither a held pointer nor a late keydown can leak across streaming sessions. - Convert the hook's inline comments to TSDoc on the relevant declarations per codebase conventions. * fix(chat): only upward scroll keys authorize a keyboard detach onKeyDown stamped the gesture window on any bubbling key, so an unrelated keypress within USER_GESTURE_WINDOW of a programmatic virtualizer re-pin could satisfy userDriven and detach mid-stream. Filter to the upward scroll keys (ArrowUp, PageUp, Home, Shift+Space), mirroring the wheel handler's upward-only rule, so only a genuine upward keyboard scroll authorizes detach.
… and remote URLs (simstudioai#5092) * feat(providers): support large agent-block attachments via Files APIs and remote URLs Agent-block file uploads were inlined as base64 with a hard 10MB cap. Files above the threshold now use each provider's native large-file path: - OpenAI / Gemini: upload to the provider Files API, reference by file_id/uri - Anthropic: GA url content-block source (no Files API beta, no upload) - OpenRouter/Groq/Together/Baseten/xAI/vLLM: remote signed URL in image_url/file - Limits live per-provider in models.ts; the agent block + /models page reflect them Files <=10MB keep the identical base64 path (zero regression). Server-only file handles are stripped from untrusted input to prevent SSRF. * fix(providers): clear forged file handles for inline providers too attachLargeFileRemoteUrls early-returned for inline-strategy providers before clearing server-only handle fields, so a forged remoteUrl on an inline-provider file could still reach a builder (e.g. buildOpenAICompatibleChatContent for mistral/ollama). Clear the handles for every provider before the strategy check. * fix(providers): correct OpenAI expiry serialization and Anthropic large-text-doc handling - OpenAI upload now uses the SDK (client.files.create) so expires_after is serialized as a real nested object; the prior expires_after[anchor] bracket FormData keys were ignored by OpenAI's server, leaving files un-expiring. - Anthropic url document source only supports PDFs/images; large non-PDF text docs now throw a clear error instead of emitting an unsupported url source. - Warn when an oversized file can't be sent because cloud storage is unavailable. * fix(providers): harden large-file path (SSRF fetch, ceiling gate, per-file UI limit) - Download files for OpenAI/Gemini uploads via validateUrlWithDNS + IP-pinned fetch so a forged URL can't reach internal addresses (covers all callers). - Reject files above the provider ceiling before downloading/uploading. - UI now validates each file against the provider's per-file ceiling instead of summing all files against it, matching server-side per-file validation. - Lower Anthropic ceiling to 50MB (documented 32MB request cap / page limits). * refactor(providers): read files-api upload bytes via storage SDK Read OpenAI/Gemini upload bytes through downloadFileFromStorage instead of HTTP-fetching the presigned URL. Removes any server-side URL fetch (no SSRF vector) and works with internal object storage (e.g. self-hosted MinIO), which an IP-pinned URL fetch would have blocked. * docs(providers): clarify files-api bytes are read from storage at upload time * fix(providers): enforce access checks and strip forged ids in the upload path uploadLargeFilesToProvider runs on raw request messages for every caller (incl. the internal providers passthrough), so harden it independently of the agent path: - verifyFileAccess on each file's storage key before reading its bytes, so a forged key can't exfiltrate another user's file. - clear any inbound providerFileId/providerFileUri up front (legit ids are only set by the upload itself), so a forged id can't reference a file in a hosted account. * fix(providers): resolve UI attachment limit with the same model->provider helper as execution The file-upload control imported getProviderFromModel from @/providers/models, but the execution path and every other consumer use the one in @/providers/utils (runtime registry + reseller patterns). Align the UI so its size cap can't disagree with server-side validation for reseller or dynamically-listed models. * test(providers): add new models.ts exports to provider mocks attachments.ts now reads getProviderFileAttachment / INLINE_ATTACHMENT_MAX_BYTES from @/providers/models; the provider unit tests that fully mock that module need both exports or attachments.ts fails to load. * fix(providers): guard Gemini upload response name before polling ai.files.upload returns name as string | undefined; guard it (instead of an as-string cast) so a missing name surfaces a clear error at the upload site rather than an opaque files.get failure on the first poll. * fix(uploads): type the file-handle key list so omit preserves UserFile fields The 'as const' readonly tuple widened omit's K to all keys, collapsing Omit<UserFile, K> to {} and failing the production build's type check. Declare the array as Array<keyof handle fields> so K is the precise literal union. * refactor(providers): run handle-clear + URL-mint in executeProviderRequest for all callers Move attachLargeFileRemoteUrls out of the agent handler and into executeProviderRequest (right before uploadLargeFilesToProvider), so every entry point — including the internal providers passthrough — clears forged handles and mints/access-checks large-file URLs uniformly. The agent handler now only hydrates base64; its missing-file guard exempts large files (resolved downstream). * fix(azure-openai): guard optional attachment dataUrl in inline image part PreparedProviderAttachment.dataUrl is now optional (large files carry a handle instead); azure-openai builds chat content inline and assigned it directly to a required url field, failing the production build's type check. * fix(providers): upload OpenAI files via multipart and fix Buffer Blob part The installed openai SDK (4.104) does not type expires_after on files.create, so upload via POST /v1/files directly with the documented expires_after[...] form fields (gives the file an auto-expiry). Also wrap the storage Buffer in a Uint8Array for the Blob, which the production build's stricter lib types require. These two type errors were masked locally because tsc was OOMing silently without the type-check script's --max-old-space-size flag. * fix(providers): forward userId from the providers API to executeProviderRequest Large-attachment prep now needs request.userId for presigned URLs and access checks; the authenticated providers proxy has auth.userId but wasn't passing it, so oversized attachments failed for logged-in callers. Forwarding it makes large files work there and keeps the access check (verifyFileAccess) intact. * fix(providers): fail clearly when a large attachment has no cloud storage The doc claimed a base64 fallback that doesn't exist — above the inline cap there is no base64, so without cloud storage the file previously reached the builder and died with a generic read error. Throw a clear 'requires cloud file storage' error at the point of detection and correct the doc.
…turn options in view (simstudioai#5094) * fix(chat): align scrollbar/keyboard detach with wheel/touch re-engage threshold The onScroll detach branch set only stickyRef.current = false, leaving userDetachedRef false, so a scrollbar-drag or keyboard detach kept the lenient 30px (STICK_THRESHOLD) re-engage threshold instead of the strict 5px (REATTACH_THRESHOLD) used after wheel/touch. A programmatic virtualizer re-pin landing within 30px could then snap autoscroll back on right after the user deliberately scrolled away. Reuse the detach() helper so all detach paths set userDetachedRef consistently. * fix(chat): keep end-of-turn options in view after streaming When a stream ends, the suggested-follow-up options and the actions row (gated on !isStreaming) mount, but the virtualizer's getTotalSize — which drives the scroll container's scrollHeight — only catches up a frame or two later via its ResizeObserver. The single scrollToBottom() on effect teardown therefore landed on a stale, too-short bottom and the options were clipped behind the input. (Pre-virtualization this worked because scrollHeight reflected the new rows immediately.) Extract the rAF follow loop already used for CSS height animations into a shared followToBottom(window) helper and run it for a short settle window on teardown, so the bottom is chased until the virtualizer re-measures. The follow is self-interrupting — height growth leaves scrollTop where we put it, while a user scroll moves it up, so it bails the instant the user scrolls and never fights a real gesture even with listeners torn down.
…h() calls (simstudioai#5419) PR simstudioai#5399 fixed the callback route forgetting to pass fetchFn into the SDK's auth(), but every call site (start + callback routes) still imported the raw SDK auth() directly and had to remember to pass fetchFn: createSsrfGuardedMcpFetch() by hand — the same omission was possible again at any future call site. Add mcpAuthGuarded() in lib/mcp/oauth/auth.ts, a thin wrapper around the SDK's auth() that always defaults fetchFn to the SSRF-guarded fetch (still overridable for tests). Both routes now import mcpAuthGuarded from @/lib/mcp/oauth instead of the raw SDK auth, so omitting the guard is no longer possible by omission. (cherry picked from commit 278c8790b2763360174c417cb565e05b95050dc3)
…ai#5423) Spread order previously let an explicit fetchFn (including fetchFn: undefined) in options silently disable the SSRF-guarded default. Fallback is now applied after the spread so the guard always wins unless a real override is passed. fix(tools): handle non-numeric Drive file size in early size check Guard the pre-download size check against a malformed metadata.size string so it's skipped explicitly instead of relying on an incidental NaN no-op; the streaming cap on the actual download still enforces the limit either way. (cherry picked from commit 85588a3d67d616ef9e204551af7152471edd1dd2)
* v0.6.29: login improvements, posthog telemetry (simstudioai#4026) * feat(posthog): Add tracking on mothership abort (simstudioai#4023) Co-authored-by: Theodore Li <theo@sim.ai> * fix(login): fix captcha headers for manual login (simstudioai#4025) * fix(signup): fix turnstile key loading * fix(login): fix captcha header passing * Catch user already exists, remove login form captcha * fix(attachments): cross tenant security hardening * address comments * fix * fix * remove dead code --------- Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Theodore Li <theodoreqili@gmail.com> Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com> Co-authored-by: Theodore Li <theo@sim.ai> (cherry picked from commit 60648615bfc7a5669c0b91b52b18b7d3a6876d29)
…imstudioai#5432) Claude-Session: https://claude.ai/code/session_011Lmib23o5aQtBr7ZPhgpgf Co-authored-by: Claude Fable 5 <noreply@anthropic.com> (cherry picked from commit aa1f8ea09fef8fd6d92049c44ccb6ed2885ccaa2)
…requests (simstudioai#5496) * fix(api-block): stop spoofing a browser fingerprint on outbound HTTP requests getDefaultHeaders() sent a Chrome User-Agent, a Referer pointing at Sim's own app, and mismatched Sec-Ch-Ua client hints on every API block request. Atlassian's Jira/Confluence Cloud REST API (and other browser-aware anti-CSRF/bot-defense layers) reject requests carrying a browser User-Agent with 403 "XSRF check failed", even with a valid Basic-auth header and X-Atlassian-Token: no-check set. Default headers now identify honestly as Sim, matching how other HTTP clients (curl, Postman, axios) behave. User-supplied headers still override any default. * fix(testing): sync shared tool-tester mock headers with new defaults createMockHeaders in the shared @sim/testing tool-tester builder still hardcoded the old Chrome UA / Referer / Sec-Ch-Ua fallback values. It was unreachable in current tests but would silently diverge from production getDefaultHeaders() for any future test hitting its fallback path. (cherry picked from commit 49e7b146a3e9c9beee6cda5dce5a6f24dc4d26d1)
…imstudioai#166/simstudioai#167) (simstudioai#5508) Patches a high-severity stored XSS in the oidc-provider and mcp plugins via javascript:/data: redirect_uri schemes. Also bumps @better-auth/sso and @better-auth/stripe to matching 1.6.13 peers. Patch-only release (1.6.11 -> 1.6.12 -> 1.6.13), no breaking changes in either changelog. (cherry picked from commit e8f716399beff067451cc5e8c0a8e3cb30a8fae4)
Partial backport of upstream 4aef0f7707 (sim v0.7.29) — only the createChunk race fix. Two overlapping workflow runs appending to the same shared KB document could both compute the same next chunk_index and collide on the (document_id, chunk_index) unique constraint. createChunk now serializes concurrent writers via a transactional pg_advisory_xact_lock keyed on the document, bounded by a 5s lock_timeout. The other half of the upstream commit (checkAndIncrementStorageUsageInTx, atomic storage-quota check-and-increment for KB document uploads) is intentionally NOT included: it depends on upstream simstudioai#5269 (extending storage metering to KB documents), which this fork never adopted — KB uploads here still don't check storage bytes at all, so there's no check-then-increment race to close on that path. The equivalent race does exist, unfixed, in lib/uploads/contexts/workspace/workspace-file-manager.ts (checkStorageQuota followed by a later non-atomic incrementStorageUsage with no transaction) but that's a separate fix upstream never wrote, not a portable commit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTj7HhYJJdr7L3jnePkzJk
…utes (simstudioai#5601) * fix(api): bound request-body reads on speech/knowledge-chunks/help routes Replace unbounded request.json()/formData() reads with the existing size-limited helpers, and move auth ahead of the body read on the knowledge chunks route so unauthenticated callers can't force a large allocation before being rejected. * fix(knowledge): reject non-string workflowId instead of silently skipping authorization A truthy non-string workflowId previously fell through the type guard and skipped the workflow-scoped write authorization entirely. Validate the type explicitly and fail closed with a 400 instead. (cherry picked from commit fcf421272da6a08d0c44eb4df1a73e2ed5960eac)
…XSS (simstudioai#5599) Sanitize anchor hrefs rendered by docx-preview after render, stripping any scheme outside http/https/mailto (same allowlist already used by the PPTX renderer). Covers both the workspace file viewer and the unauthenticated public share page, which reuse the same component. (cherry picked from commit 1ac42c7a319daa834d16264dd3562e6166dd2b22)
simstudioai#5600) PUT verify no longer trusts a stale authType at cookie-mint time — it now re-checks the chat is still email-auth before issuing the cookie, matching the existing POST guard and the public-file OTP route. (cherry picked from commit 0d75d1b071ef7e97c6be9a86e847276f7c34496c)
…ads (simstudioai#5604) * fix(files-upload): enforce workspace authorization on mothership uploads The mothership context in POST /api/files/upload skipped the workspace permission and storage quota checks that every sibling context enforces, letting a caller write files into a workspace they have no access to. * fix(files-upload): check mothership quota once against the full batch Resolve the mothership permission and quota check once per request (mirroring the existing execution-context pattern) instead of per file: a per-file quota check let a multi-file batch exceed the caller's quota since each file's own size fit even when the combined total did not. Also corrects the missing-workspaceId error message, which named the chat context instead of mothership. (cherry picked from commit b6835b48157bf70ce0d94ee359b142833595605c)
…d excludes (simstudioai#5523) (cherry picked from commit 44faebd7e7d8bc5052846c4b49f5a205f56243cd)
Merges upstream changes including: - Security hardening (SSRF/input validation/bounds checking) - Better Auth upgrade (1.6.11 -> 1.6.13) - File upload/download improvements - MCP OAuth fixes and SSRF guards - API contracts and validation improvements - Knowledge connector concurrency bounds - Supabase storage tool enhancements Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Quote frontmatter descriptions containing unescaped colons, and escape literal curly braces in JSON example tables so the MDX/acorn parser doesn't choke on them.
…seRequest Add createBillingCheckoutContract and lagoWebhookContract for two previously-uncontracted billing routes. Migrate the MCP server refresh/test-connection routes from a local params schema + safeParse to full contract-bound parseRequest, and annotate the Lago webhook's raw body read (HMAC verification requires the raw body before parsing).
The generator's template literals emit apps/sim-style '@/' imports as the CONTENT of generated tool/block/trigger files, not real imports of packages/universal-integrator into apps/sim. The boundary checker's naive text scan was flagging those literal template strings; skip its four known generator source files instead of rewriting the generated code to violate the apps/sim absolute-import convention. Also ignore .leankg/ and .omo/ local tool artifacts.
…empty types
subscription.ts had two colliding declarations of
billingCheckoutBodySchema/BillingCheckoutBody/createBillingCheckoutContract
(one predating the Lago work, one added alongside it), causing a
use-before-declaration/redeclare error. Keep the later, more complete
version and fold in the older copy's seats cap and annual field.
Replace telegram/types.ts's banned `{}` type on five genuinely-empty
Telegram webhook payload shapes with Record<string, never>, which still
rejects unexpected properties.
Fix a type-invalid MCP test-connection error branch to use the standard
`if (!parsed.success) return parsed.response` pattern. Remaining diffs
are Biome import-order autofixes.
Fix all 21 docs#build failures reported by the full build, plus two
more of the same class the build hadn't reached yet:
- 3 more YAML frontmatter descriptions with unescaped colons, quoted.
- 18 MDX/JSX parse errors from unescaped literal braces, angle-bracket
placeholders, raw emails, and inline key=value text in prose/table
cells — escaped braces as {'{'}/{'}'}, wrapped placeholders/emails in
backtick inline code, matching the existing convention in this tree.
- agents/mcp.mdx: the auto-translation pipeline had translated two JSX
component tag names themselves (<Video> to <Видео>, <Image> to
<Изображение>) instead of just their content, breaking the reference
to the actual imported component.
- integrations/wiza.mdx: a bare `{ total, company_credits }` object
shorthand in prose was valid-but-undefined JS once MDX parsed it as
an expression; escaped as literal braces.
Verified with a full `bun run build` (sim + docs) against a local
Postgres — all 4/4 tasks pass, 4252 static pages generated.
SIM_AGENT_API_KEY held a real credential instead of the empty-string placeholder every other secret field in this example file uses. The key itself has already been purged from this branch's git history via git-filter-repo; this commit brings the working tree in line with that history and the file's own convention.
|
@AAChibilyaev is attempting to deploy a commit to the Sim Team on Vercel. A member of the Team first needs to authorize it. |
Brings in 452 upstream commits (v0.7.15 → v0.7.39): Mothership v0.8, custom blocks, workspace forking, new integrations (ClickUp, Rocketlane, GitLab, Instagram, TikTok, Buffer, Flint, Jupyter, AppSheet), new providers (Kimi, NVIDIA, Z.ai, grok-4.5, GPT-5.6, Sonnet 5, Fable 5), emcn package extraction, TypeScript 7, rich markdown editor, and all security hardening. Fork-specific work preserved: Lago billing (provider flag, wallet, usage events, migration 0250 with upstream migrations renumbered 0251-0264), i18n (ru/de, NextIntl provider, language switcher re-added to the new server-component navbar), 12000-range ports, yandex_music integration re-registered in registry-maps. Conflict policy: upstream taken for refactored surfaces (i18n string wrapping to be re-extracted via bun run i18n:all), both sides combined for billing/copilot imports, threshold billing gated behind isStripeBillingProvider. Merged with a temporary replace-graft to restore the true fork-point merge-base (613e8ea) after an earlier history rewrite. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTj7HhYJJdr7L3jnePkzJk
Contributor
|
Too many files changed for review. ( |
…ocale keys - Point fork components (wallet-section, language-switcher, billing-provider-badge, integration-block-fallback) at @sim/emcn after upstream extracted the design system into a package - Restore isBillingEnabled import in auth.ts, drop removed workspaceId arg from checkActorUsageLimits call, add description field to Lago invoice mapping to match upstream's InvoiceItem contract - Remove ru email subject for the deleted polling-group-invitation type - Pin docs to root-hoisted zod so ai-sdk tool() typings resolve one zod - Lint auto-fixes across merged files Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTj7HhYJJdr7L3jnePkzJk
…tl context - Pin upstream-authored billing tests (invoices, switch-plan, billing UI) to the Stripe provider so a developer .env with BILLING_PROVIDER=lago cannot flip their code path - Mock next-intl with the real en catalog in tests rendering i18n-wrapped components (special-tags, invite, billing, logs/error — fixing a pre-existing failure too) - Add fork-introduced exports to test mocks: envNumber, getMaxExecutionTimeout, credentialTypeEnum, usePurchaseCredits, ChipInput/toast - Revert MCP refresh route to upstream's contract-schema validation: the fork's parseRequest migration read request.nextUrl, which crashes on plain Request objects and 500'd the route under test - Drop workspaceId arg expectation to match upstream's checkActorUsageLimits signature Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTj7HhYJJdr7L3jnePkzJk
- Narrow the threshold-settlement guard from !isStripeBillingProvider to isLagoBillingProvider so billing-disabled test environments preserve upstream's error-propagation semantics; Lago deployments still skip Stripe settlement - Complete the env mock (getEnv/isTruthy/isFalsy) in threshold-billing and add credentialTypeEnum to the invitations route schema mock Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTj7HhYJJdr7L3jnePkzJk
- universal-integrator: use getErrorMessage from @sim/utils/errors instead of inline instanceof-Error message extraction; add @sim/utils dependency - Bump route-count baseline 964 -> 966 for the fork's Lago billing routes on top of upstream v0.7.39 (strict audit passes) - Dedupe toast key in the billing test emcn mock Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTj7HhYJJdr7L3jnePkzJk
External links use /ru-style prefixes but locale is cookie-based; the proxy now strips the prefix, sets NEXT_LOCALE, and redirects. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JTj7HhYJJdr7L3jnePkzJk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three largely independent efforts developed together in one branch:
scripts/i18n-translate/) and auseBlockTextlayer for localizing block/connector definitions at render time.packages/universal-integrator) that scaffolds Sim tool/block/trigger integrations from API docs; used to generate the Stripe, Telegram, Yandex Music, and Yandex Metrica integrations.Validation
All of the following pass locally on this branch:
bun run check:boundariesbun run check:realtime-prunebun run check:api-validationbun run type-check(17/17 workspaces)bun run lint(17/17 workspaces)bun run build(4/4 tasks — sim, docs, simstudio, simstudio-ts-sdk; docs build verified against a local Postgres, all 4252 static pages generate)Notes for reviewers
mainpredates a large amount of upstream history (~450 commits), so it has not been rebased/merged onto latestmain— GitHub's three-dot diff should still show only this branch's actual changes, but a conflict-free merge is not guaranteed and may need a rebase pass.helm/sim/examples/values-copilot.yamlin this branch's history has been purged viagit-filter-repobefore this push (replaced with the same empty-string placeholder every other secret field in that file uses). The original key should be treated as compromised and rotated by its owner if that hasn't happened already.🤖 Generated with Claude Code