fix(search): verify federated connectors and align setup docs

Author: icecrasher321

.agents/skills/add-connector/SKILL.md

@@ -6,6 +6,12 @@ argument-hint: <service-name> [api-docs-url]
 
 # Add Connector Skill
 
+## Choose the connector runtime first
+
+For **Sim Search**, use the live provider workflow in [the federated Search developer guide](../../../apps/sim/lib/sim-search/live/README.md#adding-a-live-search-connector). Its browser-safe provider catalog owns provider IDs, API origins, credential aliases, and account modes; its typed runtime registry requires both search and read handlers. `ConnectorMeta` remains the owner of logos and setup fields. Member mode has no admin resource filters. Service mode requires independent live source verification and shared selectors. Do not implement a Search source by adding a crawler, embeddings, or a scheduled ACL build.
+
+The ingestion instructions below apply to **ordinary knowledge-base connectors** and the explicit legacy Search backend (`SIM_SEARCH_LIVE=false`). If a provider supports both, implement and test both runtimes; adding `search: true` to metadata alone does not implement federated search. Preserve indexing documentation and behavior for those KB/legacy callers.
+
 You are an expert at adding knowledge base connectors to Sim. A connector syncs documents from an external source (Confluence, Google Drive, Notion, etc.) into a knowledge base.
 
 ## Your Task

.agents/skills/validate-connector/SKILL.md

@@ -6,6 +6,12 @@ argument-hint: <service-name> [api-docs-url]
 
 # Validate Connector Skill
 
+## Identify the runtime under review
+
+For **Sim Search**, validate the [live provider registration and access pipeline](../../../apps/sim/lib/sim-search/live/README.md#adding-a-live-search-connector): catalog and metadata parity, both search/read handlers, current member grants, service-source restrictions, safe scoped references, pagination, provenance, and provider failure behavior. Test real localhost setup/search/read with authorized fixtures when available, and distinguish those results from mocked provider tests. Live Search must not enqueue content indexing or background ACL/directory builds; GitLab still computes request-time permissions or uses current CSV grants.
+
+The ingestion-specific checks below apply to ordinary workspace KB connectors and legacy Search selected with `SIM_SEARCH_LIVE=false`. Keep those checks for providers supporting both runtimes; do not require a live-only provider to implement content hashes, ingestion cursors, embeddings, or stored ACL snapshots.
+
 You are an expert auditor for Sim knowledge base connectors. Your job is to thoroughly validate that an existing connector is correct, complete, and follows all conventions.
 
 ## Your Task

apps/docs/content/docs/integrations/atlassian-service-account.mdx

@@ -98,4 +98,4 @@ The credential appears on all three integration pages, but adding it only valida
 | Token expires or needs different scopes | Create a replacement token, add it as a new Sim service-account credential, and select it in the affected workflows. Test them before revoking the old token. Sim does not refresh pasted API tokens. |
 | Data Center or Server host | This credential supports Atlassian Cloud only. |
 
-For a Search source, follow [Confluence's indexing-account replacement steps](/search/confluence#using-a-service-account).
+For a Search source, follow [Confluence's service-account setup and replacement steps](/search/confluence#using-a-service-account).

apps/docs/content/docs/knowledgebase/connectors.mdx

@@ -12,6 +12,8 @@ For organization Search with each person's source permissions, use the [Search c
 
 Connectors continuously sync documents from external services into your knowledge base, so you never have to upload files manually. New content is added, changed content is re-processed, and deleted content is removed — all automatically.
 
+This page covers indexed workspace knowledge bases. [Organization Search](/search) has a separate live-provider catalog and queries its supported providers on demand; adding a knowledge-base connector does not automatically enable it in Search.
+
 ## Available Connectors
 
 <Image src="/static/connectors/connectors-sources.png" alt="The current Connect Source picker showing searchable connectors including Airtable, Asana, Ashby, Azure DevOps, Bitbucket, Box, and ClickUp" width={750} height={680} className="mx-auto h-auto w-full max-w-md" />

apps/docs/content/docs/knowledgebase/export-import.mdx

@@ -33,7 +33,7 @@ An export never includes:
 - Who uploaded each document
 - Internal storage keys
 
-Organization-wide search indexes cannot be exported.
+Live [Organization Search](/search) queries providers directly and has no index to export. If an operator selects the legacy indexed Search backend, its organization index is not included in a workspace knowledge-base export.
 
 ## Vectors
 

apps/docs/content/docs/mcp/index.mdx

@@ -104,6 +104,6 @@ an app to reads.
 
 This server is for operating Sim. Two other MCP features do different jobs:
 
-- [Search MCP](/search/mcp) searches your organization's indexed sources.
+- [Search MCP](/search/mcp) searches your organization's live provider sources with current access checks.
 - [MCP deployment](/workflows/deployment/mcp) exposes your own workflows as
   tools, and [MCP tools](/agents/mcp) connect external servers to Sim agents.

apps/docs/content/docs/platform/connected-accounts.mdx

@@ -1,11 +1,11 @@
 ---
-title: Connected accounts
+title: Credential Groups
 description: Collect accounts in one organization credential group and control which workspaces can use them.
 ---
 
 import { Callout } from 'fumadocs-ui/components/callout'
 
-**Connected accounts** collects people's accounts in one shared pool for your organization, also called a **credential group**. An organization admin chooses the providers, invites people to connect, and allows specific workspaces to use the pool. Workflows in those workspaces can find an account by email through the [Credential block](/workflows/blocks/credential).
+**Credential Groups** collects people's accounts in one shared pool for your organization, also called a **credential group**. An organization admin chooses the providers, invites people to connect, and allows specific workspaces to use the pool. Workflows in those workspaces can find an account by email through the [Credential block](/workflows/blocks/credential).
 
 Each organization has at most one pool. Creating it does not give any workspace access; the workspace allowlist starts empty.
 
@@ -15,11 +15,11 @@ Connected accounts must be enabled for your organization. Sim Cloud also require
 
 For self-hosted deployments using environment-based feature flags, set `CREDENTIAL_GROUPS=true`. Availability is organization-scoped; personal workspaces cannot use an organization pool.
 
-The setup instructions below describe **Settings → Connected accounts**, shown when Knowledge Member Access is disabled. When `KNOWLEDGE_MEMBER_ACCESS=true` and connected accounts is available, organization settings shows the Search **Integrations** page instead. Only the selected page is available, including through direct links. Switching pages does not remove existing connections or workspace access.
+Manage the pool under **Settings → Credential Groups**. It remains available independently of Search. Search administrators use **Settings → Sources**, while members connect personal Search accounts under **Integrations** in the main sidebar. These surfaces can share grants without sharing their purpose or resource settings.
 
 ## Set up connected accounts
 
-Open your organization’s **Settings → Connected accounts**. Select **Set up connected accounts** if this is your first time.
+Open your organization’s **Settings → Credential Groups**. Select **Set up connected accounts** if this is your first time.
 
 The page has three tabs:
 
@@ -44,11 +44,11 @@ For Databricks, **Add** validates and saves the configuration before the provide
 
 When editing Databricks, leaving the client secret blank preserves the saved secret. Changing the MCP URL or OAuth client requires people to reconnect.
 
-Adding a provider makes account connections available without enabling indexing. Search source setup is managed on **Settings → Integrations** when Search is enabled. Connected accounts has no indexing controls or status indicators. Managed MCP providers currently support live tool calls only.
+Adding a provider makes account connections available; it does not add a Search source or start document indexing. Search source setup is managed under **Settings → Sources**. Live Search queries providers on demand. Credential Groups has no indexing controls or status indicators. Managed MCP providers support live tool calls.
 
 ### 2. Invite people
 
-For Search-enabled organizations, open **Settings → Integrations → People**. Otherwise, use **Settings → Connected accounts → People**. Set up a provider for personal account connections before inviting people; approving a Search integration alone is not enough.
+Open **Settings → Credential Groups → People**. Set up the provider before inviting people; a Search source does not replace required OAuth or managed MCP configuration.
 
 1. Select **Request connections**.
 2. Enter their email addresses and select **Send requests**.
@@ -63,7 +63,7 @@ Use the invitation email in workflow lookups. For example, if you invite `alex@e
 
 On first use, the invitation email must match the signed-in user's verified Sim email. Sim then binds the invitation to that user's account. Another person cannot take it over by opening the link, and a later email change does not transfer it to a different Sim user. The invitation email remains the workflow lookup key.
 
-OAuth account providers also verify that the provider email matches the invitation email. Managed MCP connections are associated with the verified Sim user who completes authorization; they do not independently verify the MCP provider's account email. Keep using the invitation email when looking up those connections.
+The verified Sim user who starts authorization must complete it. A provider account can use a different email from the invitation; the saved contribution belongs to the bound Sim user. Managed MCP connections likewise use that Sim user’s authorization. Keep using the invitation email for workflow lookups. Search separately checks current provider identity and any source-specific requirements.
 
 In **People**, open a person's actions menu and select **Resend** when they need another invitation. **Revoke** stops the organization from using that person's contributions. The person cannot undo an administrator's revocation by reconnecting on their own.
 

apps/docs/content/docs/platform/self-hosting/background-jobs.mdx

@@ -48,8 +48,8 @@ Point cron at an **internal** address where possible (the in-cluster Service, or
 | Outbox processing | `/api/webhooks/outbox/process` | `*/1 * * * *` | Transactional-outbox retries for billing, membership, enterprise issuance, and workflow-deployment side effects |
 | Workspace file search dispatch | `/api/cron/workspace-file-search-dispatch` | `*/1 * * * *` | Dispatches indexing work for workspace file search |
 | Connector sync | `/api/knowledge/connectors/sync` | `*/5 * * * *` | Knowledge base connector syncs |
-| Connector member sync | `/api/knowledge/connectors/member-sync` | `*/5 * * * *` | Per-member access sync for permission-aware connectors |
-| Connector directory sync | `/api/knowledge/connectors/directory-sync` | `*/5 * * * *` | Refreshes the directory groups administrator-mode connectors mirror, so a membership change takes effect without waiting for a content sync |
+| Connector member sync | `/api/knowledge/connectors/member-sync` | `*/5 * * * *` | Per-member content/access sync for indexed connectors; live Search sources are skipped |
+| Connector directory sync | `/api/knowledge/connectors/directory-sync` | `*/5 * * * *` | Refreshes stored groups for indexed administrator-mode connectors; live Search sources are skipped |
 | Workspace events poll | `/api/workspace-events/poll` | `*/15 * * * *` | Workspace event triggers |
 | Table row TTL cleanup | `/api/cron/cleanup-table-row-ttl` | `*/15 * * * *` | Deletes table rows whose TTL column has expired |
 | OAuth token cleanup | `/api/cron/cleanup-oauth-tokens` | `0 * * * *` | Deletes access and refresh tokens after the retention tail |

apps/docs/content/docs/platform/self-hosting/integrations-oauth.mdx

@@ -117,7 +117,7 @@ The same variables also power "Sign in with Microsoft".
 
 ### GitHub Search
 
-Self-hosted GitHub Search uses a GitHub App for account connections and organization installation indexing. Register your own App and configure the server variables below. Sim Cloud users use the [GitHub Search setup flow](/search/github#add-a-repository) directly.
+Self-hosted GitHub Search uses a GitHub App for personal account connections and optional organization repository boundaries. Register your own App and configure the server variables below. Sim Cloud users use the [GitHub Search setup flow](/search/github#add-a-repository) directly.
 
 <Steps>
 <Step>
@@ -150,14 +150,14 @@ Authorization starts from Sim so the callback can finish the pending connection.
 
 For an existing deployment, deploy the application with the setup callback before changing the App registration. Use a separate GitHub App for each deployment origin, such as staging and production, so installation approval returns to the instance that started it. Keep the existing user authorization callback above unchanged.
 
-The connector polls GitHub's API and does not need a webhook.
+Live Search queries GitHub's API on demand and does not need a webhook. Ordinary knowledge-base connectors and explicitly restored legacy Search use their separate polling path.
 
 </Step>
 <Step>
 
 #### Set read permissions
 
-Expand **Permissions → Repository permissions**. Set **Contents → Access: Read-only**; leave the mandatory **Metadata** permission at **Read-only**.
+Expand **Permissions → Repository permissions**. Set **Contents**, **Issues**, and **Pull requests** to **Read-only**; leave mandatory **Metadata** at **Read-only**. Contents supports code reads; Issues and Pull requests allow the connected member’s App user token to search and read private issues and pull requests.
 
 <Image className="mx-auto h-auto w-full max-w-md" src="/static/search/github-app-repository-permissions.jpg" alt="GitHub repository permissions with Contents set to Read-only and Metadata shown as mandatory Read-only" width={768} height={929} />
 
@@ -170,11 +170,13 @@ Expand **Organization permissions** and set **Members → Access: Read-only**. S
 | Permission area | Permission | Access |
 |---|---|---|
 | Repository | Contents | Read-only |
+| Repository | Issues | Read-only |
+| Repository | Pull requests | Read-only |
 | Organization | Members | Read-only |
 | Repository | Metadata | Read-only |
 | Account | Email addresses | Read-only |
 
-Leave every other permission at **No access**. Sim does not need issue, pull-request, administration, or write permissions. GitHub's [registration guide](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app) explains these settings.
+Leave other permissions at **No access**. Search does not need administration or write permissions. Existing installations must approve the additional issue and pull-request read permissions; members may need to reconnect. The service-side installation token remains narrowed to repository contents/metadata to verify source coverage; content search uses the member’s App user token. GitHub's [registration guide](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app) explains these settings.
 
 GitHub App user tokens use these permissions rather than OAuth scopes. An empty `scope` value in the token response is expected; see GitHub's [user token reference](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app).
 
@@ -203,20 +205,20 @@ The private key must include its PEM header, footer, and contents. Sim accepts a
 
 Keep the private key and client secret in the deployment's server configuration; organization admins select installations in Sim without entering these secrets.
 
-If you run Search indexing on Trigger.dev, configure the same five variables in its matching environment. The app server and indexing worker both need the App credentials; updating the app's secret store alone does not update a separately configured worker.
+The live Search application needs these five variables. If a worker also indexes ordinary GitHub knowledge bases, or Search has explicitly reverted to `SIM_SEARCH_LIVE=false`, configure the variables in that worker’s matching environment too. Updating the app’s secret store does not update a separately configured worker. Live Search does not schedule a GitHub indexing worker.
 
 The **Client ID** is different from the numeric **App ID**. Use credentials from **Developer settings → GitHub Apps**. `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET` belong to the separate GitHub sign-in integration and remain unchanged. Search does not read `GITHUB_REPO_CLIENT_ID` or `GITHUB_REPO_CLIENT_SECRET`.
 
 Keep **Expire user authorization tokens** enabled so Sim receives the refresh token it needs to [renew personal connections](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/refreshing-user-access-tokens).
 
 Complete the installation through [the GitHub Search source setup](/search/github#add-a-repository).
 
-If you replace a deployment's GitHub App, an organization admin selects **Settings → Sources → More → Refresh connection settings**. When Search is disabled, use **Connected accounts → Providers → Update configurations** in organization settings. This applies the deployment's current App configuration to all existing providers while preserving their option IDs. Accounts whose App configuration changed must reconnect. Then reconnect personal GitHub accounts and connect an installation of the new App. Reconnecting alone cannot update the organization's saved App configuration.
+If you replace a deployment's GitHub App, an organization admin selects **Settings → Credential Groups → Providers → Update configurations**. This applies the deployment's current App configuration to all existing providers while preserving their option IDs. Accounts whose App configuration changed must reconnect. Then reconnect personal GitHub accounts and connect an installation of the new App. Reconnecting alone cannot update the organization's saved App configuration.
 
 </Step>
 </Steps>
 
-If GitHub rejects `redirect_uri`, compare the App's registered callback with `NEXT_PUBLIC_APP_URL` followed by `/api/auth/oauth2/callback/github-repositories`. Keep wildcard matching disabled. If installation indexing is unavailable, confirm all five `GITHUB_APP_*` variables belong to the same App and include a complete RSA private key.
+If GitHub rejects `redirect_uri`, compare the App's registered callback with `NEXT_PUBLIC_APP_URL` followed by `/api/auth/oauth2/callback/github-repositories`. Keep wildcard matching disabled. If App installation setup is unavailable, confirm all five `GITHUB_APP_*` variables belong to the same App and include a complete RSA private key.
 
 GitHub workflow blocks and knowledge-base token connections continue to use personal access tokens.