ARCHITECTURE.md

AutoSEO — Architecture & Conventions

Open-source, self-hosted AI-visibility (GEO/AEO) + classic SEO platform. Feature parity target: finseo.ai (see docs/research/finseo-features.md) + every-app/open-seo (see docs/research/open-seo-inventory.md), plus a local CLI agent system (Claude Code / Codex) that powers all AI features (API-key fallback).

Stack

• Next.js 16 (App Router, Turbopack, proxy.ts instead of middleware, async request APIs), React 19.3
• Tailwind v4 + shadcn (radix base, preset nova) — all primitives in src/components/ui
• PostgreSQL 17 + Drizzle ORM (postgres driver). Migrations in drizzle/, applied automatically at boot.
• In-process job queue (Postgres SKIP LOCKED) + cron scheduler started from src/instrumentation.ts.
• Deployment: Docker Compose (Coolify). .env only contains DOMAIN. Everything else lives in the DB and is configured in the Admin panel (/admin). Secrets are encrypted at rest (AES-256-GCM) with a key auto-generated into the data volume (/data/secret.key).

Directory layout

src/
  app/
    (auth)/            login, verify, invite, setup (public pages)
    (app)/             authenticated app shell (sidebar + topbar)
      p/[projectId]/   project-scoped feature pages
      settings/        account, api keys, workspace, usage
      agents/          local agents dashboard
      admin/           instance admin panel (requires admin.access)
    api/               route handlers (v1 REST, mcp, agent, webhooks, oauth)
    share/             public share pages (reports)
  components/
    ui/                shadcn primitives (do not edit casually)
    app/               app-shell, shared composites (page-header, data-table, kpi-card, charts, filters…)
    agent-ui/          original agent UI blocks (thinking state, tool calls, streaming text, orb…)
  features/<module>/   feature modules: components/, actions.ts (server actions), queries.ts, types.ts
  server/              server-only code
    db/                client + schema/<module>.ts (one schema file per module) + index re-export
    auth/              sessions, magic links, invitations, permissions
    settings/          typed settings registry + encryption
    jobs/              queue, worker, scheduler, handler registry
    email/             SMTP (Amazon SES compatible) mailer + templates
    ai/                LLM router (local agent → API fallback), engines for AI visibility
    dataforseo/        DataForSEO client + cost tracking
    <module>/          module services (pure server logic, used by actions, API, MCP, jobs)
  lib/                 isomorphic helpers (utils, formatters, constants)
agent/                 local agent runtime (agent.mjs) + install scripts (served by the app)
drizzle/               SQL migrations (generated by drizzle-kit)
docker/                Dockerfile, entrypoint

Conventions

• Server-only modules start with import "server-only";.
• Auth in every entrypoint: pages call requireUser() / requireProject(projectId, permission?) (src/server/auth/guards.ts); server actions call the same guards; route handlers use requireApiAuth().
• Permissions are string keys (src/server/auth/permissions.ts). Roles map to permission sets and are editable in Admin → Roles. Check with can(ctx, "prompts.manage").
• Settings: never read process.env for config (only DOMAIN, DATABASE_URL, DATA_DIR, NODE_ENV). Use getSetting("smtp") etc. from src/server/settings. Secrets are declared secret: true and encrypted.
• Jobs: long work goes through enqueueJob(type, payload, opts); handlers register in src/server/jobs/handlers/*.ts and are imported by src/server/jobs/registry.ts.
• Usage/cost: every paid external call records recordUsage() (src/server/usage.ts).
• DB schema: each module owns src/server/db/schema/<module>.ts; ids are text (nanoid/prefix), timestamps timestamp with time zone. Always scope project data by projectId and check access.
• UI: use shadcn primitives + components/app/* composites. Pages: PageHeader + content. Tables: DataTable (TanStack) with mobile card fallback. Charts: components/app/charts/* (recharts via shadcn chart). Every page must be fully usable at 375px width (no horizontal page scroll; tables scroll inside their card).
• Empty states everywhere, explaining what to configure (e.g. "Connect DataForSEO in Admin → Providers").
• No mock data in production paths. Demo/seed data only behind the explicit "Load demo project" action.

Foundation (already built — reuse, don't re-implement)

Concern	Where
Env (only DOMAIN)	src/server/env.ts
DB client / schema	src/server/db/client.ts, src/server/db/schema/*.ts (_helpers.ts: id(prefix), createdAt(), updatedAt(), ts(), newId())
Settings (typed, encrypted secrets)	src/server/settings/{registry,index}.ts → getSetting("ai" | "dataforseo" | "google" | …), updateSetting(), getPublicSetting()
Crypto	src/server/crypto.ts (encryptJson/decryptJson, sha256, randomToken, hmac)
Auth / guards	src/server/auth/guards.ts (requireUser, requireProject(id, perm?), requireAdmin, runAction, actionUser, actionProject(id, perm?), actionAdmin, ActionError)
Permissions	src/server/auth/permissions.ts (PERMISSIONS, BUILTIN_ROLES) — roles in DB table roles
Sessions	src/server/auth/session.ts (1-year multi-device sessions)
Magic link / invites	src/server/auth/login.ts, src/server/auth/membership.ts
Jobs & cron	src/server/jobs/{queue,define,worker}.ts → enqueueJob(type, payload, opts), defineJob({type, run}), defineSchedule({name, cron, tick}); register in src/server/jobs/handlers/<module>.ts
Email (SMTP / SES)	src/server/email/{index,templates}.ts → sendMail, simpleEmail
LLM router	src/server/ai/llm.ts → runLlm({purpose, prompt, schema?, webSearch?, route?}) (local agent → Anthropic/OpenAI/OpenRouter fallback), extractJson, AiNotConfiguredError
Local agent dispatch contract	src/server/agents/dispatch.ts (dispatchAgentLlm, hasOnlineAgent)
DataForSEO	src/server/dataforseo/client.ts → dfsPost(path, body, ctx), dfsGet, isDataForSeoConfigured, DataForSeoNotConfiguredError
Usage / budgets	src/server/usage.ts → recordUsage, assertBudget
Audit log	src/server/audit.ts → logAudit(action, …)
Rate limit	src/server/rate-limit.ts
Projects	src/server/projects.ts → createProject, normalizeDomain
AI bootstrap contract	src/server/ai/bootstrap.ts (suggestBrandProfile/Competitors/Prompts)
Constants	src/lib/countries.ts (143 markets + DataForSEO location codes, flagEmoji), src/lib/engines.ts (16 AI engines, AI_BOTS), src/lib/navigation.ts (ALL routes)
URL state	src/hooks/use-url-state.ts (useUrlState, useUrlListState, useUrlPatch)
Shell	src/components/app/{app-shell,app-sidebar,topbar,shell-context}.tsx (useShell, useCan, useProjectHref)
Page layout	src/components/app/page.tsx (PageContainer, PageHeader, Panel, TabNav)
Metrics UI	src/components/app/metrics.tsx (KpiStrip, StatCard, Delta, Meter, SegmentBar, formatNumber/Compact/Percent/Currency)
Tables	src/components/app/data-table.tsx (DataTable<T> — sorting, pagination, selection, expandable rows, grouping, mobile cards). Do NOT use TanStack Table.
Filters	src/components/app/filters.tsx (PeriodSelect, resolvePeriod, MultiSelect, TagFilter, SearchInput, FilterBar)
Charts	src/components/app/charts.tsx (TrendChart, BarsChart, DonutChart, LegendList, RadarView, Sparkline, RankedBars, Heatmap, TickGauge, ScoreRing, StackedBar), world-map.tsx (WorldMap), sankey.tsx (SankeyChart)
Misc UI	src/components/app/misc.tsx (CountryFlag, CopyButton, ConfirmButton, StatusBadge, TimeAgo, TagChip), engine-icon.tsx (EngineIcon, EngineStack), favicon.tsx (Favicon), empty-state.tsx (EmptyState)
Dev auth for curl	node scripts/dev-session.mjs → prints autoseo_session=… cookie

Module ownership

Module	Owns
ai-tracking	src/server/ai/{engines,tracking,analysis,bootstrap}*, jobs handlers/ai.ts, pages ai/tracker, ai/tracker/fanouts, ai/models
ai-insights	src/server/ai/metrics*, pages home dashboard, ai/competitors, ai/sentiment, ai/sources, ai/products, ai/ads, demo data generator
ai-research	pages ai/prompt-research, knowledge, ai/brand-lookup, ai/prompt-explorer
seo	schema seo.ts, src/server/seo/**, pages seo/* (except audit), jobs handlers/seo.ts
audit	schema audit.ts, src/server/audit-crawler/**, pages seo/audit, crawlability, jobs handlers/audit.ts
analytics	schema analytics.ts, pages analytics/*, integrations, Google/Bing OAuth, log ingest
attribution	schema attribution.ts, page attribution, snippet + webhooks
reports	schema reports.ts, pages reports/**, /share/**
optimize	schema optimize.ts, pages tasks, content, fact-check
agents	schema agents.ts, agent/, /install*, src/app/api/agent/**, pages /agents/**, admin/agents, src/server/agents/**
platform-api	schema oauth.ts, src/app/api/v1/**, src/app/api/mcp/**, OAuth 2.1, page settings/api
chat	schema chat.ts, pages p/[id]/agent/**, src/components/agent-ui/**
free-tools	schema free-tools.ts, src/server/free-tools/**, pages p/[id]/seo/tools/**, public /free-tools/**, api/free-tools/**
admin	pages admin/**, settings/{account,workspace,usage,projects}, onboarding, p/[id]/settings, p/[id]/tour