diff --git a/.env.example b/.env.example index b721b64d..5f685586 100644 --- a/.env.example +++ b/.env.example @@ -164,9 +164,9 @@ BACKUP_DIR=./backups AIR_GAPPED_MODE=false # ── Observability (optional) ──────────────────────────────── -# Bearer token protecting GET /api/metrics (Prometheus). Set a value in -# production; when empty, the endpoint is public and should be restricted -# at the reverse proxy. +# Bearer token protecting GET /api/metrics (Prometheus). Empty = public in +# non-production; denied with 403 in production when empty. +# Production requires METRICS_TOKEN to be set. METRICS_TOKEN= # Grafana Cloud OTLP diff --git a/AGENTS.md b/AGENTS.md index 74362f4c..196119db 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -201,6 +201,8 @@ - `issues` — Issue tracking - `projects` / `milestones` / `labels` — Project organization - `wiki` — Repository wiki pages +- `gists` — Standalone code snippets (public/private, JSON file arrays) +- `discussions` / `discussionComments` — GitHub-style repo discussions ### CI/CD - `workflows` / `workflowRuns` / `workflowJobs` / `workflowSteps` — Pipeline execution @@ -374,7 +376,7 @@ docker-compose up -d # Full stack with postgres + redis + runner - **Load**: Custom load tests in `tests/load/` - **Accessibility**: `@axe-core/playwright` -**Current status: 546 tests passing across 114 test files (100% pass rate)** +**Current status: 682 tests passing across 130 test files (100% pass rate)** --- @@ -420,7 +422,7 @@ docker-compose up -d # Full stack with postgres + redis + runner ## 14. Common Pitfalls 1. **Git operations timeout**: `GIT_PROCESS_TIMEOUT_SECS` default is 300s; increase for large repos -2. **Pack size limits**: `MAX_PACK_SIZE_MB` default is 500MB +2. **Pack size limits**: `MAX_PACK_SIZE_MB` default is 2048MB 3. **SSH host key**: Generated automatically at `GIT_SSH_HOST_KEY` on first start 4. **Runner privileges**: CI runner needs `--privileged` for Docker-in-Docker 5. **Database driver mismatch**: Schemas use `pgTable` — PostgreSQL is required for production @@ -435,12 +437,15 @@ docker-compose up -d # Full stack with postgres + redis + runner - Git hosting (HTTP + SSH) - PRs, issues, milestones, project boards - Wiki, organizations, teams +- Discussions and gists - Stacked PRs (web + CLI) - Merge queue with speculative builds - CI/CD pipeline engine -- Webhooks and automations -- Rate limiting, CSRF, MFA, OAuth -- REST API (140+ routes) + GraphQL +- Webhooks with queued delivery (worker-processed, dead-letter) and automations +- Push + pull repo mirroring +- Commit signature verification ("Verified" via openpgp + DB GPG keys) +- Rate limiting, CSRF, MFA, OAuth, login lockout +- REST API (140+ routes) + GraphQL (cursor-paginated) - CLI with 20+ command groups **Expanding**: diff --git a/README.md b/README.md index 578ad0d0..cec668ab 100644 --- a/README.md +++ b/README.md @@ -1,196 +1,180 @@

- - - - OpenCodeHub - + + OpenCodeHub Logo +

-

The self-hosted Git platform that doesn't compromise.

+

OpenCodeHub

- CI + The self-hosted Git collaboration platform that doesn't compromise.
+ Stacked PRs · Speculative Merge Queue · GitHub Actions CI/CD · AI Code Review · Enterprise Security · High-Performance CLI +

+ +

+ CI License - CLI version - Docs - Docker + CLI version + Docker + Docs

--- -OpenCodeHub is a self-hosted Git platform with **stacked PRs**, **merge queue**, **CI/CD pipelines**, and **AI code review**. One platform for everything your team needs — no vendor lock-in, no per-seat pricing. +OpenCodeHub is an open-source, self-hosted alternative to GitHub and GitLab engineered as a modular monolith. It provides stack-first developer workflows (Graphite-style stacked diffs), an automated merge queue with speculative builds, Docker-based CI/CD pipelines, and multi-model AI code reviews out of the box — with zero per-seat licensing fees. -**[Documentation](https://docs.opencodehub.space)** · **[Deploy in 5 minutes](#deploy)** · **[CLI Reference](https://docs.opencodehub.space/reference/cli-commands/)** +**[Documentation](https://docs.opencodehub.space)** · **[CLI Quickstart](#-opencodehub-cli-och)** · **[Deploy in 5 Minutes](#-quickstart--deployment)** · **[API Reference](https://docs.opencodehub.space/api/rest-api/)** --- -## Demo - - -[![OpenCodeHub Demo](https://img.youtube.com/vi/VIDEO_ID/maxresdefault.jpg)](https://youtu.be/VIDEO_ID) +## ✨ Key Differentiators -*Watch the full walkthrough — deployment, stacked PRs, AI review, and merge queue in action.* +| Capability | What It Does | Why It Matters | +|---|---|---| +| **Stacked PRs** | Break complex features into small, dependent branches and PRs (`och stack`) | Faster reviews, zero merge blocking, unblocked teammates | +| **Merge Queue** | Parallel CI validation with speculative builds and priority lanes | Protects `main` from broken builds without serialized slow merges | +| **CI/CD Pipelines** | GitHub Actions YAML compatibility with isolated Docker executors | Native pipeline execution without external CI SaaS dependencies | +| **Multi-Provider AI Review** | Automated code reviews powered by GPT-4, Claude 3.5, Gemini 1.5, Groq, Ollama | Instant feedback on PRs before human peer review | +| **Pluggable Storage** | Local filesystem or S3-compatible object storage (AWS, MinIO, R2, B2, Ceph) | Flexible deployment across homelabs, NAS servers, or hyper-scale clouds | +| **Federation** | Fork, push branches, and submit cross-instance pull requests across servers | Seamless collaboration across autonomous OpenCodeHub instances | --- -## Why OpenCodeHub? +## 💻 OpenCodeHub CLI (`och`) -| Problem | Solution | -|---------|----------| -| PRs too large to review | **Stacked PRs** — break changes into small, dependent branches | -| Merge conflicts on main | **Merge Queue** — stack-aware ordering with speculative CI builds | -| Slow review cycles | **AI Code Review** — catches bugs before humans even look | -| Split across 5+ services | **All-in-one** — Git, PRs, CI/CD, issues, wiki in one place | -| Data leaves your servers | **Self-hosted** — your code stays on your hardware | +The official OpenCodeHub CLI (`opencodehub-cli`) delivers a terminal-first workflow for stacks, code reviews, merge queues, and repository management. ---- +### Installation -## Features - -### Core Platform -- **Git Hosting** — HTTP smart protocol + SSH push/pull, forks, mirroring, LFS -- **Pull Requests** — Inline comments, approvals, suggested changes, draft PRs -- **Issues & Projects** — Labels, milestones, custom fields, kanban boards -- **Wiki** — Repository wiki with revision history -- **Organizations** — Teams, collaborators, role-based access control - -### Delivery Workflows -- **Stacked PRs** — Graphite-style stacked branches with web + CLI support -- **Merge Queue** — Stack-aware queue with speculative builds and priority lanes -- **CI/CD Pipelines** — GitHub Actions-compatible engine with Docker-based runners -- **Webhooks** — Outbound webhooks with event filtering and HMAC signing -- **Automations** — Rule-based workflow automation for PRs and deployments - -### AI & Quality -- **AI Code Review** — 10+ providers: GPT-4, Claude, Gemini, Groq, Ollama, OpenRouter -- **Secret Scanning** — Detect secrets in commits before they reach production -- **Branch Protection** — Required reviews, status checks, push restrictions -- **Developer Metrics** — PR velocity, review efficiency, time-to-merge tracking - -### Security -- **Authentication** — JWT sessions, OAuth (GitHub, Google, GitLab), 2FA/TOTP, SSO/SAML -- **Authorization** — RBAC with roles, team permissions, collaborator levels -- **Rate Limiting** — Redis-backed per-endpoint rate limiting -- **Audit Logging** — Track all administrative actions - -### Extensibility -- **REST API** — 175+ endpoints covering all platform features -- **GraphQL** — Full GraphQL endpoint for flexible queries -- **CLI** — `och` command line tool with 20+ command groups -- **Storage** — Local, S3, MinIO, R2, or any S3-compatible backend +Install globally using your preferred package manager: ---- +```bash +# via npm +npm install -g opencodehub-cli -## Deploy +# via bun +bun add -g opencodehub-cli -### Docker (Recommended) +# via pnpm +pnpm add -g opencodehub-cli -```bash -git clone https://github.com/swadhinbiswas/OpencodeHub.git -cd OpenCodeHub -cp .env.example .env -docker compose up -d -docker compose exec app bun run scripts/seed-admin.ts +# via yarn +yarn global add opencodehub-cli ``` -Open **http://localhost:4321** and create your admin account. - -### Render (Free) - -Deploy to Render with free PostgreSQL + Upstash Redis: +Or run directly without installing: ```bash -# 1. Create free Redis at upstash.com (Singapore region) -# 2. Push to GitHub -# 3. Render → New → Blueprint → Select your repo -# 4. Set REDIS_URL and SITE_URL -# 5. Deploy +npx opencodehub-cli --help ``` -See the [Render Deployment Guide](docs/RENDER-DEPLOYMENT.md) for step-by-step instructions. +### 1. Authenticate -### More Deployment Options +```bash +# Interactive login +och auth login --url https://git.yourcompany.com -| Platform | Guide | Cost | -|----------|-------|------| -| Docker Compose | [Deployment Guide](docs/administration/deployment.md) | Free | -| Render (Asia) | [Render Guide](docs/RENDER-DEPLOYMENT.md) | Free | -| Oracle Cloud | [Free Deployment](docs/FREE-DEPLOYMENT.md) | Free forever | -| NAS (Synology/TrueNAS) | [NAS Guide](docs/administration/deploy-nas.md) | Free | -| Kubernetes | [K8s Guide](docs/administration/kubernetes.md) | Free | -| Cloudflare Tunnel | [Cloudflare Guide](docs/administration/deploy-cloudflare.md) | Free | +# Non-interactive / CI login with Personal Access Token +och auth login --url https://git.yourcompany.com --token och_xxxxxxxxxxxx ---- +# Check health & auth status +och config doctor +``` -## CLI +### 2. Stacked PR Workflow + +Create and submit Graphite-style stacked branches in seconds: ```bash -# Install -npm install -g opencodehub-cli +# Create first stacked branch +och stack create feature/user-auth -# Login -och auth login --url http://localhost:4321 +# Commit changes, then create dependent branch +git commit -am "feat: implement auth middleware" +och stack create feature/user-profile -# Stacked PR workflow -och stack create feature/auth-step-1 +# Push all branches in the stack and generate linked PRs automatically och stack submit + +# Visualize your stack topology in terminal +och stack log + +# Rebase the whole stack when upstream main updates och stack sync +``` -# Merge queue -och queue list -och queue add +### 3. Interactive Focus Cockpit (`och focus`) -# Interactive cockpit +Launch the terminal dashboard for PRs, reviews, and merge queue operations: + +```bash och focus ``` -[Full CLI Reference](https://docs.opencodehub.space/reference/cli-commands/) +- **Interactive Timeline**: Switch branches, view diff stats, and trigger CI runs without leaving the CLI. +- **Review Cockpit**: Approve, request changes, or trigger AI code reviews inline. +- **Queue Controls**: Enqueue PRs and monitor speculative build states in realtime. + +### 4. Merge Queue Management + +```bash +# View active queue status and speculative lanes +och queue list + +# Enqueue a PR for automated CI validation and merge +och queue add 42 + +# Check queue position and build attempts +och queue status 42 +``` --- -## API +## 🚀 Quickstart & Deployment + +### Production Stack (Docker Compose) ```bash -# Create a repository -curl -X POST http://localhost:4321/api/repos \ - -H "Authorization: Bearer YOUR_TOKEN" \ - -H "Content-Type: application/json" \ - -d '{"name":"my-project","visibility":"public"}' - -# List pull requests -curl http://localhost:4321/api/repos/owner/repo/pulls \ - -H "Authorization: Bearer YOUR_TOKEN" - -# GraphQL -curl -X POST http://localhost:4321/api/graphql \ - -H "Authorization: Bearer YOUR_TOKEN" \ - -H "Content-Type: application/json" \ - -d '{"query": "{ repositories { name owner { username } } }"}' +# 1. Clone the repository +git clone https://github.com/swadhinbiswas/OpencodeHub.git +cd OpenCodeHub + +# 2. Configure environment +cp .env.example .env +# Edit .env and set your secrets (JWT_SECRET, SESSION_SECRET, POSTGRES_PASSWORD, REDIS_PASSWORD) + +# 3. Launch the container stack +docker compose up -d + +# 4. Initialize the admin account +docker compose exec app bun scripts/seed-admin.ts ``` -[Full API Reference](https://docs.opencodehub.space/api/rest-api/) +Open **`http://localhost:4321`** in your browser to start collaborating. + +### Container Images + +Official minimal production images (251 MB) are published on Docker Hub: + +| Service | Image Tag | Purpose | +|---|---|---| +| **Platform** | `opencodehub/opencodehub:latest` | Web UI + REST & GraphQL API + Git HTTP / SSH Server | +| **Worker** | `opencodehub/opencodehub-worker:latest` | Background queues, webhooks, and automation jobs | +| **Runner** | `opencodehub/opencodehub-runner:latest` | Docker-in-Docker CI/CD pipeline execution runner | --- -## Tech Stack - -| Layer | Technology | -|-------|-----------| -| Framework | Astro 4.x (SSR) + React 18 | -| UI | Tailwind CSS + Radix UI | -| Database | PostgreSQL / SQLite / Turso (Drizzle ORM) | -| Auth | JWT + OAuth + 2FA/TOTP + SSO/SAML | -| Git | Native git CLI + simple-git + isomorphic-git | -| SSH | ssh2 library | -| CI/CD | Docker-based runners, GitHub Actions syntax | -| Storage | Local filesystem or S3-compatible (AWS, MinIO, R2) | -| AI | OpenAI, Anthropic, Google, Groq, Ollama, OpenRouter | -| Queue | BullMQ + Redis | -| CLI | Commander.js + Inquirer | +## 🎨 Design System & Theme + +OpenCodeHub features a modern developer UI built on **Tailwind CSS**, **Radix UI**, and clean GitHub/Linear design principles: + +- **Adaptive Theming**: Native dark and light theme support with zero layout flicker. +- **High-Contrast Dark Mode**: Pure zinc/slate dark surfaces (`#0d1117` / `#161b22`) paired with crisp emerald status indicators (`#238636`). +- **Responsive Workspace**: Full mobile, tablet, and desktop fidelity with integrated keyboard shortcuts. --- -## Architecture +## 🏛 Architecture ``` ┌─────────────────────────────────────────────────────────────┐ @@ -200,63 +184,43 @@ curl -X POST http://localhost:4321/api/graphql \ │ ┌─────────────────────────────────────────────────────────────┐ │ OPENCODEHUB PLATFORM │ -│ Web UI (Astro+React) │ REST API (175+ routes) │ GraphQL │ -│ Git Server (HTTP) │ SSH Server (ssh2) │ │ -│ Pipeline Runner (Docker) │ │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ +│ │ Web UI │ │ REST API │ │ GraphQL Endpoint │ │ +│ │ (Astro+React│ │ (175+ routes│ │ (src/pages/api/...) │ │ +│ └─────────────┘ └─────────────┘ └─────────────────────┘ │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ +│ │ Git Server │ │ SSH Server │ │ Pipeline Runner │ │ +│ │ (HTTP RPC) │ │ (ssh2 daemon│ │ (Docker Executor) │ │ +│ └─────────────┘ └─────────────┘ └─────────────────────┘ │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ -│ PostgreSQL/SQLite/Turso │ Redis │ Pluggable Storage │ +│ PERSISTENCE & INFRASTRUCTURE │ +│ PostgreSQL / SQLite / Turso │ Redis (Queues) │ Pluggable S3 │ └─────────────────────────────────────────────────────────────┘ ``` --- -## Documentation - -| Topic | Link | -|-------|------| -| Installation | [docs.opencodehub.space/getting-started/installation](https://docs.opencodehub.space/getting-started/installation/) | -| Configuration | [docs.opencodehub.space/administration/configuration](https://docs.opencodehub.space/administration/configuration/) | -| Stacked PRs | [docs.opencodehub.space/features/stacked-prs](https://docs.opencodehub.space/features/stacked-prs/) | -| AI Code Review | [docs.opencodehub.space/features/ai-review](https://docs.opencodehub.space/features/ai-review/) | -| CLI Reference | [docs.opencodehub.space/reference/cli-commands](https://docs.opencodehub.space/reference/cli-commands/) | -| API Reference | [docs.opencodehub.space/api/rest-api](https://docs.opencodehub.space/api/rest-api/) | -| Deployment | [docs.opencodehub.space/administration/deployment](https://docs.opencodehub.space/administration/deployment/) | - ---- - -## Contributing +## 🧪 Development & Testing ```bash -# Clone and setup -git clone https://github.com/swadhinbiswas/OpencodeHub.git -cd OpenCodeHub -cp .env.example .env +# Install dependencies npm install -npm run db:push -bun run scripts/seed-admin.ts -# Start development -npm run dev +# Push database schema +npm run db:push -# Run tests +# Run full test suite (679 unit, integration & contract tests) npm run test -``` -See [CONTRIBUTING.md](CONTRIBUTING.md) for development workflow and standards. - ---- - -## Community - -- **GitHub**: [github.com/swadhinbiswas/OpencodeHub](https://github.com/swadhinbiswas/OpencodeHub) -- **Documentation**: [docs.opencodehub.space](https://docs.opencodehub.space) -- **Issues**: [GitHub Issues](https://github.com/swadhinbiswas/OpencodeHub/issues) -- **Discussions**: [GitHub Discussions](https://github.com/swadhinbiswas/OpencodeHub/discussions) +# Typecheck & verify build +npm run typecheck +npm run build +``` --- -## License +## 📄 License -[MIT](LICENSE) — Use it however you want. +OpenCodeHub is released under the [MIT License](LICENSE). diff --git a/cli/README.md b/cli/README.md index 561c70dd..712de258 100644 --- a/cli/README.md +++ b/cli/README.md @@ -1,236 +1,143 @@ -

- OpenCodeHub CLI -

- -

OpenCodeHub CLI (OCH)

+

OpenCodeHub CLI (och)

- Production-ready Git workflows and stack-first pull request management from your terminal. + Stack-first PR workflows, speculative merge queues, and terminal cockpit for OpenCodeHub.

npm version npm downloads - license + license

-## Why OCH +--- -- Fast command-line workflows for repositories, pull requests, issues, and reviews. -- Stack-first branch/PR flow for multi-PR delivery. -- Secure authentication model with OS credential storage support. -- Built-in API tooling for scripting and automation. +## 📦 Installation -## Installation +Install `opencodehub-cli` globally with your favorite package manager: ```bash +# npm npm install -g opencodehub-cli -``` -Verify: +# bun +bun add -g opencodehub-cli -```bash -och --version -och --help -``` - -## Quick Start - -```bash -# 1) Authenticate -och auth login --url https://git.example.com - -# 2) Inspect your setup -och config doctor +# pnpm +pnpm add -g opencodehub-cli -# 3) Push and open PR workflow -cd your-repo -och repo push - -# 4) Create a pull request -och pr create --base main --title "feat: add onboarding" +# yarn +yarn global add opencodehub-cli ``` -## Core Commands - -- `och auth` authentication (`login`, `logout`, `status`) -- `och config` CLI settings and diagnostics (`list`, `set`, `doctor`) -- `och repo` repository operations (`create`, `clone`, `push`, `list`) -- `och pr` pull request lifecycle (`create`, `list`, `view`, `merge`, ...) -- `och stack` stacked branch/PR workflows (`create`, `submit`, `sync`, ...) -- `och review` code review + AI review flows -- `och issue` issue management -- `och ci`, `och queue`, `och metrics`, `och insights`, `och notify`, `och automate` -- `och api` direct API requests (useful for scripts) - -Run `och --help` for full command options. - -## Authentication and Security - -### Login +Or execute commands on-demand without installing: ```bash -och auth login --url https://git.example.com +npx opencodehub-cli focus ``` -For non-interactive environments: +Verify your installation: ```bash -och auth login --url https://git.example.com --token +och --version +och --help ``` -### Token storage behavior - -- `OCH_TOKEN` env var always takes precedence. -- macOS: Keychain (`security`) -- Linux: Secret Service (`secret-tool`) -- Windows: DPAPI-encrypted storage via PowerShell -- Fallback: local CLI config storage if secure backend is unavailable - -### Helpful environment variables +--- -- `OCH_TOKEN`: inject token from environment -- `OCH_HTTP_TIMEOUT_MS`: HTTP timeout in ms (default `15000`) -- `OCH_DISABLE_KEYCHAIN=1`: disable credential backend (useful in CI/tests) +## ⚡ Quick Start -## Configuration - -Inspect current configuration: +### 1. Authenticate with your OpenCodeHub Instance ```bash -och config list -och config path -och config doctor -``` +# Interactive login prompt +och auth login --url https://git.yourcompany.com -Set values: - -```bash -och config set serverUrl https://git.example.com -och config set defaultBranch main -och config set insecure false +# Non-interactive / CI login +och auth login --url https://git.yourcompany.com --token och_xxxxxxxxxxxxxxxx ``` -## Common Workflows - -### Repository lifecycle +Check configuration and credential storage health: ```bash -# Create a remote repository -och repo create my-service --description "Internal API service" - -# Clone repository -och repo clone acme/my-service - -# Push local repository -och repo push --branch main +och config doctor ``` -### Pull request lifecycle +### 2. Stacked PR Workflow -```bash -# Create PR from current branch -och pr create --base main --title "feat: add API pagination" - -# List open PRs -och pr list --state open - -# View PR details -och pr view 42 - -# Merge PR -och pr merge 42 -``` - -### Stack workflow +OpenCodeHub supports Graphite-style stacked branches from your terminal: ```bash -# Create first stack branch -och stack create auth-foundation +# 1. Create your first stack branch +och stack create feature/part-1 -# Create subsequent branch -och stack create auth-ui +# Make edits and commit +git commit -am "feat: part 1 implementation" -# Submit stack and create/update PRs -och stack submit -``` +# 2. Create the next dependent branch on top of part-1 +och stack create feature/part-2 -### AI review workflow +git commit -am "feat: part 2 implementation" -```bash -# Trigger AI review -och review ai 42 +# 3. Submit all branches in the stack (pushes refs & creates linked PRs) +och stack submit -# Wait for completion -och review ai 42 --wait +# 4. Visualize the stack hierarchy +och stack log -# Check latest review status -och review status 42 +# 5. Rebase stack when target base branch changes +och stack sync ``` -### API mode for scripting +--- -```bash -# Read current user -och api /user +## 🕹 Interactive Focus Cockpit (`och focus`) -# Create issue via API -och api /repos/acme/platform/issues -X POST -F title="Bug: timeout" -F body="Steps to reproduce" -``` +The `och focus` command opens an interactive terminal dashboard for fast daily development: -## CI Usage Example - -```yaml -name: OCH Automation -on: [push] - -jobs: - sync: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: actions/setup-node@v4 - with: - node-version: "20" - - run: npm install -g opencodehub-cli - - run: | - export OCH_TOKEN="${{ secrets.OCH_TOKEN }}" - och config set serverUrl https://git.example.com - och config doctor - och repo push --branch main +```bash +och focus ``` -## Troubleshooting +- **Stack Cockpit**: Navigate branches, view parent/child dependencies, and submit changes. +- **Review Inbox**: Inspect assigned reviews, view diff snippets, and submit approvals or change requests. +- **Merge Queue Status**: Track speculative build progress and priority lanes in real time. +- **AI Reviews**: Trigger AI review analysis directly from the terminal. -### `Not logged in. Run 'och auth login' first.` +--- -- Run `och auth login --url `. -- Or export `OCH_TOKEN` for automation. +## 📋 Command Reference -### `Server URL not configured` +| Command | Description | Example | +|---|---|---| +| `och auth` | Manage authentication credentials | `och auth login --url http://localhost:4321` | +| `och stack` | Stacked PR creation, sync, log, submit | `och stack submit` | +| `och focus` | Interactive terminal review & stack cockpit | `och focus` | +| `och pr` | Pull request lifecycle management | `och pr create --base main --title "feat: demo"` | +| `och queue` | Merge queue control & speculative runs | `och queue list`, `och queue add 42` | +| `och review` | Code review and AI review trigger | `och review start 42` | +| `och repo` | Repository operations (clone, push, create) | `och repo create my-app` | +| `och issue` | Issue tracking & milestone management | `och issue list`, `och issue create` | +| `och ci` | CI pipeline logs and execution control | `och ci list`, `och ci view 12` | +| `och config` | CLI configuration & doctor check | `och config doctor` | +| `och whoami` | Display active user profile & server URL | `och whoami` | +| `och api` | Make direct authenticated REST API calls | `och api GET /api/user` | -- Run `och config set serverUrl https://git.example.com`. +Run `och --help` for specific flags and subcommands. -### TLS / certificate issues +--- -- Use `och config set caFile /path/to/ca.pem` for custom CA. -- Use `och config set insecure true` only for temporary debugging. +## 🔒 Security & Token Storage -### Validate setup end-to-end +The CLI automatically stores credentials in your operating system's secure credential manager: -```bash -och config doctor -``` +- **macOS**: Apple Keychain (`security`) +- **Linux**: Secret Service API / libsecret (`secret-tool`) +- **Windows**: Windows Credential Manager / DPAPI +- **CI / Headless**: Set the `OCH_TOKEN` and `OCH_URL` environment variables. -## Development - -```bash -cd cli -npm install -npm run build -npm run test -``` +--- -## License +## 📄 License -MIT +MIT © OpenCodeHub Contributors diff --git a/docs-site/astro.config.mjs b/docs-site/astro.config.mjs index eb077a97..ceda9da7 100644 --- a/docs-site/astro.config.mjs +++ b/docs-site/astro.config.mjs @@ -14,6 +14,8 @@ export default defineConfig({ replacesTitle: false, }, customCss: ["./src/custom.css"], + defaultColorTheme: "github-dark", + defaultColorThemeKeep: "Always", components: { Hero: "./src/components/StarlightHero.astro", }, diff --git a/docs-site/src/assets/houston.webp b/docs-site/src/assets/houston.webp deleted file mode 100644 index 62ebfd69..00000000 Binary files a/docs-site/src/assets/houston.webp and /dev/null differ diff --git a/docs-site/src/assets/logo-dark.png b/docs-site/src/assets/logo-dark.png deleted file mode 100644 index dd490610..00000000 Binary files a/docs-site/src/assets/logo-dark.png and /dev/null differ diff --git a/docs-site/src/assets/logo-dark.svg b/docs-site/src/assets/logo-dark.svg deleted file mode 100644 index 32095f91..00000000 --- a/docs-site/src/assets/logo-dark.svg +++ /dev/null @@ -1,29 +0,0 @@ - - OpenCodeHub logo - OpenCodeHub wordmark with a rounded gradient code mark and connected hub node. - - - - - - - - - - - - - - - - - - - - - - - Open - CodeHub - - diff --git a/docs-site/src/assets/logo-light.png b/docs-site/src/assets/logo-light.png deleted file mode 100644 index 62ebfd69..00000000 Binary files a/docs-site/src/assets/logo-light.png and /dev/null differ diff --git a/docs-site/src/assets/logo-light.svg b/docs-site/src/assets/logo-light.svg deleted file mode 100644 index 19305551..00000000 --- a/docs-site/src/assets/logo-light.svg +++ /dev/null @@ -1,29 +0,0 @@ - - OpenCodeHub logo - OpenCodeHub wordmark with a rounded gradient code mark and connected hub node. - - - - - - - - - - - - - - - - - - - - - - - Open - CodeHub - - diff --git a/docs-site/src/content/docs/getting-started/installation.md b/docs-site/src/content/docs/getting-started/installation.md index c20e8005..4a01a3c7 100644 --- a/docs-site/src/content/docs/getting-started/installation.md +++ b/docs-site/src/content/docs/getting-started/installation.md @@ -13,6 +13,33 @@ OpenCodeHub is a self-hosted Git platform. You can run it via **Docker** (recomm | **Disk** | 10GB | 50GB SSD | | **OS** | Linux (Ubuntu/Debian) | Linux | +## CLI Installation (`opencodehub-cli`) + +Install the official OpenCodeHub CLI (`och`) to manage stacks, pull requests, and merge queues from your terminal: + +```bash +# via npm +npm install -g opencodehub-cli + +# via bun +bun add -g opencodehub-cli + +# via pnpm +pnpm add -g opencodehub-cli + +# via yarn +yarn global add opencodehub-cli +``` + +Verify your installation: + +```bash +och --version +och --help +``` + +--- + ## 1-Click Installation (Recommended) ```bash diff --git a/docs/features/federation.md b/docs/features/federation.md new file mode 100644 index 00000000..9b96ed35 --- /dev/null +++ b/docs/features/federation.md @@ -0,0 +1,95 @@ +# Federation & Cross-Instance PRs + +> Connect two self-hosted OpenCodeHub instances so contributors on one can fork, contribute, and open pull requests against repositories hosted on another. + +Federation lets a user on instance **A** fork a repository hosted on instance **B**, push branches back to B, and open **cross-instance pull requests** whose head branch lives on A's fork. B pulls the head from A's fork URL, creates a normal PR (with `headRepositoryId` unset), and the usual review and merge pipeline applies. + +## Table of Contents + +- [Overview](#overview) +- [How It Works](#how-it-works) +- [Setting Up Federation](#setting-up-federation) +- [Forking a Repository from Another Instance](#forking-a-repository-from-another-instance) +- [Contributing Back](#contributing-back) +- [Cross-Instance Pull Requests](#cross-instance-pull-requests) +- [Permissions](#permissions) +- [SSRF Protection](#ssrf-protection) + +## Overview + +The two instances interoperate over HTTP. Instance A talks to B's REST API using a Personal Access Token belonging to the contributing user on B. All git traffic flows over the normal smart-HTTP protocol using basic auth with that PAT. + +``` +┌─────────────────┐ import (clone URL + PAT) ┌─────────────────┐ +│ Instance A │ ───────────────────────────▶ │ Instance B │ +│ bob/fedbase │ │ alice/fedbase │ +│ (fork) │ ◀─────────────────────────── │ (upstream) │ +│ │ push-upstream (git push) │ │ +│ │ ───────────────────────────▶ │ │ +│ │ federation/open-pull │ external-pulls │ +└─────────────────┘ ◀─────────────────────────── └─────────────────┘ +``` + +## How It Works + +1. **Import (fork)**: On instance A, import `https://B/alice/fedbase.git` and authenticate with a PAT for the user on B. A detects that the source is another OpenCodeHub instance (`GET /api/instance`) and records the relationship as `forkedFromUrl`. +2. **Contribute back**: A can push any branch on its fork back to B (`federation/push-upstream`). B's existing `git-receive-pack` authorizes the push via `canWriteRepo` — B controls who may contribute. +3. **Cross-instance PR**: A calls `federation/open-pull` on B. B validates the fork URL (SSRF check), fetches the head branch from A's fork into the B repo, computes diff stats, and creates a PR whose head lives on A's fork. +4. **Review & merge**: The PR is a normal PR on B. Reviews, approvals, CI, and the merge queue all apply. Merging fetches the head ref and merges it into the base branch. + +## Setting Up Federation + +Federation is enabled out of the box. Two environment variables control SSRF behavior: + +| Variable | Purpose | +|----------|---------| +| `FEDERATION_ALLOW_LOCALHOST` | Set `true` to allow fetching from `localhost`/loopback URLs. **Only** for two-instance testing on a single host or a trusted private network. Never enable in production. | + +The upstream instance does not need any special config beyond the normal collaboration settings: the A user simply needs read access (to see the repo) and write access (to push branches and open PRs). + +## Forking a Repository from Another Instance + +On instance A, use the normal **Import Repository** flow: + +1. Paste the upstream clone URL, e.g. `http://instance-b.local/swadhinbiswas/fedbase.git`. +2. Enter the **auth username** and a **PAT** for the user on instance B (the username must match the B account, e.g. `bob`). +3. Complete the import. Instance A records `forkedFromUrl` and stores an encrypted mirror PAT so it can push back later. + +> For cross-instance PRs the source URL must be fetchable by B — the fork must be on a host B can reach, and the fork repo must be readable by B (public, or accessible with the embedded PAT). + +## Contributing Back + +From the fork's repository page on A, the **Federation** panel offers **Push branch to upstream**: + +- Pick a branch on the fork. +- A pushes `branch:branch` to the upstream clone URL, authenticating with the stored B user's PAT. + +B's permission model decides whether the push succeeds — the B user must be a collaborator (or the repo must allow external writes). + +## Cross-Instance Pull Requests + +From the same **Federation** panel, **Open cross-instance PR**: + +1. A calls `POST /api/repos/{owner}/{repo}/external-pulls` on B (server-to-server) with the fork URL, head branch, base branch, title, and body. +2. B validates the fork URL, fetches the head into the B repo, computes additions/deletions/changed files, and creates the PR. +3. The PR's head branch resolves to the fetched ref on B, so the diff, comments, and merge all work normally. + +### Endpoints + +| Endpoint (on B) | Purpose | +|-----------------|---------| +| `POST /api/repos/{owner}/{repo}/external-pulls` | Create a PR whose head is fetched from an external fork URL | +| `GET /api/instance` | Instance metadata probe used for detection | + +## Permissions + +- **Import on A**: requires a valid PAT for the B user. +- **Push upstream**: B's `git-receive-pack` requires the B user to have write access to the repo. +- **Open a cross-instance PR**: B's `external-pulls` endpoint requires the caller to have read access plus either write access to the repo, or the repo must have **Allow external pull requests** enabled (repo → Settings → Federation). +- **Merge**: normal PR merge gates apply (approvals, required checks, branch protection). + +## SSRF Protection + +Every server-initiated fetch (import on A, external-pulls on B) runs through `validateGitCloneUrl`, which blocks private ranges, cloud metadata endpoints, and non-HTTP(S)/git/ssh schemes. The localhost bypass is only available when `FEDERATION_ALLOW_LOCALHOST=true`, and callers gate it explicitly. + +Tokens embedded into fetch/push URLs are used transiently and never persisted. diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index f5b91ae7..554a32eb 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -1,6 +1,6 @@ # Installation Guide -OpenCodeHub allows you to host your own GitHub-like platform. You can run it via **Docker** (recommended for production) or **Node.js** (for development/custom setups). +OpenCodeHub allows you to host your own GitHub-like platform. You can run it via **Docker** (recommended for production) or **Node.js / Bun** (for development/custom setups). ## 📋 System Requirements @@ -9,7 +9,41 @@ OpenCodeHub allows you to host your own GitHub-like platform. You can run it via | **CPU** | 1 vCPU | 2 vCPU | | **RAM** | 512MB | 2GB | | **Disk** | 10GB | 50GB SSD | -| **OS** | Linux (Ubuntu/Debian) | Linux | +| **OS** | Linux (Ubuntu/Debian) | Linux / macOS / Windows | + +--- + +## 💻 CLI Installation (`opencodehub-cli`) + +Install the official OpenCodeHub CLI (`och`) to interact with your instance directly from the terminal: + +```bash +# via npm +npm install -g opencodehub-cli + +# via bun +bun add -g opencodehub-cli + +# via pnpm +pnpm add -g opencodehub-cli + +# via yarn +yarn global add opencodehub-cli +``` + +Verify the CLI is ready: + +```bash +och --version +och --help +``` + +Authenticate with your server: + +```bash +och auth login --url https://git.yourcompany.com +och config doctor +``` --- @@ -44,6 +78,8 @@ Edit `.env` and set **production values**. JWT_SECRET= SESSION_SECRET= INTERNAL_HOOK_SECRET= +CRON_SECRET= +RUNNER_SECRET= # Domain Configuration SITE_URL=https://git.yourcompany.com @@ -51,27 +87,30 @@ PORT=4321 # Database (Using the Postgres container defined in compose) DATABASE_URL=postgresql://opencodehub:securepassword@postgres:5432/opencodehub +POSTGRES_USER=opencodehub +POSTGRES_PASSWORD=securepassword +POSTGRES_DB=opencodehub + +# Redis +REDIS_URL=redis://:redispassword@redis:6379 +REDIS_PASSWORD=redispassword -# Object Storage (Highly Recommended for Production) -STORAGE_TYPE=s3 -STORAGE_BUCKET=my-git-bucket -STORAGE_REGION=us-east-1 -STORAGE_ACCESS_KEY_ID=... -STORAGE_SECRET_ACCESS_KEY=... -# STORAGE_ENDPOINT=https://.r2.cloudflarestorage.com # set for non-AWS vendors +# Object Storage (Optional for S3 backends) +STORAGE_TYPE=local +STORAGE_PATH=/data/storage ``` ### 4. Start Services ```bash -docker-compose up -d +docker compose up -d ``` ### 5. Initialization Initialize the admin user: ```bash -docker-compose exec app bun run scripts/seed-admin.ts +docker compose exec app bun scripts/seed-admin.ts ``` --- @@ -98,30 +137,14 @@ server { client_max_body_size 500M; location / { - proxy_pass http://localhost:4321; + proxy_pass http://127.0.0.1:4321; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; + proxy_buffering off; + proxy_read_timeout 600s; + proxy_send_timeout 600s; } } ``` - ---- - -## ✅ Production Checklist - -Before going live to users: - -- [ ] **Secrets Rotated**: Default secrets replaced with strong random strings. -- [ ] **HTTPS Enabled**: SSL certificate configured via Nginx/Caddy. -- [ ] **Rate Limiting**: `RATE_LIMIT_*` env vars adjusted for expected load. -- [ ] **Monitoring**: Grafana/Sentry configured for error tracking ([Guide](../administration/monitoring.md)). - ---- - -## Next Steps - -Now that you have OpenCodeHub up and running, let's look around: - -👉 **[Quick Start Guide](quick-start.md)** diff --git a/drizzle/0008_add_push_mirror.sql b/drizzle/0008_add_push_mirror.sql new file mode 100644 index 00000000..a1974e6c --- /dev/null +++ b/drizzle/0008_add_push_mirror.sql @@ -0,0 +1,5 @@ +ALTER TABLE "repositories" ADD COLUMN "push_mirror_enabled" boolean DEFAULT false NOT NULL;--> statement-breakpoint +ALTER TABLE "repositories" ADD COLUMN "push_mirror_url" text;--> statement-breakpoint +ALTER TABLE "repositories" ADD COLUMN "push_mirror_token" text;--> statement-breakpoint +ALTER TABLE "repositories" ADD COLUMN "last_push_mirror_at" timestamp;--> statement-breakpoint +ALTER TABLE "repositories" ADD COLUMN "push_mirror_status" text; diff --git a/drizzle/0009_add_webhook_queue.sql b/drizzle/0009_add_webhook_queue.sql new file mode 100644 index 00000000..47e80e18 --- /dev/null +++ b/drizzle/0009_add_webhook_queue.sql @@ -0,0 +1,5 @@ +ALTER TABLE "webhook_deliveries" ADD COLUMN "attempts" integer DEFAULT 0 NOT NULL;--> statement-breakpoint +ALTER TABLE "webhook_deliveries" ADD COLUMN "next_attempt_at" timestamp;--> statement-breakpoint +ALTER TABLE "webhook_deliveries" ADD COLUMN "locked_at" timestamp;--> statement-breakpoint +ALTER TABLE "webhook_deliveries" ADD COLUMN "failure_reason" text;--> statement-breakpoint +CREATE INDEX IF NOT EXISTS "webhook_deliveries_queue_idx" ON "webhook_deliveries" ("status","next_attempt_at"); diff --git a/drizzle/meta/_journal.json b/drizzle/meta/_journal.json index bbc99ced..d4eca5dd 100644 --- a/drizzle/meta/_journal.json +++ b/drizzle/meta/_journal.json @@ -57,6 +57,20 @@ "when": 1787125181115, "tag": "0007_add_mirror_username", "breakpoints": true + }, + { + "idx": 8, + "version": "7", + "when": 1787400000000, + "tag": "0008_add_push_mirror", + "breakpoints": true + }, + { + "idx": 9, + "version": "7", + "when": 1787500000000, + "tag": "0009_add_webhook_queue", + "breakpoints": true } ] } \ No newline at end of file diff --git a/package.json b/package.json index 23334bf5..302c2f29 100644 --- a/package.json +++ b/package.json @@ -106,7 +106,6 @@ "astro-icon": "^1.1.5", "bcryptjs": "^2.4.3", "better-sqlite3": "^11.1.2", - "bullmq": "^5.65.1", "class-variance-authority": "^0.7.0", "clsx": "^2.1.1", "cmdk": "^1.1.1", diff --git a/scripts/worker.ts b/scripts/worker.ts index 246afabc..b15703bd 100644 --- a/scripts/worker.ts +++ b/scripts/worker.ts @@ -3,7 +3,10 @@ import type { NodePgDatabase } from "drizzle-orm/node-postgres"; import { cleanupAllRepos } from "@/lib/cron/cleanup-branches"; import { logger } from "@/lib/logger"; import { syncAllMirrors } from "@/lib/mirror-sync"; +import { processDuePushMirrors } from "@/lib/push-mirror"; +import { processWebhookQueue } from "@/lib/webhooks"; import { queueWorker } from "@/lib/queue-worker"; +import { publishRealtimeEvent } from "@/lib/realtime"; import { runDueDigests } from "@/lib/chat-notifications"; import { eq } from "drizzle-orm"; import { createServer } from "http"; @@ -14,6 +17,10 @@ const MIRROR_SYNC_INTERVAL = parseInt(process.env.MIRROR_SYNC_INTERVAL || "60000 const CLEANUP_INTERVAL = parseInt(process.env.CLEANUP_INTERVAL || "3600000", 10); const DIGEST_INTERVAL = parseInt(process.env.DIGEST_INTERVAL || "300000", 10); const SCHEDULE_INTERVAL = parseInt(process.env.SCHEDULE_INTERVAL || "60000", 10); +const WEBHOOK_POLL_INTERVAL_MS = parseInt(process.env.WEBHOOK_POLL_INTERVAL_MS || "5000", 10); +const WEBHOOK_QUEUE_BATCH = parseInt(process.env.WEBHOOK_QUEUE_BATCH || "20", 10); +const PUSH_MIRROR_INTERVAL_MS = parseInt(process.env.PUSH_MIRROR_INTERVAL_MS || "60000", 10); +const PUSH_MIRROR_BATCH = parseInt(process.env.PUSH_MIRROR_BATCH || "10", 10); const HEALTH_PORT = parseInt(process.env.WORKER_HEALTH_PORT || "9090", 10); const MAX_RETRIES = parseInt(process.env.WORKER_MAX_RETRIES || "3", 10); const STALE_JOB_TIMEOUT_MS = parseInt(process.env.WORKER_STALE_TIMEOUT || "300000", 10); @@ -29,6 +36,8 @@ let lastMirrorRun = 0; let lastCleanupRun = 0; let lastDigestRun = 0; let lastScheduleRun = 0; +let lastWebhookRun = 0; +let lastPushMirrorRun = 0; // Circuit breaker state per task interface CircuitBreakerState { @@ -140,6 +149,13 @@ async function runQueueProcessor() { if (isShuttingDown) break; try { await queueWorker.processQueue(repoId); + // Notify browsers connected to the web process (separate OS process) + // that queue positions may have changed for this repository. + await publishRealtimeEvent({ kind: "repository", repositoryId: repoId }, { + type: "queue:position_changed", + timestamp: new Date(), + data: { repositoryId: repoId }, + }); } catch (error) { logger.error({ err: error, repoId }, "Failed to process queue for repo"); recordFailure("queue-processor"); @@ -155,6 +171,62 @@ async function runQueueProcessor() { } } +// ── Webhook Delivery Processor ──────────────────────────────────────────────── +async function runWebhookProcessor() { + if (isCircuitOpen("webhook-processor")) { + logger.warn("Webhook processor circuit open — skipping"); + return; + } + + try { + const result = await processWebhookQueue(WEBHOOK_QUEUE_BATCH); + if (result.claimed > 0 || result.swept > 0) { + logger.info( + { + swept: result.swept, + claimed: result.claimed, + delivered: result.delivered, + retried: result.retried, + dead: result.dead, + }, + "Processed webhook delivery queue", + ); + } + recordSuccess("webhook-processor"); + lastWebhookRun = Date.now(); + } catch (error) { + recordFailure("webhook-processor"); + logger.error({ err: error }, "Fatal error in webhook processor loop"); + } +} + +// ── Push Mirror Processor ───────────────────────────────────────────────────── +async function runPushMirrorProcessor() { + if (isCircuitOpen("push-mirror")) { + logger.warn("Push mirror circuit open — skipping"); + return; + } + + try { + const result = await processDuePushMirrors({ limit: PUSH_MIRROR_BATCH }); + if (result.pushed > 0 || result.failed > 0) { + logger.info( + { + pushed: result.pushed, + failed: result.failed, + durationMs: result.durationMs, + }, + "Processed push mirrors", + ); + } + recordSuccess("push-mirror"); + lastPushMirrorRun = Date.now(); + } catch (error) { + recordFailure("push-mirror"); + logger.error({ err: error }, "Fatal error in push mirror loop"); + } +} + // ── Controlled Loop with Exponential Backoff ─────────────────────────────────── async function runLoop( name: string, @@ -200,6 +272,7 @@ function startHealthServer() { status: healthy ? "healthy" : "unhealthy", uptime: process.uptime(), lastQueueRun: new Date(lastQueueRun).toISOString(), + lastWebhookRun: new Date(lastWebhookRun).toISOString(), circuitBreakers: Object.fromEntries( Array.from(circuitBreakers.entries()).map(([k, v]) => [ k, @@ -265,6 +338,10 @@ async function startWorker() { queueInterval: WORKER_INTERVAL, mirrorInterval: MIRROR_SYNC_INTERVAL, cleanupInterval: CLEANUP_INTERVAL, + webhookPollIntervalMs: WEBHOOK_POLL_INTERVAL_MS, + webhookQueueBatch: WEBHOOK_QUEUE_BATCH, + pushMirrorIntervalMs: PUSH_MIRROR_INTERVAL_MS, + pushMirrorBatch: PUSH_MIRROR_BATCH, healthPort: HEALTH_PORT, maxRetries: MAX_RETRIES, circuitBreakerThreshold: CIRCUIT_BREAKER_THRESHOLD, @@ -291,6 +368,8 @@ async function startWorker() { const { pipelineRunner } = await import("@/lib/pipeline"); await runScheduledWorkflows(pipelineRunner); }, SCHEDULE_INTERVAL, () => lastScheduleRun, (t) => { lastScheduleRun = t; }), + runLoop("webhook-processor", runWebhookProcessor, WEBHOOK_POLL_INTERVAL_MS, () => lastWebhookRun, (t) => { lastWebhookRun = t; }), + runLoop("push-mirror", runPushMirrorProcessor, PUSH_MIRROR_INTERVAL_MS, () => lastPushMirrorRun, (t) => { lastPushMirrorRun = t; }), ]); } diff --git a/src/components/layout/Header.astro b/src/components/layout/Header.astro index 283d1ec0..8b74201d 100644 --- a/src/components/layout/Header.astro +++ b/src/components/layout/Header.astro @@ -129,6 +129,10 @@ const { token: csrfToken } = getCsrfToken(Astro.request); Your stars + + + Your gists +
diff --git a/src/components/repo/RepoHeader.astro b/src/components/repo/RepoHeader.astro index a3ebf337..31bff626 100644 --- a/src/components/repo/RepoHeader.astro +++ b/src/components/repo/RepoHeader.astro @@ -14,6 +14,7 @@ import { GitBranch, GitMerge, CircleDot, + MessageSquare, BookOpen } from "lucide-react"; import { Badge } from "@/components/ui/badge"; @@ -44,6 +45,7 @@ interface Props { activeTab: | "code" | "issues" + | "discussions" | "pulls" | "actions" | "merge-queue" @@ -93,6 +95,12 @@ const tabs = [ href: `/${repo.owner}/${repo.name}/issues`, count: repo.openIssueCount, }, + { + id: "discussions", + label: "Discussions", + icon: MessageSquare, + href: `/${repo.owner}/${repo.name}/discussions`, + }, { id: "pulls", label: "Pull requests", diff --git a/src/db/schema/discussions.ts b/src/db/schema/discussions.ts new file mode 100644 index 00000000..20b4b31b --- /dev/null +++ b/src/db/schema/discussions.ts @@ -0,0 +1,89 @@ +/** + * Discussions Schema - Drizzle ORM + * GitHub-style repository discussions and threaded comments + */ + +import { + boolean, + index, + integer, + pgTable, + text, + timestamp, +} from "drizzle-orm/pg-core"; +import { repositories } from "./repositories"; +import { users } from "./users"; + +export const DISCUSSION_CATEGORIES = [ + "General", + "Ideas", + "Q&A", + "Show and tell", +] as const; + +export type DiscussionCategory = (typeof DISCUSSION_CATEGORIES)[number]; + +export const discussions = pgTable( + "discussions", + { + id: text("id").primaryKey(), + repositoryId: text("repository_id") + .notNull() + .references(() => repositories.id, { onDelete: "cascade" }), + authorId: text("author_id") + .notNull() + .references(() => users.id), + title: text("title").notNull(), + body: text("body").notNull(), + category: text("category").notNull().default("General"), + pinned: boolean("pinned").default(false), + closed: boolean("closed").default(false), + commentCount: integer("comment_count").default(0), + lastActivityAt: timestamp("last_activity_at"), + createdAt: timestamp("created_at").notNull().defaultNow(), + updatedAt: timestamp("updated_at").notNull().defaultNow(), + }, + (t) => ({ + repoClosedIdx: index("discussions_repo_closed_idx").on( + t.repositoryId, + t.closed, + ), + repoActivityIdx: index("discussions_repo_activity_idx").on( + t.repositoryId, + t.lastActivityAt, + ), + authorIdx: index("discussions_author_idx").on(t.authorId), + }), +); + +export const discussionComments = pgTable( + "discussion_comments", + { + id: text("id").primaryKey(), + discussionId: text("discussion_id") + .notNull() + .references(() => discussions.id, { onDelete: "cascade" }), + // Self-reference for one-level threading; stored but treated flat in UI v1. + // Plain text column (no FK) mirrors the issues.parent_id convention to avoid + // circular type references in Drizzle. + parentId: text("parent_id"), + authorId: text("author_id") + .notNull() + .references(() => users.id), + body: text("body").notNull(), + createdAt: timestamp("created_at").notNull().defaultNow(), + updatedAt: timestamp("updated_at").notNull().defaultNow(), + }, + (t) => ({ + discussionIdx: index("discussion_comments_discussion_idx").on( + t.discussionId, + ), + parentIdx: index("discussion_comments_parent_idx").on(t.parentId), + }), +); + +// Types +export type Discussion = typeof discussions.$inferSelect; +export type NewDiscussion = typeof discussions.$inferInsert; +export type DiscussionComment = typeof discussionComments.$inferSelect; +export type NewDiscussionComment = typeof discussionComments.$inferInsert; diff --git a/src/db/schema/gists.ts b/src/db/schema/gists.ts new file mode 100644 index 00000000..916507ed --- /dev/null +++ b/src/db/schema/gists.ts @@ -0,0 +1,54 @@ +/** + * Gists Schema - Drizzle ORM + * GitHub-style standalone code snippets (multi-file, public or secret). + * V1 scope: no stars, forks, or comments. + */ + +import { relations } from "drizzle-orm"; +import { + boolean, + index, + jsonb, + pgTable, + text, + timestamp, +} from "drizzle-orm/pg-core"; +import { users } from "./users"; + +/** A single file inside a gist */ +export interface GistFile { + filename: string; + content: string; +} + +export const gists = pgTable( + "gists", + { + id: text("id").primaryKey(), + userId: text("user_id") + .notNull() + .references(() => users.id, { onDelete: "cascade" }), + description: text("description").notNull().default(""), + // false = secret (unlisted but link-accessible), true = public + public: boolean("public").notNull().default(false), + files: jsonb("files").$type().notNull(), + createdAt: timestamp("created_at").notNull().defaultNow(), + updatedAt: timestamp("updated_at").notNull().defaultNow(), + }, + (t) => ({ + userUpdatedIdx: index("gists_user_updated_idx").on(t.userId, t.updatedAt), + publicIdx: index("gists_public_idx").on(t.public), + }), +); + +// Relations +export const gistsRelations = relations(gists, ({ one }) => ({ + user: one(users, { + fields: [gists.userId], + references: [users.id], + }), +})); + +// Types +export type Gist = typeof gists.$inferSelect; +export type NewGist = typeof gists.$inferInsert; diff --git a/src/db/schema/index.ts b/src/db/schema/index.ts index 14b0fd85..15b0f5a3 100644 --- a/src/db/schema/index.ts +++ b/src/db/schema/index.ts @@ -112,3 +112,6 @@ export * from "./packages"; export * from "./review-analysis"; export * from "./oauth-apps"; export * from "./org-invites"; + +// Gists (standalone code snippets) +export * from "./gists"; diff --git a/src/db/schema/repositories.ts b/src/db/schema/repositories.ts index 3936795e..b06066e5 100644 --- a/src/db/schema/repositories.ts +++ b/src/db/schema/repositories.ts @@ -50,6 +50,11 @@ export const repositories = pgTable( mirrorUsername: text("mirror_username"), // Upstream username for basic-auth (federation push-back) lastMirrorSyncAt: timestamp("last_mirror_sync_at"), mirrorSyncStatus: text("mirror_sync_status"), // pending, syncing, success, failed + pushMirrorEnabled: boolean("push_mirror_enabled").default(false).notNull(), + pushMirrorUrl: text("push_mirror_url"), // Downstream remote for push mirroring (one per repo) + pushMirrorToken: text("push_mirror_token"), // Encrypted downstream auth token for push mirrors + lastPushMirrorAt: timestamp("last_push_mirror_at"), + pushMirrorStatus: text("push_mirror_status"), // pending, pushing, success, failed hasIssues: boolean("has_issues").default(true), hasWiki: boolean("has_wiki").default(true), hasActions: boolean("has_actions").default(true), diff --git a/src/db/schema/webhooks.ts b/src/db/schema/webhooks.ts index 9751bca8..88b02003 100644 --- a/src/db/schema/webhooks.ts +++ b/src/db/schema/webhooks.ts @@ -57,7 +57,14 @@ export const webhookDeliveries = pgTable( event: text("event").notNull(), payload: text("payload").notNull(), // JSON - status: text("status").notNull(), // success | failure + // Queue lifecycle: pending | processing | delivered | dead. + // Legacy rows (pre-queue) may still carry success | failure. + status: text("status").notNull(), + attempts: integer("attempts").notNull().default(0), + nextAttemptAt: timestamp("next_attempt_at"), + lockedAt: timestamp("locked_at"), + failureReason: text("failure_reason"), + responseCode: integer("response_code"), responseBody: text("response_body"), durationMs: integer("duration_ms"), @@ -71,6 +78,7 @@ export const webhookDeliveries = pgTable( (t) => ({ webhookIdx: index("webhook_deliveries_webhook_idx").on(t.webhookId), createdAtIdx: index("webhook_deliveries_created_idx").on(t.createdAt), + queueIdx: index("webhook_deliveries_queue_idx").on(t.status, t.nextAttemptAt), }), ); diff --git a/src/lib/auth-cache.ts b/src/lib/auth-cache.ts new file mode 100644 index 00000000..70c9f363 --- /dev/null +++ b/src/lib/auth-cache.ts @@ -0,0 +1,129 @@ +/** + * Short-TTL in-memory cache for middleware auth lookups. + * + * The request pipeline resolves the full user row (and session row) on every + * request — 2 DB queries per request under load. This cache absorbs that cost + * with a small, bounded, short-TTL window. Revocation latency is therefore + * bounded by AUTH_CACHE_TTL_MS; logout calls invalidateAuthCache for + * immediate effect. + * + * Hit/miss outcomes are recorded via the cacheHits/cacheMisses Prometheus + * counters from src/lib/metrics.ts. + */ +import { cacheHits, cacheMisses } from "@/lib/metrics"; +import { logger } from "@/lib/logger"; + +const TTL_MS = Math.max( + 1000, + parseInt(process.env.AUTH_CACHE_TTL_MS || "15000", 10), +); +const MAX_ENTRIES = 5000; + +interface CacheEntry { + value: T; + expiresAt: number; +} + +const userCache = new Map>(); +interface SessionCacheEntry extends CacheEntry { + ownerUserId?: string; +} +const sessionCache = new Map(); + +function evictOldest( + map: Map | SessionCacheEntry>, +): void { + while (map.size >= MAX_ENTRIES) { + const oldest = map.keys().next().value; + if (oldest === undefined) break; + map.delete(oldest); + } +} + +function getCached( + map: Map>, + key: string, +): unknown | undefined { + const entry = map.get(key); + if (!entry) { + cacheMisses.inc(); + return undefined; + } + if (Date.now() > entry.expiresAt) { + map.delete(key); + cacheMisses.inc(); + return undefined; + } + cacheHits.inc(); + return entry.value; +} + +function setCached( + map: Map | SessionCacheEntry>, + key: string, + value: unknown, + ownerUserId?: string, +): void { + evictOldest(map); + map.set(key, { + value, + expiresAt: Date.now() + TTL_MS, + ...(ownerUserId ? { ownerUserId } : {}), + }); +} + +export async function cachedUserLookup( + userId: string, + lookup: () => Promise, +): Promise { + const hit = getCached(userCache, userId); + if (hit !== undefined) return hit as T; + const value = await lookup(); + if (value !== null && value !== undefined) { + setCached(userCache, userId, value); + } + return value ?? null; +} + +export async function cachedSessionLookup( + sessionId: string, + lookup: () => Promise, +): Promise { + const hit = getCached(sessionCache, sessionId); + if (hit !== undefined) return hit as T; + const value = await lookup(); + if ( + value !== null && + value !== undefined && + typeof value === "object" && + "userId" in (value as Record) + ) { + setCached( + sessionCache, + sessionId, + value, + (value as { userId?: string }).userId, + ); + } + return value ?? null; +} + +/** Invalidate all cached entries for a user (call on logout/password change). */ +export function invalidateAuthCache(userId?: string, sessionId?: string): void { + if (userId) userCache.delete(userId); + if (sessionId) sessionCache.delete(sessionId); + logger.debug({ userId, sessionId }, "Auth cache invalidated"); +} + +/** + * Drop every cached session row owned by a user — used when all their + * sessions are revoked server-side (password change/reset) so cached rows + * cannot outlive the revocation for the TTL window. + */ +export function revokeUserSessionCache(userId: string): void { + userCache.delete(userId); + for (const [sessionId, entry] of sessionCache) { + if (entry.ownerUserId === userId) sessionCache.delete(sessionId); + } + logger.debug({ userId }, "User session cache revoked"); +} diff --git a/src/lib/commit-signature.ts b/src/lib/commit-signature.ts new file mode 100644 index 00000000..c2cd958e --- /dev/null +++ b/src/lib/commit-signature.ts @@ -0,0 +1,317 @@ +/** + * Commit Signature Verification Library + * GitHub-style "Verified" commit badges + * Verifies git commit GPG signatures against user-uploaded keys + * in the gpg_keys table using openpgp (no system keyring needed) + */ + +import { getDatabase, schema } from "@/db"; +import { and, eq, inArray } from "drizzle-orm"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import * as openpgp from "openpgp"; + +import { logger } from "./logger"; + +// Types + +export interface CommitVerification { + /** Commit object contains a gpgsig/gpgsig-sha256 header */ + signed: boolean; + /** Signature cryptographically valid AND owned by the committer's account */ + verified: boolean; + /** A public key stored in gpg_keys verified the signature */ + validKeyInDb: boolean; + /** users.id of the committer matched by committer email, if registered */ + signerUserId: string | null; +} + +const UNSIGNED_RESULT: CommitVerification = { + signed: false, + verified: false, + validKeyInDb: false, + signerUserId: null, +}; + +// Cache + +const CACHE_TTL_MS = 10 * 60 * 1000; +const CACHE_MAX_ENTRIES = 2000; + +interface CacheEntry { + result: CommitVerification; + expiresAt: number; +} + +const verificationCache = new Map(); + +function cacheGet(key: string): CommitVerification | null { + const entry = verificationCache.get(key); + if (!entry) return null; + if (Date.now() > entry.expiresAt) { + verificationCache.delete(key); + return null; + } + return entry.result; +} + +function cacheSet(key: string, result: CommitVerification): void { + while (verificationCache.size >= CACHE_MAX_ENTRIES) { + const oldest = verificationCache.keys().next().value; + if (oldest === undefined) break; + verificationCache.delete(oldest); + } + verificationCache.set(key, { result, expiresAt: Date.now() + CACHE_TTL_MS }); +} + +// Signature extraction + +/** + * Parse the raw output of `git cat-file commit `. + * Returns the armored signature block and the exact commit bytes with all + * gpgsig/gpgsig-sha256 header lines stripped (this is what Git signs). + */ +export function extractSignatureFromCommit( + rawCommitObject: string, +): { signature: string; signedData: string } | null { + const separatorIndex = rawCommitObject.indexOf("\n\n"); + const headerSection = + separatorIndex === -1 ? rawCommitObject : rawCommitObject.slice(0, separatorIndex); + const tail = separatorIndex === -1 ? "" : rawCommitObject.slice(separatorIndex); + + const headerLines = headerSection.split("\n"); + const keptLines: string[] = []; + let signatureLines: string[] | null = null; + let collecting = false; + + for (const line of headerLines) { + if (/^gpgsig(-sha256)?( |$)/.test(line)) { + if (!signatureLines) { + signatureLines = []; + collecting = true; + } else { + collecting = false; + keptLines.push(line); + } + continue; + } + if (collecting && line.startsWith(" ")) { + signatureLines!.push(line.slice(1)); + continue; + } + collecting = false; + keptLines.push(line); + } + + if (!signatureLines || signatureLines.length === 0) { + return null; + } + + const signature = signatureLines.join("\n"); + const signedData = `${keptLines.join("\n")}${tail}`; + return { signature, signedData }; +} + +function parseCommitterEmail(rawCommitObject: string): string | null { + const match = rawCommitObject.match(/^committer [^\n]*<([^>]*)>/m); + return match ? match[1] : null; +} + +// Verification + +interface VerificationContext { + db: NodePgDatabase; + repoPath: string; +} + +async function loadContext( + owner: string, + repo: string, +): Promise { + const db = getDatabase() as NodePgDatabase; + const ownerUser = await db.query.users.findFirst({ + where: eq(schema.users.username, owner), + }); + if (!ownerUser) return null; + + const repoRow = await db.query.repositories.findFirst({ + where: and( + eq(schema.repositories.ownerId, ownerUser.id), + eq(schema.repositories.name, repo), + ), + }); + if (!repoRow) return null; + + const { resolveRepoPath } = await import("./git-storage"); + const repoPath = await resolveRepoPath(repoRow.diskPath); + return { db, repoPath }; +} + +async function readRawCommit(repoPath: string, sha: string): Promise { + if (!/^[0-9a-f]{7,64}$/i.test(sha)) return null; + const { simpleGit } = await import("simple-git"); + try { + return await simpleGit(repoPath).raw(["cat-file", "commit", sha]); + } catch { + return null; + } +} + +async function verifyWithKeys( + ctx: VerificationContext, + sha: string, + extracted: { signature: string; signedData: string }, +): Promise { + const sig = await openpgp.readSignature({ armoredSignature: extracted.signature }); + const signingKeyIds = sig + .getSigningKeyIDs() + .map((id) => id.toHex().toLowerCase()) + .filter(Boolean); + + for (const keyId of signingKeyIds) { + const cached = cacheGet(`${sha}:${keyId}`); + if (cached) return cached; + } + + let keyRows = signingKeyIds.length + ? await ctx.db.select().from(schema.gpgKeys).where(inArray(schema.gpgKeys.keyId, signingKeyIds)) + : []; + if (!keyRows.length) { + keyRows = await ctx.db.select().from(schema.gpgKeys).limit(500); + } + + const message = await openpgp.createMessage({ text: extracted.signedData }); + + for (const row of keyRows) { + try { + const keyObj = await openpgp.readKey({ armoredKey: row.publicKey }); + const keyHex = keyObj.getKeyID().toHex().toLowerCase(); + const fingerprint = keyObj.getFingerprint().toLowerCase(); + if ( + signingKeyIds.length && + !signingKeyIds.some( + (id) => keyHex === id || fingerprint.endsWith(id) || keyHex.endsWith(id), + ) + ) { + continue; + } + + const result = await openpgp.verify({ + message, + verificationKeys: keyObj, + signature: sig, + }); + const firstSignature = result.signatures[0]; + if (!firstSignature) continue; + + const isValid = await firstSignature.verified.then( + () => true, + () => false, + ); + if (!isValid) continue; + + const committerEmail = parseCommitterEmail(extracted.signedData); + const signerUser = committerEmail + ? await ctx.db.query.users.findFirst({ + where: eq(schema.users.email, committerEmail), + }) + : null; + + const verification: CommitVerification = { + signed: true, + verified: !!signerUser && row.userId === signerUser.id, + validKeyInDb: true, + signerUserId: signerUser?.id ?? null, + }; + cacheSet(`${sha}:${fingerprint}`, verification); + cacheSet(`${sha}:${keyHex}`, verification); + return verification; + } catch { + continue; + } + } + + const fallback: CommitVerification = { + signed: true, + verified: false, + validKeyInDb: false, + signerUserId: null, + }; + for (const keyId of signingKeyIds) { + cacheSet(`${sha}:${keyId}`, fallback); + } + return fallback; +} + +async function verifySha( + ctx: VerificationContext, + sha: string, +): Promise { + const raw = await readRawCommit(ctx.repoPath, sha); + if (!raw) return { ...UNSIGNED_RESULT }; + + const extracted = extractSignatureFromCommit(raw); + if (!extracted) return { ...UNSIGNED_RESULT }; + + try { + return await verifyWithKeys(ctx, sha, extracted); + } catch (e) { + logger.warn({ err: e, sha }, "Commit signature verification failed"); + return { ...UNSIGNED_RESULT }; + } +} + +/** + * Verify a single commit's GPG signature against user-uploaded public keys. + * Never throws — any failure yields an "unsigned"-shaped result. + */ +export async function verifyCommitSignature(opts: { + owner: string; + repo: string; + sha: string; +}): Promise { + try { + const ctx = await loadContext(opts.owner, opts.repo); + if (!ctx) return { ...UNSIGNED_RESULT }; + return await verifySha(ctx, opts.sha); + } catch (e) { + logger.warn( + { err: e, owner: opts.owner, repo: opts.repo, sha: opts.sha }, + "Commit signature verification failed", + ); + return { ...UNSIGNED_RESULT }; + } +} + +/** + * Verify a page of commits sequentially (openpgp is CPU-bound). + * Returns a Map keyed by sha. Empty map when disabled via + * COMMIT_SIGNATURE_VERIFICATION=false. + */ +export async function verifyCommitsSignatures( + owner: string, + repo: string, + shas: string[], + maxBatch = 20, +): Promise> { + const results = new Map(); + if (process.env.COMMIT_SIGNATURE_VERIFICATION === "false") return results; + + const batch = shas.filter(Boolean).slice(0, maxBatch); + if (batch.length === 0) return results; + + let ctx: VerificationContext | null = null; + try { + ctx = await loadContext(owner, repo); + } catch (e) { + logger.warn({ err: e, owner, repo }, "Commit signature verification context failed"); + } + if (!ctx) { + for (const sha of batch) results.set(sha, { ...UNSIGNED_RESULT }); + return results; + } + + for (const sha of batch) { + results.set(sha, await verifySha(ctx, sha)); + } + return results; +} diff --git a/src/lib/docker-registry-upload.ts b/src/lib/docker-registry-upload.ts new file mode 100644 index 00000000..57a5b675 --- /dev/null +++ b/src/lib/docker-registry-upload.ts @@ -0,0 +1,175 @@ +/** + * Docker/OCI blob upload session management. + * + * Tracks in-flight chunked uploads (POST /v2/.../blobs/uploads/ → PATCH → PUT). + * Chunks land in a temp file under .tmp/docker-uploads/; the session map + * lives in-process (single-node self-hosted deployments are the primary target, + * mirroring how src/lib/action-resolver.ts handles its local cache). + * + * Security properties enforced here: + * - digest verification (sha256) before any blob is admitted to storage + * - TTL expiry (1h) and bounded session count so abandoned uploads cannot leak disk + */ +import { createHash, randomUUID } from "node:crypto"; +import { createReadStream } from "node:fs"; +import { appendFile, mkdir, rm, stat } from "node:fs/promises"; +import path from "node:path"; +import { logger } from "@/lib/logger"; + +const UPLOAD_TTL_MS = 60 * 60 * 1000; +const MAX_SESSIONS = 500; +const CLEANUP_INTERVAL_MS = 5 * 60 * 1000; + +export function maxBlobBytes(): number { + const mb = parseInt(process.env.DOCKER_MAX_BLOB_MB || "8192", 10); + return (Number.isFinite(mb) && mb > 0 ? mb : 8192) * 1024 * 1024; +} + +export interface BlobUploadSession { + id: string; + imageName: string; + size: number; + createdAt: number; +} + +const sessions = new Map(); +let cleanupTimer: NodeJS.Timeout | null = null; + +function uploadDir(): string { + return path.join(process.cwd(), ".tmp", "docker-uploads"); +} + +function uploadPath(id: string): string { + return path.join(uploadDir(), id); +} + +function ensureCleanup(): void { + if (cleanupTimer) return; + cleanupTimer = setInterval(() => { + const now = Date.now(); + for (const [id, session] of sessions) { + if (now - session.createdAt > UPLOAD_TTL_MS) { + sessions.delete(id); + void rm(uploadPath(id), { force: true }).catch(() => {}); + } + } + }, CLEANUP_INTERVAL_MS); + cleanupTimer.unref(); +} + +function evictOldestIfNeeded(): void { + while (sessions.size >= MAX_SESSIONS) { + let oldestId: string | null = null; + let oldestAt = Infinity; + for (const [id, session] of sessions) { + if (session.createdAt < oldestAt) { + oldestAt = session.createdAt; + oldestId = id; + } + } + if (!oldestId) break; + sessions.delete(oldestId); + void rm(uploadPath(oldestId), { force: true }).catch(() => {}); + } +} + +export async function createUploadSession( + imageName: string, +): Promise { + ensureCleanup(); + evictOldestIfNeeded(); + await mkdir(uploadDir(), { recursive: true }); + const id = randomUUID(); + const session: BlobUploadSession = { + id, + imageName, + size: 0, + createdAt: Date.now(), + }; + sessions.set(id, session); + await appendFile(uploadPath(id), Buffer.alloc(0)); + return session; +} + +export function getUploadSession(id: string): BlobUploadSession | null { + return sessions.get(id) ?? null; +} + +export async function appendToUpload( + id: string, + chunk: Buffer, +): Promise< + { ok: true; session: BlobUploadSession } | { ok: false; reason: string } +> { + const session = sessions.get(id); + if (!session) return { ok: false, reason: "BLOB_UPLOAD_UNKNOWN" }; + const newSize = session.size + chunk.length; + if (newSize > maxBlobBytes()) { + await cancelUpload(id); + return { ok: false, reason: "BLOB_UPLOAD_INVALID" }; + } + await appendFile(uploadPath(id), chunk); + session.size = newSize; + return { ok: true, session }; +} + +/** + * Verify the assembled temp file against the expected digest and move it into + * the storage adapter at the canonical content-addressed key. + */ +export async function finalizeUpload( + id: string, + expectedDigest: string, + storageKey: string, +): Promise< + | { ok: true; digest: string; size: number } + | { ok: false; reason: string; status: number } +> { + const session = sessions.get(id); + if (!session) { + return { ok: false, reason: "BLOB_UPLOAD_UNKNOWN", status: 404 }; + } + + const filePath = uploadPath(id); + let fileInfo; + try { + fileInfo = await stat(filePath); + } catch { + sessions.delete(id); + return { ok: false, reason: "BLOB_UPLOAD_UNKNOWN", status: 404 }; + } + + const hash = createHash("sha256"); + await new Promise((resolve, reject) => { + const stream = createReadStream(filePath); + stream.on("data", (chunk) => hash.update(chunk)); + stream.on("end", () => resolve()); + stream.on("error", reject); + }); + const computed = `sha256:${hash.digest("hex")}`; + + if (computed !== expectedDigest) { + logger.warn( + { expected: expectedDigest, computed, image: session.imageName }, + "Docker blob digest mismatch", + ); + await cancelUpload(id); + return { ok: false, reason: "DIGEST_INVALID", status: 400 }; + } + + const { getStorage } = await import("@/lib/storage"); + const storage = await getStorage(); + await storage.put(storageKey, createReadStream(filePath), { + contentType: "application/octet-stream", + }); + + sessions.delete(id); + await rm(filePath, { force: true }).catch(() => {}); + + return { ok: true, digest: computed, size: fileInfo.size }; +} + +export async function cancelUpload(id: string): Promise { + sessions.delete(id); + await rm(uploadPath(id), { force: true }).catch(() => {}); +} diff --git a/src/lib/git.ts b/src/lib/git.ts index 60abb773..09ffeee1 100644 --- a/src/lib/git.ts +++ b/src/lib/git.ts @@ -15,6 +15,43 @@ import { tmpdir } from "os"; import { basename, dirname, extname, join, resolve } from "path"; import { simpleGit, SimpleGit, SimpleGitOptions } from "simple-git"; +// Memory-bounded operation limits (read once at module load, mirroring the +// GIT_PROCESS_TIMEOUT pattern in src/lib/git-server.ts) +const BLAME_MAX_LINES = Math.max( + 1, + parseInt(process.env.BLAME_MAX_LINES || "20000", 10), +); +const DIFF_MAX_BYTES = Math.max( + 1024, + parseInt(process.env.DIFF_MAX_BYTES || "2000000", 10), +); +const SEARCH_MAX_RESULTS = Math.max( + 1, + parseInt(process.env.SEARCH_MAX_RESULTS || "500", 10), +); +const ACTIVITY_MAX_COMMITS = Math.max( + 1, + parseInt(process.env.ACTIVITY_MAX_COMMITS || "50000", 10), +); +const FILE_CONTENT_MAX_BYTES = Math.max( + 1024, + parseInt(process.env.FILE_CONTENT_MAX_BYTES || "1000000", 10), +); + +// Truncate a string to at most maxBytes UTF-8 bytes without splitting a +// multi-byte character (walks back over continuation bytes) +function utf8SafeSlice(input: string, maxBytes: number): string { + if (Buffer.byteLength(input, "utf8") <= maxBytes) { + return input; + } + const buf = Buffer.from(input, "utf8"); + let end = maxBytes; + while (end > 0 && (buf[end] & 0xc0) === 0x80) { + end--; + } + return buf.subarray(0, end).toString("utf8"); +} + export interface RepoInitOptions { defaultBranch?: string; readme?: boolean; @@ -93,6 +130,7 @@ export interface BlameInfo { author: string; email: string; date: Date; + truncated?: boolean; } /** @@ -506,7 +544,12 @@ export async function getFileContent( repoPath: string, filePath: string, ref: string = "HEAD", -): Promise<{ content: string; isBinary: boolean; size: number } | null> { +): Promise<{ + content: string; + isBinary: boolean; + size: number; + truncated?: boolean; +} | null> { const git = getGit(repoPath); try { @@ -516,10 +559,25 @@ export async function getFileContent( // Check if binary const isBinary = await isFileBinary(repoPath, filePath, ref); + const size = Buffer.byteLength(content, "utf8"); + + if (!isBinary && size > FILE_CONTENT_MAX_BYTES) { + logger.warn( + { repoPath, filePath, ref, size, max: FILE_CONTENT_MAX_BYTES }, + "File content exceeds FILE_CONTENT_MAX_BYTES, truncating", + ); + return { + content: utf8SafeSlice(content, FILE_CONTENT_MAX_BYTES), + isBinary, + size, + truncated: true, + }; + } + return { content: isBinary ? "" : content, isBinary, - size: Buffer.byteLength(content, "utf8"), + size, }; } catch (error) { return null; @@ -692,11 +750,13 @@ export async function getCommitActivity( } // Get just the dates of commits + // -n caps the scan so huge histories cannot produce unbounded output const output = await git.raw([ "log", `--since=${since}`, "--date=short", "--format=%ad", + `-n${ACTIVITY_MAX_COMMITS}`, ]); const dates = output.trim().split("\n").filter(Boolean); @@ -721,6 +781,7 @@ export interface SearchResult { file: string; line: number; content: string; + truncated?: boolean; } /** @@ -765,6 +826,18 @@ export async function searchCode( }); } + if (results.length > SEARCH_MAX_RESULTS) { + const truncatedResults = results.slice(0, SEARCH_MAX_RESULTS); + for (const result of truncatedResults) { + result.truncated = true; + } + logger.warn( + { repoPath, query, ref, total: results.length, max: SEARCH_MAX_RESULTS }, + "Search results exceed SEARCH_MAX_RESULTS, truncating", + ); + return truncatedResults; + } + return results; } catch (error) { const msg = error instanceof Error ? error.message : String(error); @@ -1011,6 +1084,13 @@ export async function getCommitPatchDiff( "-M", sha, ]); + if (Buffer.byteLength(output, "utf8") > DIFF_MAX_BYTES) { + logger.warn( + { repoPath, sha, max: DIFF_MAX_BYTES }, + "Commit patch diff exceeds DIFF_MAX_BYTES, truncating", + ); + return utf8SafeSlice(output, DIFF_MAX_BYTES); + } return output; } catch (error) { if (!isExpectedGitError(error)) { @@ -1030,9 +1110,13 @@ export async function getBlame( const git = getGit(repoPath); try { + // -L 1, caps blame output upstream; git clamps the end line to EOF + // so this is safe for files smaller than the cap const output = await git.raw([ "blame", "--line-porcelain", + "-L", + `1,${BLAME_MAX_LINES}`, ref, "--", filePath, @@ -1072,6 +1156,26 @@ export async function getBlame( } as BlameInfo); } + // If we filled the cap, probe one line past it to detect truncation + if (blameLines.length >= BLAME_MAX_LINES) { + try { + await git.raw([ + "blame", + "--line-porcelain", + "-L", + `${BLAME_MAX_LINES + 1},${BLAME_MAX_LINES + 1}`, + ref, + "--", + filePath, + ]); + for (const blameLine of blameLines) { + blameLine.truncated = true; + } + } catch { + // Probe failed: file ends at/below the cap, nothing truncated + } + } + return blameLines; } catch (error) { if (!isExpectedGitError(error)) { @@ -1165,7 +1269,7 @@ export async function compareBranches( repoPath: string, base: string, head: string, -): Promise<{ commits: CommitInfo[]; diffs: DiffInfo[] }> { +): Promise<{ commits: CommitInfo[]; diffs: DiffInfo[]; truncated?: boolean }> { // Verify branches exist first to avoid "ambiguous argument" errors const fullBase = base.startsWith("refs/") ? base : `refs/heads/${base}`; const fullHead = head.startsWith("refs/") ? head : `refs/heads/${head}`; @@ -1203,8 +1307,13 @@ export async function compareBranches( `${fullBase}...${fullHead}`, // Triple dot finds merge base automatically ]); + const numstatTruncated = Buffer.byteLength(output, "utf8") > DIFF_MAX_BYTES; + const numstat = numstatTruncated + ? utf8SafeSlice(output, DIFF_MAX_BYTES) + : output; + const diffs: DiffInfo[] = []; - for (const line of output.trim().split("\n").filter(Boolean)) { + for (const line of numstat.trim().split("\n").filter(Boolean)) { const parts = line.split("\t"); if (parts.length >= 3) { const [additions, deletions, file] = parts; @@ -1221,7 +1330,14 @@ export async function compareBranches( } } - return { commits, diffs }; + if (numstatTruncated) { + logger.warn( + { repoPath, base, head, max: DIFF_MAX_BYTES }, + "Branch comparison numstat exceeds DIFF_MAX_BYTES, truncating", + ); + } + + return { commits, diffs, truncated: numstatTruncated || undefined }; } catch (error: any) { // Fallback for any other errors logger.error({ err: error }, "Error comparing branches"); @@ -1256,6 +1372,13 @@ export async function getComparePatchDiff( "--unified=5", "-M", // Detect renames ]); + if (Buffer.byteLength(output, "utf8") > DIFF_MAX_BYTES) { + logger.warn( + { repoPath, base, head, max: DIFF_MAX_BYTES }, + "Compare patch diff exceeds DIFF_MAX_BYTES, truncating", + ); + return utf8SafeSlice(output, DIFF_MAX_BYTES); + } return output; } catch (error: any) { logger.error({ err: error }, "Error getting compare patch diff"); diff --git a/src/lib/graphql/resolvers.ts b/src/lib/graphql/resolvers.ts index 0f41ba98..eb82410a 100644 --- a/src/lib/graphql/resolvers.ts +++ b/src/lib/graphql/resolvers.ts @@ -20,13 +20,42 @@ export interface GraphQLContext { user?: typeof schema.users.$inferSelect; } -// Helper to create page info -function createPageInfo(nodes: any[], first: number, after?: string) { +// Offset-based cursor helpers (GitHub-style opaque base64 cursors) +const CURSOR_PREFIX = "cursor:"; + +function encodeCursor(offset: number): string { + return Buffer.from(`${CURSOR_PREFIX}${offset}`).toString("base64"); +} + +function decodeCursor(cursor?: string | null): number | null { + if (!cursor) return null; + try { + const decoded = Buffer.from(cursor, "base64").toString("utf8"); + if (!decoded.startsWith(CURSOR_PREFIX)) return null; + const offset = Number.parseInt(decoded.slice(CURSOR_PREFIX.length), 10); + if (!Number.isInteger(offset) || offset < 0) return null; + return offset; + } catch { + return null; + } +} + +// Helper to create page info. Callers fetch first+1 rows and pass the fetched +// row count as `total`, yielding an honest hasNextPage without a COUNT query. +function createPageInfo( + nodes: any[], + first: number, + after?: string | null, + total?: number +) { + const offset = decodeCursor(after) ?? 0; + const hasNextPage = + typeof total === "number" ? offset + nodes.length < total : false; return { - hasNextPage: nodes.length === first, - hasPreviousPage: !!after, - startCursor: nodes[0]?.id || null, - endCursor: nodes[nodes.length - 1]?.id || null, + hasNextPage, + hasPreviousPage: offset > 0, + startCursor: nodes.length > 0 ? encodeCursor(offset) : null, + endCursor: nodes.length > 0 ? encodeCursor(offset + nodes.length) : null, }; } @@ -70,50 +99,66 @@ export const resolvers = { search: async ( _: unknown, - { query, type, first = 10 }: { query: string; type: string; first: number }, + { query, type, first = 10, after }: { + query: string; + type: string; + first: number; + after?: string | null; + }, ctx: GraphQLContext ) => { const searchTerm = `%${query}%`; + const offset = decodeCursor(after) ?? 0; switch (type) { case "REPOSITORY": { const repos = await ctx.db.query.repositories.findMany({ where: like(schema.repositories.name, searchTerm), - limit: first, + limit: first + 1, + offset, }); + const nodes = repos.slice(0, first); return { - nodes: repos, - pageInfo: createPageInfo(repos, first), - totalCount: repos.length, + nodes, + pageInfo: createPageInfo(nodes, first, after, repos.length), + totalCount: nodes.length, }; } case "USER": { const users = await ctx.db.query.users.findMany({ where: like(schema.users.username, searchTerm), - limit: first, + limit: first + 1, + offset, }); + const nodes = users.slice(0, first); return { - nodes: users, - pageInfo: createPageInfo(users, first), - totalCount: users.length, + nodes, + pageInfo: createPageInfo(nodes, first, after, users.length), + totalCount: nodes.length, }; } case "PULL_REQUEST": { const prs = await ctx.db.query.pullRequests.findMany({ where: like(schema.pullRequests.title, searchTerm), - limit: first, + limit: first + 1, + offset, }); + const nodes = prs.slice(0, first); return { - nodes: prs, - pageInfo: createPageInfo(prs, first), - totalCount: prs.length, + nodes, + pageInfo: createPageInfo(nodes, first, after, prs.length), + totalCount: nodes.length, }; } default: - return { nodes: [], pageInfo: createPageInfo([], first), totalCount: 0 }; + return { + nodes: [], + pageInfo: createPageInfo([], first, after), + totalCount: 0, + }; } }, }, @@ -121,18 +166,21 @@ export const resolvers = { User: { repositories: async ( user: typeof schema.users.$inferSelect, - { first = 10 }: { first: number }, + { first = 10, after }: { first: number; after?: string | null }, ctx: GraphQLContext ) => { + const offset = decodeCursor(after) ?? 0; const repos = await ctx.db.query.repositories.findMany({ where: eq(schema.repositories.ownerId, user.id), - limit: first, + limit: first + 1, + offset, orderBy: [desc(schema.repositories.updatedAt)], }); + const nodes = repos.slice(0, first); return { - nodes: repos, - pageInfo: createPageInfo(repos, first), - totalCount: repos.length, + nodes, + pageInfo: createPageInfo(nodes, first, after, repos.length), + totalCount: nodes.length, }; }, @@ -142,18 +190,24 @@ export const resolvers = { ctx: GraphQLContext ) => { const prs = await ctx.db.query.pullRequests.findMany({ - where: eq(schema.pullRequests.authorId, user.id), - limit: first, + where: states + ? and( + eq(schema.pullRequests.authorId, user.id), + inArray( + schema.pullRequests.state, + states.map((s) => s.toLowerCase()) + ) + ) + : eq(schema.pullRequests.authorId, user.id), + limit: first + 1, orderBy: [desc(schema.pullRequests.updatedAt)], }); - const filtered = states - ? prs.filter((pr) => states.includes(pr.state.toUpperCase())) - : prs; + const filtered = prs.slice(0, first); return { nodes: filtered, - pageInfo: createPageInfo(filtered, first), + pageInfo: createPageInfo(filtered, first, null, prs.length), totalCount: filtered.length, }; }, @@ -219,18 +273,24 @@ export const resolvers = { ctx: GraphQLContext ) => { const prs = await ctx.db.query.pullRequests.findMany({ - where: eq(schema.pullRequests.repositoryId, repo.id), - limit: first, + where: states + ? and( + eq(schema.pullRequests.repositoryId, repo.id), + inArray( + schema.pullRequests.state, + states.map((s) => s.toLowerCase()) + ) + ) + : eq(schema.pullRequests.repositoryId, repo.id), + limit: first + 1, orderBy: [desc(schema.pullRequests.updatedAt)], }); - const filtered = states - ? prs.filter((pr) => states.includes(pr.state.toUpperCase())) - : prs; + const filtered = prs.slice(0, first); return { nodes: filtered, - pageInfo: createPageInfo(filtered, first), + pageInfo: createPageInfo(filtered, first, null, prs.length), totalCount: filtered.length, }; }, @@ -242,13 +302,14 @@ export const resolvers = { ) => { const issues = await ctx.db.query.issues.findMany({ where: eq(schema.issues.repositoryId, repo.id), - limit: first, + limit: first + 1, orderBy: [desc(schema.issues.updatedAt)], }); + const nodes = issues.slice(0, first); return { - nodes: issues, - pageInfo: createPageInfo(issues, first), - totalCount: issues.length, + nodes, + pageInfo: createPageInfo(nodes, first, null, issues.length), + totalCount: nodes.length, }; }, @@ -326,12 +387,13 @@ export const resolvers = { ) => { const reviews = await ctx.db.query.pullRequestReviews.findMany({ where: eq(schema.pullRequestReviews.pullRequestId, pr.id), - limit: first, + limit: first + 1, }); + const nodes = reviews.slice(0, first); return { - nodes: reviews, - pageInfo: createPageInfo(reviews, first), - totalCount: reviews.length, + nodes, + pageInfo: createPageInfo(nodes, first, null, reviews.length), + totalCount: nodes.length, }; }, @@ -342,12 +404,13 @@ export const resolvers = { ) => { const comments = await ctx.db.query.pullRequestComments.findMany({ where: eq(schema.pullRequestComments.pullRequestId, pr.id), - limit: first, + limit: first + 1, }); + const nodes = comments.slice(0, first); return { - nodes: comments, - pageInfo: createPageInfo(comments, first), - totalCount: comments.length, + nodes, + pageInfo: createPageInfo(nodes, first, null, comments.length), + totalCount: nodes.length, }; }, diff --git a/src/lib/login-lockout.ts b/src/lib/login-lockout.ts new file mode 100644 index 00000000..989464cb --- /dev/null +++ b/src/lib/login-lockout.ts @@ -0,0 +1,189 @@ +import { logger } from "@/lib/logger"; +import { isDistributed } from "@/lib/rate-limit"; +import { redis } from "@/lib/redis"; + +const USER_KEY_PREFIX = "u:"; +const IP_KEY_PREFIX = "i:"; +const REDIS_KEY_PREFIX = "login-lockout:"; + +const USER_IP_MAX_FAILURES = 5; +const IP_MAX_FAILURES = 20; +const FAILURE_WINDOW_MS = 15 * 60_000; +const BASE_LOCKOUT_MS = 15 * 60_000; +const MAX_LOCKOUT_MS = 24 * 60 * 60_000; +const CLEANUP_INTERVAL_MS = 5 * 60_000; +const MAX_MEMORY_ENTRIES = 10_000; + +export interface LockoutStatus { + locked: boolean; + retryAfterSecs?: number; +} + +interface LockoutEntry { + failures: number; + windowStart: number; + lockedUntil: number; + lockoutCount: number; +} + +interface LockoutStore { + get(identifier: string): Promise; + set(identifier: string, entry: LockoutEntry, ttlMs: number): Promise; + delete(identifier: string): Promise; +} + +class InMemoryLockoutStore implements LockoutStore { + private store: Map = new Map(); + + constructor() { + setInterval(() => this.cleanup(), CLEANUP_INTERVAL_MS).unref(); + } + + private cleanup(): void { + const now = Date.now(); + for (const [identifier, entry] of this.store.entries()) { + if ( + entry.lockedUntil < now && + now - entry.windowStart > FAILURE_WINDOW_MS + ) { + this.store.delete(identifier); + } + } + } + + async get(identifier: string): Promise { + return this.store.get(identifier) ?? null; + } + + async set(identifier: string, entry: LockoutEntry): Promise { + this.store.delete(identifier); + this.store.set(identifier, entry); + while (this.store.size > MAX_MEMORY_ENTRIES) { + const oldest = this.store.keys().next().value; + if (oldest === undefined) break; + this.store.delete(oldest); + } + } + + async delete(identifier: string): Promise { + this.store.delete(identifier); + } +} + +class RedisLockoutStore implements LockoutStore { + constructor(private readonly fallback: InMemoryLockoutStore) {} + + async get(identifier: string): Promise { + try { + const raw = await redis.get(`${REDIS_KEY_PREFIX}${identifier}`); + if (!raw) return null; + return JSON.parse(raw) as LockoutEntry; + } catch (error) { + logger.error({ error, identifier }, "Redis login lockout read failed, falling back to in-memory store"); + return this.fallback.get(identifier); + } + } + + async set(identifier: string, entry: LockoutEntry, ttlMs: number): Promise { + try { + await redis.set( + `${REDIS_KEY_PREFIX}${identifier}`, + JSON.stringify(entry), + "PX", + Math.ceil(ttlMs), + ); + return; + } catch (error) { + logger.error({ error, identifier }, "Redis login lockout write failed, falling back to in-memory store"); + } + await this.fallback.set(identifier, entry); + } + + async delete(identifier: string): Promise { + try { + await redis.del(`${REDIS_KEY_PREFIX}${identifier}`); + } catch (error) { + logger.error({ error, identifier }, "Redis login lockout delete failed"); + } + await this.fallback.delete(identifier); + } +} + +export function normalizeLogin(value: string): string { + return value.trim().toLowerCase(); +} + +export function userIpKey(login: string, ip: string | null | undefined): string { + return `${USER_KEY_PREFIX}${normalizeLogin(login)}|${ip || "unknown"}`; +} + +export function ipKey(ip: string | null | undefined): string { + return `${IP_KEY_PREFIX}${ip || "unknown"}`; +} + +function maxFailuresFor(identifier: string): number { + return identifier.startsWith(IP_KEY_PREFIX) + ? IP_MAX_FAILURES + : USER_IP_MAX_FAILURES; +} + +function ttlFor(entry: LockoutEntry, now: number): number { + const windowRemaining = entry.windowStart + FAILURE_WINDOW_MS - now; + const lockRemaining = entry.lockedUntil - now; + return Math.max(Math.max(windowRemaining, lockRemaining), FAILURE_WINDOW_MS); +} + +const memoryStore = new InMemoryLockoutStore(); +const store: LockoutStore = isDistributed + ? new RedisLockoutStore(memoryStore) + : memoryStore; + +export async function recordLoginFailure(identifier: string): Promise { + const now = Date.now(); + const entry = (await store.get(identifier)) ?? { + failures: 0, + windowStart: now, + lockedUntil: 0, + lockoutCount: 0, + }; + + if (entry.lockedUntil > now) { + return; + } + + if (now - entry.windowStart > FAILURE_WINDOW_MS) { + entry.windowStart = now; + entry.failures = 0; + } + + entry.failures += 1; + + if (entry.failures >= maxFailuresFor(identifier)) { + entry.lockoutCount += 1; + const lockMs = Math.min( + BASE_LOCKOUT_MS * 2 ** (entry.lockoutCount - 1), + MAX_LOCKOUT_MS, + ); + entry.lockedUntil = now + lockMs; + entry.failures = 0; + logger.warn({ identifier }, "Login lockout triggered"); + } + + await store.set(identifier, entry, ttlFor(entry, now)); +} + +export async function clearLoginFailures(identifier: string): Promise { + await store.delete(identifier); +} + +export async function isLockedOut(identifier: string): Promise { + const now = Date.now(); + const entry = await store.get(identifier); + if (!entry || entry.lockedUntil <= now) { + return { locked: false }; + } + return { + locked: true, + retryAfterSecs: Math.ceil((entry.lockedUntil - now) / 1000), + }; +} diff --git a/src/lib/push-mirror.ts b/src/lib/push-mirror.ts new file mode 100644 index 00000000..612833cd --- /dev/null +++ b/src/lib/push-mirror.ts @@ -0,0 +1,455 @@ +/** + * Push Mirror Library + * + * Mirrors repository refs OUT to an external remote — complements the + * pull-only mirror sync in ./mirror-sync.ts. + * + * Storage: config lives on the `repositories` table, mirroring the pull-side + * pattern exactly (mirrorUrl / mirrorToken / lastMirrorSyncAt / mirrorSyncStatus): + * - pushMirrorEnabled master switch + * - pushMirrorUrl destination remote (never contains credentials) + * - pushMirrorToken encrypted auth token (workflow-secret-crypto) + * - pushMirrorStatus pending | pushing | success | failed + * - lastPushMirrorAt timestamp of last completed push attempt + * + * Limitation: ONE push remote per repository. + * + * Security: + * - Tokens are encrypted at rest and decrypted only transiently into the + * remote URL per attempt (same mechanism as pull mirrors); they are never + * persisted in git config or returned by any API. + * - Destination URLs are SSRF-validated with validateGitCloneUrl. Private + * targets are rejected unless PUSH_MIRROR_ALLOW_PRIVATE=true (same opt-in + * pattern as FEDERATION_ALLOW_LOCALHOST). + */ + +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import { and, asc, eq, isNull, lt } from "drizzle-orm"; +import { getDatabase, schema } from "@/db"; +import { repositories } from "@/db/schema/repositories"; +import { logger } from "./logger"; +import { resolveRepoPath } from "./git-storage"; +import { validateGitCloneUrl } from "./url-validator"; +import { encryptWorkflowSecret } from "./workflow-secret-crypto"; +// Shared transient-token URL builder — identical semantics to pull-side fetch auth. +import { buildFetchUrl as buildAuthenticatedRemoteUrl } from "./mirror-sync"; + +export interface ConfigurePushMirrorInput { + url: string; + /** Plaintext token; encrypted before storage. null clears a stored token. Omit to keep existing. */ + authToken?: string | null; +} + +export interface PushMirrorConfig { + enabled: boolean; + url: string | null; + hasToken: boolean; + status: string | null; + lastPushMirrorAt: Date | null; +} + +export interface PushMirrorResult { + success: boolean; + refsUpdated: number; + error?: string; + durationMs?: number; +} + +export interface ProcessDuePushMirrorsOptions { + limit?: number; + minIntervalSeconds?: number; +} + +export interface ProcessDuePushMirrorsResult { + total: number; + eligible: number; + pushed: number; + failed: number; + failedRepoIds: string[]; + durationMs: number; +} + +function getDb(): NodePgDatabase { + return getDatabase() as NodePgDatabase; +} + +/** + * SSRF-validate a push destination URL. Rejects file://, non-git schemes, + * localhost/private networks unless explicitly allowed via + * PUSH_MIRROR_ALLOW_PRIVATE=true. + */ +export async function validatePushMirrorUrl( + url: string +): Promise<{ valid: true } | { valid: false; reason: string }> { + const allowPrivate = process.env.PUSH_MIRROR_ALLOW_PRIVATE === "true"; + return validateGitCloneUrl(url, allowPrivate); +} + +/** Timeout in seconds for a single push attempt. */ +function getPushTimeoutSecs(): number { + const raw = parseInt( + process.env.PUSH_MIRROR_TIMEOUT_SECS || + process.env.GIT_PROCESS_TIMEOUT_SECS || + "300", + 10 + ); + return Number.isFinite(raw) && raw > 0 ? raw : 300; +} + +/** Min interval between automatic pushes of the same repo. */ +function getMinIntervalSecs(override?: number): number { + if (override !== undefined && Number.isFinite(override) && override >= 0) { + return override; + } + const raw = parseInt(process.env.PUSH_MIRROR_MIN_INTERVAL_SECS || "300", 10); + return Number.isFinite(raw) && raw >= 0 ? raw : 300; +} + +/** + * Git error output can echo the authenticated remote URL (with embedded + * token). Redact any userinfo password before storing/logging. + */ +export function redactCredentials(message: string): string { + return message.replace( + /(https?:\/\/[^:@/\s]+):([^@\s/]+)@/gi, + "$1:***@" + ); +} + +async function markStatus(repoId: string, values: Record): Promise { + const db = getDb(); + await db + .update(repositories) + .set({ ...values, updatedAt: new Date() }) + .where(eq(repositories.id, repoId)); +} + +/** + * Enable/update the push mirror configuration for a repository. + */ +export async function configurePushMirror( + repoId: string, + input: ConfigurePushMirrorInput +): Promise<{ success: boolean; config?: PushMirrorConfig; error?: string }> { + const validation = await validatePushMirrorUrl(input.url); + if (!validation.valid) { + return { success: false, error: validation.reason }; + } + + const db = getDb(); + const rows = await db + .select({ id: repositories.id }) + .from(repositories) + .where(eq(repositories.id, repoId)) + .limit(1); + if (rows.length === 0) { + return { success: false, error: "Repository not found" }; + } + + try { + await markStatus(repoId, { + pushMirrorEnabled: true, + pushMirrorUrl: input.url, + // Undefined = keep existing stored token; explicit value encrypts/replaces; null clears. + ...(input.authToken === undefined + ? {} + : { + pushMirrorToken: + input.authToken === null || input.authToken === "" + ? null + : encryptWorkflowSecret(input.authToken), + }), + pushMirrorStatus: "pending", + }); + } catch (error) { + const message = error instanceof Error ? error.message : "Unknown error"; + logger.error({ repoId, error: message }, "Failed to configure push mirror"); + return { success: false, error: message }; + } + + const config = await getPushMirror(repoId); + return { success: true, config: config ?? undefined }; +} + +/** + * Remove the push mirror configuration for a repository. + */ +export async function removePushMirror( + repoId: string +): Promise<{ success: boolean; error?: string }> { + const db = getDb(); + try { + const rows = await db + .select({ id: repositories.id }) + .from(repositories) + .where(eq(repositories.id, repoId)) + .limit(1); + if (rows.length === 0) { + return { success: false, error: "Repository not found" }; + } + + await markStatus(repoId, { + pushMirrorEnabled: false, + pushMirrorUrl: null, + pushMirrorToken: null, + pushMirrorStatus: null, + }); + return { success: true }; + } catch (error) { + const message = error instanceof Error ? error.message : "Unknown error"; + logger.error({ repoId, error: message }, "Failed to remove push mirror"); + return { success: false, error: message }; + } +} + +/** + * Read the push mirror configuration. Never exposes the token itself. + */ +export async function getPushMirror(repoId: string): Promise { + const db = getDb(); + const rows = await db + .select({ + enabled: repositories.pushMirrorEnabled, + url: repositories.pushMirrorUrl, + token: repositories.pushMirrorToken, + status: repositories.pushMirrorStatus, + lastPushMirrorAt: repositories.lastPushMirrorAt, + }) + .from(repositories) + .where(eq(repositories.id, repoId)) + .limit(1); + + const row = rows[0]; + if (!row) return null; + + return { + enabled: row.enabled, + url: row.url, + hasToken: row.token !== null && row.token !== undefined, + status: row.status, + lastPushMirrorAt: row.lastPushMirrorAt, + }; +} + +/** + * Push all branches and tags to the configured external remote right now. + * + * Uses an explicit forced refspec list (+refs/heads/* and +refs/tags/*) so we + * never push hidden refs (refs/pull/*, notes, etc.) unintentionally — unlike + * a blind `git push --mirror`. Credentials are injected into the URL per + * attempt and never persisted. Never throws. + */ +export async function pushMirrorNow(repositoryId: string): Promise { + const startedAt = Date.now(); + + let repo: typeof repositories.$inferSelect | undefined; + try { + const db = getDb(); + const rows = await db + .select() + .from(repositories) + .where(eq(repositories.id, repositoryId)) + .limit(1); + repo = rows[0]; + } catch (error) { + const message = error instanceof Error ? error.message : "Unknown error"; + logger.error({ repoId: repositoryId, error: message }, "Failed to load repository for push mirror"); + return { success: false, refsUpdated: 0, error: message }; + } + + if (!repo) { + return { success: false, refsUpdated: 0, error: "Repository not found" }; + } + + if (!repo.pushMirrorEnabled || !repo.pushMirrorUrl) { + return { success: false, refsUpdated: 0, error: "Push mirror not configured" }; + } + + const timeoutSecs = getPushTimeoutSecs(); + + try { + const repoPath = await resolveRepoPath(repo.diskPath); + + // Mark as pushing + try { + await markStatus(repositoryId, { pushMirrorStatus: "pushing" }); + } catch (statusError) { + logger.warn( + { repoId: repositoryId, error: statusError instanceof Error ? statusError.message : "unknown" }, + "Failed to mark push mirror status as pushing" + ); + } + + // Lazy-load simple-git (via the sanitized-env wrapper) so this module + // stays cheap to import in workers/cron contexts. + const { createSimpleGit } = await import("./git"); + const git = createSimpleGit({ + baseDir: repoPath, + // simple-git kills the underlying git process when the block + // timeout elapses — our process kill guarantee. + timeout: { block: timeoutSecs * 1000 }, + }); + + // Transient credential injection — decrypted per attempt, never persisted. + const pushUrl = buildAuthenticatedRemoteUrl(repo.pushMirrorUrl, repo.pushMirrorToken); + + const pushArgs = [ + "push", + "--prune", + pushUrl, + "+refs/heads/*:refs/heads/*", + "+refs/tags/*:refs/tags/*", + ]; + + logger.info({ repoId: repositoryId }, "Starting push mirror"); + + let timer: ReturnType | undefined; + const output = await Promise.race([ + git.raw(pushArgs), + new Promise((_, reject) => { + timer = setTimeout( + () => reject(new Error(`Push mirror timed out after ${timeoutSecs}s`)), + timeoutSecs * 1000 + ); + }), + ]).finally(() => clearTimeout(timer)); + + const refsUpdated = String(output) + .split("\n") + .filter((line) => line.includes("->")) + .length; + + await markStatus(repositoryId, { + pushMirrorStatus: "success", + lastPushMirrorAt: new Date(), + }); + + const durationMs = Date.now() - startedAt; + logger.info({ repoId: repositoryId, refsUpdated, durationMs }, "Push mirror completed"); + + return { success: true, refsUpdated, durationMs }; + } catch (error) { + const errorMessage = redactCredentials( + error instanceof Error ? error.message : "Unknown error" + ); + + logger.error({ repoId: repositoryId, error: errorMessage }, "Push mirror failed"); + + try { + await markStatus(repositoryId, { pushMirrorStatus: "failed" }); + } catch (statusError) { + logger.error( + { repoId: repositoryId, error: statusError instanceof Error ? statusError.message : "unknown" }, + "Failed to record push mirror failure status" + ); + } + + return { success: false, refsUpdated: 0, error: errorMessage, durationMs: Date.now() - startedAt }; + } +} + +/** + * Find and process repos whose push mirror is due. + * + * Due means: push mirroring enabled AND either never pushed or last push older + * than the min interval (PUSH_MIRROR_MIN_INTERVAL_SECS, default 300). Oldest + * first. Repos are processed sequentially with per-repo isolation — one + * failure never aborts the batch and this function never throws. + */ +export async function processDuePushMirrors( + options: ProcessDuePushMirrorsOptions = {} +): Promise { + const startedAt = Date.now(); + const limit = options.limit && options.limit > 0 ? Math.floor(options.limit) : 10; + const minIntervalSecs = getMinIntervalSecs(options.minIntervalSeconds); + const cutoff = new Date(Date.now() - minIntervalSecs * 1000); + + const empty: ProcessDuePushMirrorsResult = { + total: 0, + eligible: 0, + pushed: 0, + failed: 0, + failedRepoIds: [], + durationMs: 0, + }; + + let dueRepoIds: string[] = []; + try { + const db = getDb(); + const enabledAndNeverPushed = and( + eq(repositories.pushMirrorEnabled, true), + isNull(repositories.lastPushMirrorAt) + ); + + // Never-pushed repos first (oldest created), then stale ones oldest-push-first. + const neverPushed = await db + .select({ id: repositories.id }) + .from(repositories) + .where(enabledAndNeverPushed) + .orderBy(asc(repositories.createdAt)) + .limit(limit); + + dueRepoIds = neverPushed.map((row) => row.id); + + if (dueRepoIds.length < limit) { + const stale = await db + .select({ id: repositories.id }) + .from(repositories) + .where( + and( + eq(repositories.pushMirrorEnabled, true), + lt(repositories.lastPushMirrorAt, cutoff) + ) + ) + .orderBy(asc(repositories.lastPushMirrorAt)) + .limit(limit - dueRepoIds.length); + + dueRepoIds = dueRepoIds.concat(stale.map((row) => row.id)); + } + } catch (error) { + const message = error instanceof Error ? error.message : "Unknown error"; + logger.error({ error: message }, "Failed to select due push mirrors"); + return { ...empty, durationMs: Date.now() - startedAt }; + } + + let pushed = 0; + let failed = 0; + const failedRepoIds: string[] = []; + + for (const repoId of dueRepoIds) { + try { + const result = await pushMirrorNow(repoId); + if (result.success) { + pushed++; + } else { + failed++; + failedRepoIds.push(repoId); + } + } catch (error) { + // Defensive: pushMirrorNow should not throw, but isolate anyway. + failed++; + failedRepoIds.push(repoId); + logger.error( + { repoId, error: error instanceof Error ? error.message : "Unknown error" }, + "Unexpected error processing push mirror" + ); + } + } + + const durationMs = Date.now() - startedAt; + if (dueRepoIds.length > 0) { + logger.info( + { total: dueRepoIds.length, pushed, failed, durationMs }, + "Push mirror batch completed" + ); + } + + return { + total: dueRepoIds.length, + eligible: dueRepoIds.length, + pushed, + failed, + failedRepoIds, + durationMs, + }; +} diff --git a/src/lib/realtime.ts b/src/lib/realtime.ts index 9b007d65..b3e21e6d 100644 --- a/src/lib/realtime.ts +++ b/src/lib/realtime.ts @@ -5,8 +5,10 @@ import { eq } from "drizzle-orm"; import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import type Redis from "ioredis"; import { getDatabase, schema } from "@/db"; import { logger } from "@/lib/logger"; +import { redis } from "@/lib/redis"; // Event types for real-time updates export type RealtimeEventType = @@ -43,6 +45,187 @@ interface Connection { const connections = new Map(); +// === Cross-process fanout (Redis Pub/Sub bridge) === +// +// SSE connections live in this process's memory only. Background processes +// (worker, runner) run in separate OS processes, so their events must be +// relayed over Redis pub/sub to reach browsers connected to the web process. +// +// - Channel: `och:realtime` +// - Envelope: JSON `{ originId, target, event }` — receivers drop messages +// whose originId matches their own (prevents double-delivery). +// - Degradation: if Redis is unavailable, delivery silently falls back to +// local-only with a single warn log until the connection recovers. + +const REALTIME_CHANNEL = "och:realtime"; +const ORIGIN_ID = crypto.randomUUID(); + +type RealtimeBridgeTarget = + | { kind: "all" } + | { kind: "user"; userId: string } + | { kind: "repository"; repositoryId: string } + | { kind: "connection"; connectionId: string }; + +interface RealtimeBridgeMessage { + originId: string; + target: RealtimeBridgeTarget; + event: RealtimeEvent; +} + +interface BridgeConnections { + subscriber: Redis; + publisher: Redis; +} + +let bridge: BridgeConnections | null = null; +let bridgeInitPromise: Promise | null = null; +let bridgeDegradedLogged = false; + +function isRedisBridgeEnabled(): boolean { + return !( + process.env.SKIP_REDIS_CHECK === "1" || process.env.NODE_ENV === "test" + ); +} + +function logBridgeDegraded(scope: string, error: unknown): void { + if (bridgeDegradedLogged) return; + bridgeDegradedLogged = true; + logger.warn( + "Realtime Redis bridge degraded; falling back to local-only delivery", + { scope, error } + ); +} + +function logBridgeRecovered(scope: string): void { + if (!bridgeDegradedLogged) return; + bridgeDegradedLogged = false; + logger.info("Realtime Redis bridge recovered", { scope }); +} + +/** + * Lazily create dedicated publisher/subscriber connections and subscribe. + * Pub/sub requires one connection per role, so both are duplicated from the + * shared client factory. Never throws — on failure the bridge stays disabled + * and delivery remains local-only. + */ +async function ensureBridge(): Promise { + if (bridge || !isRedisBridgeEnabled()) return; + if (bridgeInitPromise) return bridgeInitPromise; + + bridgeInitPromise = (async () => { + try { + const subscriber = redis.duplicate(); + const publisher = redis.duplicate(); + + // duplicated instances do not inherit listeners; without these an + // 'error' event would crash the process + subscriber.on("error", (error: Error) => { + logBridgeDegraded("subscriber", error); + }); + publisher.on("error", (error: Error) => { + logBridgeDegraded("publisher", error); + }); + + subscriber.on("message", (channel: string, raw: string) => { + if (channel !== REALTIME_CHANNEL) return; + handleBridgeMessage(raw); + }); + + await subscriber.subscribe(REALTIME_CHANNEL); + logBridgeRecovered("subscriber"); + + bridge = { subscriber, publisher }; + } catch (error) { + logBridgeDegraded("init", error); + } finally { + bridgeInitPromise = null; + } + })(); + + return bridgeInitPromise; +} + +function handleBridgeMessage(raw: string): void { + let message: RealtimeBridgeMessage; + try { + message = JSON.parse(raw) as RealtimeBridgeMessage; + } catch { + logger.warn("Dropping malformed realtime bridge message"); + return; + } + + if (!message || message.originId === ORIGIN_ID) return; + + try { + // timestamp round-trips as an ISO string through JSON; revive it so + // local consumers still receive a Date + const event: RealtimeEvent = { + ...message.event, + timestamp: new Date(message.event.timestamp), + }; + deliverLocally(message.target, event); + } catch (error) { + logger.error("Failed to deliver bridged realtime event", { error }); + } +} + +function deliverLocally(target: RealtimeBridgeTarget, event: RealtimeEvent): number { + switch (target.kind) { + case "user": + return sendToUserLocal(target.userId, event); + case "repository": + return broadcastToRepositoryLocal(target.repositoryId, event); + case "connection": + return sendToConnection(target.connectionId, event) ? 1 : 0; + case "all": + default: + return broadcastToAllLocal(event); + } +} + +/** + * Fire-and-forget publish of an event to other processes via Redis. + * Never throws and never affects local delivery; returns whether the + * message was handed to Redis. + */ +async function publishToBridge( + target: RealtimeBridgeTarget, + event: RealtimeEvent +): Promise { + try { + await ensureBridge(); + if (!bridge) return false; + + const payload: RealtimeBridgeMessage = { originId: ORIGIN_ID, target, event }; + const receivers = await bridge.publisher.publish( + REALTIME_CHANNEL, + JSON.stringify(payload) + ); + logBridgeRecovered("publisher"); + logger.debug("Realtime event published cross-process", { + type: event.type, + targetKind: target.kind, + receivers, + }); + return true; + } catch (error) { + logBridgeDegraded("publish", error); + return false; + } +} + +/** + * Publish a realtime event from a process that holds no SSE connections + * itself (e.g. the background worker or CI runner). The web process(s) + * subscribed to `och:realtime` will fan the event out to connected browsers. + */ +export async function publishRealtimeEvent( + target: RealtimeBridgeTarget, + event: RealtimeEvent +): Promise { + return publishToBridge(target, event); +} + /** * Create a unique connection ID */ @@ -58,6 +241,10 @@ export function registerConnection( controller: ReadableStreamDefaultController, repositories: string[] = [] ): string { + // Web processes are primarily receivers of cross-process events; make sure + // the pub/sub subscription exists as soon as the first browser connects + void ensureBridge(); + const connectionId = generateConnectionId(); connections.set(connectionId, { @@ -116,9 +303,9 @@ export function sendToConnection(connectionId: string, event: RealtimeEvent): bo } /** - * Broadcast an event to a specific user's connections + * Local-only delivery to a specific user's connections */ -export function sendToUser(userId: string, event: RealtimeEvent): number { +function sendToUserLocal(userId: string, event: RealtimeEvent): number { let sent = 0; for (const [connectionId, connection] of connections.entries()) { @@ -133,9 +320,19 @@ export function sendToUser(userId: string, event: RealtimeEvent): number { } /** - * Broadcast an event to all users subscribed to a repository + * Broadcast an event to a specific user's connections locally and to + * subscribers in all other processes via Redis */ -export function broadcastToRepository(repositoryId: string, event: RealtimeEvent): number { +export function sendToUser(userId: string, event: RealtimeEvent): number { + const sent = sendToUserLocal(userId, event); + void publishToBridge({ kind: "user", userId }, event); + return sent; +} + +/** + * Local-only delivery to all users subscribed to a repository + */ +function broadcastToRepositoryLocal(repositoryId: string, event: RealtimeEvent): number { let sent = 0; for (const [connectionId, connection] of connections.entries()) { @@ -150,9 +347,19 @@ export function broadcastToRepository(repositoryId: string, event: RealtimeEvent } /** - * Broadcast an event to all connected users + * Broadcast an event to all users subscribed to a repository locally and to + * subscribers in all other processes via Redis */ -export function broadcastToAll(event: RealtimeEvent): number { +export function broadcastToRepository(repositoryId: string, event: RealtimeEvent): number { + const sent = broadcastToRepositoryLocal(repositoryId, event); + void publishToBridge({ kind: "repository", repositoryId }, event); + return sent; +} + +/** + * Local-only delivery to all connected users + */ +function broadcastToAllLocal(event: RealtimeEvent): number { let sent = 0; for (const connectionId of connections.keys()) { @@ -164,6 +371,16 @@ export function broadcastToAll(event: RealtimeEvent): number { return sent; } +/** + * Broadcast an event to all connected users locally and to subscribers in + * all other processes via Redis + */ +export function broadcastToAll(event: RealtimeEvent): number { + const sent = broadcastToAllLocal(event); + void publishToBridge({ kind: "all" }, event); + return sent; +} + /** * Get connection statistics */ diff --git a/src/lib/webhooks.ts b/src/lib/webhooks.ts index 48d1cea9..a9eba2b4 100644 --- a/src/lib/webhooks.ts +++ b/src/lib/webhooks.ts @@ -1,11 +1,16 @@ /** - * Webhook Dispatch Service - * Handles triggering and delivering webhooks + * Webhook Dispatch Service — DB-backed delivery queue + * + * triggerWebhooks() only ENQUEUES one webhook_deliveries row per matching + * hook (status 'pending'); the background worker drains the queue via + * processWebhookQueue(), which claims rows atomically (FOR UPDATE SKIP + * LOCKED), delivers them reusing the shared HMAC/SSRF/timeout paths, and + * reschedules failures with exponential backoff until they succeed or die. */ import { getDatabase, schema } from "@/db"; import crypto from "crypto"; -import { and, eq, sql } from "drizzle-orm"; +import { and, eq, lt, sql } from "drizzle-orm"; import type { NodePgDatabase } from "drizzle-orm/node-postgres"; import { logger } from "./logger"; import { validateWebhookUrl } from "./url-validator"; @@ -15,14 +20,29 @@ interface WebhookPayload { [key: string]: any; } +type WebhookRow = typeof schema.webhooks.$inferSelect; + +/** Total delivery attempts per queued row = WEBHOOK_MAX_RETRIES retries + initial attempt. */ +function maxAttempts(): number { + return Math.max(1, parseInt(process.env.WEBHOOK_MAX_RETRIES || "4", 10)) + 1; +} + +/** Delay before the next attempt after `completedAttempts` failed tries: 1s→16s cap. */ +function backoffDelay(completedAttempts: number): number { + return Math.min(1000 * 2 ** (completedAttempts - 1), 16_000); +} + /** - * Trigger webhooks for a specific repository and event + * Trigger webhooks for a specific repository and event. + * Enqueues one pending delivery per matching hook and returns the count. + * Actual HTTP delivery happens asynchronously in the background worker + * (scripts/worker.ts → processWebhookQueue). */ export async function triggerWebhooks( repositoryId: string, event: string, payload: WebhookPayload, -): Promise { +): Promise { const db = getDatabase() as NodePgDatabase; // Find active webhooks for this repo @@ -50,105 +70,331 @@ export async function triggerWebhooks( return events.includes(event) || events.includes("*"); }); - if (matchingWebhooks.length === 0) return; + if (matchingWebhooks.length === 0) return 0; + + await db.insert(schema.webhookDeliveries).values( + matchingWebhooks.map((hook) => ({ + id: generateId(), + webhookId: hook.id, + event, + payload: JSON.stringify(payload), + status: "pending", + attempts: 0, + nextAttemptAt: new Date(), + })), + ); logger.info( - { repositoryId, event, webhooks: matchingWebhooks.length }, - "Triggering webhooks", + { repositoryId, event, enqueued: matchingWebhooks.length }, + "Enqueued webhooks for delivery", ); - const results = await Promise.allSettled( - matchingWebhooks.map((hook) => dispatchWebhook(hook, event, payload)), + return matchingWebhooks.length; +} + +// ── Queue processing ───────────────────────────────────────────────────────── + +export interface WebhookQueueResult { + swept: number; + claimed: number; + delivered: number; + retried: number; + dead: number; +} + +/** + * Drain the webhook delivery queue. + * 1. Re-queue deliveries stuck in 'processing' beyond STALE_WEBHOOK_LOCK_SECS. + * 2. Atomically claim up to `limit` due pending rows (multi-instance safe). + * 3. Deliver each claimed row sequentially. + */ +export async function processWebhookQueue( + limit = 20, +): Promise { + const swept = await reclaimStaleLocks(); + const ids = await claimPendingDeliveries(limit); + + const result: WebhookQueueResult = { + swept, + claimed: ids.length, + delivered: 0, + retried: 0, + dead: 0, + }; + + for (const id of ids) { + try { + const outcome = await deliverWebhookDelivery(id); + if (outcome === "delivered") result.delivered++; + else if (outcome === "retrying") result.retried++; + else if (outcome === "dead") result.dead++; + } catch (error) { + // Leave the row as 'processing' — the stale-lock sweep will re-queue it. + logger.error( + { err: error, deliveryId: id }, + "Webhook delivery crashed — row left for stale-lock reclaim", + ); + } + } + + return result; +} + +/** + * Claim up to `limit` due pending deliveries by flipping them to 'processing' + * in a single atomic statement. FOR UPDATE SKIP LOCKED keeps concurrent + * workers from claiming the same row. + */ +async function claimPendingDeliveries(limit: number): Promise { + const db = getDatabase() as NodePgDatabase; + const claimed = await db.execute(sql` + UPDATE webhook_deliveries + SET status = 'processing', locked_at = now() + WHERE id IN ( + SELECT id FROM webhook_deliveries + WHERE status = 'pending' AND next_attempt_at <= now() + ORDER BY next_attempt_at + LIMIT ${limit} + FOR UPDATE SKIP LOCKED + ) + RETURNING id + `); + return (claimed.rows ?? []).map((row) => String(row.id)); +} + +/** + * Re-queue deliveries whose worker died mid-flight ('processing' with a lock + * older than STALE_WEBHOOK_LOCK_SECS, default 300s). Returns the count. + */ +export async function reclaimStaleLocks(): Promise { + const db = getDatabase() as NodePgDatabase; + const staleSecs = Math.max( + 1, + parseInt(process.env.STALE_WEBHOOK_LOCK_SECS || "300", 10), ); + const cutoff = new Date(Date.now() - staleSecs * 1000); + const reclaimed = await db + .update(schema.webhookDeliveries) + .set({ status: "pending", lockedAt: null }) + .where( + and( + eq(schema.webhookDeliveries.status, "processing"), + lt(schema.webhookDeliveries.lockedAt, cutoff), + ), + ) + .returning({ id: schema.webhookDeliveries.id }); - const failures = results.filter( - (result) => result.status === "rejected", - ).length; - if (failures > 0) { + if (reclaimed.length > 0) { logger.warn( - { repositoryId, event, failures }, - "Some webhook dispatches failed", + { count: reclaimed.length }, + "Reclaimed stale webhook deliveries stuck in processing", ); } + return reclaimed.length; } +export type DeliveryOutcome = + | "delivered" + | "retrying" + | "dead" + | "skipped"; + /** - * Dispatch a single webhook with retry + exponential backoff. - * Non-2xx responses and network errors are retried up to - * WEBHOOK_MAX_RETRIES times (default 4) with 1s→16s backoff. - * One delivery row is logged per attempt. + * Perform exactly one delivery attempt for a queued delivery row, then + * transition it: delivered / back to pending (with backoff) / dead. */ -async function dispatchWebhook( - webhook: typeof schema.webhooks.$inferSelect, - event: string, - payload: WebhookPayload, -): Promise { +export async function deliverWebhookDelivery( + deliveryId: string, +): Promise { const db = getDatabase() as NodePgDatabase; - const maxRetries = Math.max( - 0, - parseInt(process.env.WEBHOOK_MAX_RETRIES || "4", 10), - ); - let lastError: unknown = null; - let lastResponse: Response | null = null; + const delivery = await db.query.webhookDeliveries.findFirst({ + where: eq(schema.webhookDeliveries.id, deliveryId), + }); + if (!delivery) return "skipped"; + if (delivery.status === "delivered" || delivery.status === "dead") { + return "skipped"; + } - for (let attempt = 0; attempt <= maxRetries; attempt++) { - if (attempt > 0) { - const backoffMs = Math.min(1000 * 2 ** (attempt - 1), 16_000); - logger.info( - { webhookId: webhook.id, attempt, backoffMs }, - "Retrying webhook delivery", - ); - await new Promise((r) => setTimeout(r, backoffMs)); - } + const webhook = await db.query.webhooks.findFirst({ + where: eq(schema.webhooks.id, delivery.webhookId), + }); + if (!webhook) { + await markDead(db, deliveryId, attemptNumber(delivery), "Webhook no longer exists"); + return "dead"; + } - const outcome = await deliverOnce( - webhook, - event, - payload, - attempt + 1, - maxRetries + 1, + let payload: WebhookPayload; + try { + payload = JSON.parse(delivery.payload); + } catch (error: any) { + await markDead( + db, + deliveryId, + attemptNumber(delivery), + `Stored payload is not valid JSON: ${error?.message ?? error}`, ); - lastError = outcome.error ?? null; - lastResponse = outcome.response ?? null; - - // Success (2xx) — stop retrying - if (outcome.success) return; - // 4xx from the receiver: retrying won't help (bad payload/secret) — stop - if ( - lastResponse && - lastResponse.status >= 400 && - lastResponse.status < 500 - ) { - logger.warn( - { webhookId: webhook.id, status: lastResponse.status }, - "Webhook rejected with 4xx, not retrying", - ); - return; - } + return "dead"; } - if (!lastError && lastResponse) { - lastError = new Error(`Webhook failed with HTTP ${lastResponse.status}`); + const attempt = attemptNumber(delivery); + const total = maxAttempts(); + const outcome = await attemptDelivery( + webhook, + delivery.event, + payload, + delivery.id, + attempt, + total, + ); + + if (outcome.success) { + await db + .update(schema.webhookDeliveries) + .set({ + status: "delivered", + attempts: attempt, + responseCode: outcome.responseCode ?? null, + responseBody: outcome.responseBody ?? null, + durationMs: outcome.durationMs ?? null, + requestHeaders: outcome.requestHeaders + ? JSON.stringify(outcome.requestHeaders) + : null, + responseHeaders: outcome.responseHeaders + ? JSON.stringify(outcome.responseHeaders) + : null, + error: null, + failureReason: null, + lockedAt: null, + }) + .where(eq(schema.webhookDeliveries.id, deliveryId)); + return "delivered"; + } + + const errorMessage = outcome.error ?? "Unknown delivery error"; + + // 4xx from the receiver: retrying won't help (bad payload/secret) — die now. + if (!outcome.nonRetryable && attempt < total) { + const delayMs = backoffDelay(attempt); + await db + .update(schema.webhookDeliveries) + .set({ + status: "pending", + attempts: attempt, + nextAttemptAt: new Date(Date.now() + delayMs), + responseCode: outcome.responseCode ?? null, + responseBody: outcome.responseBody ?? null, + durationMs: outcome.durationMs ?? null, + requestHeaders: outcome.requestHeaders + ? JSON.stringify(outcome.requestHeaders) + : null, + error: errorMessage, + failureReason: null, + lockedAt: null, + }) + .where(eq(schema.webhookDeliveries.id, deliveryId)); + logger.info( + { + webhookId: webhook.id, + deliveryId, + attempt, + total, + delayMs, + }, + "Webhook delivery failed — scheduled retry", + ); + return "retrying"; } + + const reason = outcome.nonRetryable + ? `Not retryable: ${errorMessage}` + : `Failed after ${attempt}/${total} attempts`; + await markDeadWithOutcome(db, deliveryId, attempt, reason, outcome); logger.error( - { webhookId: webhook.id, error: lastError }, - "Webhook delivery failed after retries", + { webhookId: webhook.id, deliveryId, attempt, total, error: errorMessage }, + "Webhook delivery dead-lettered", ); + return "dead"; +} + +function attemptNumber( + delivery: typeof schema.webhookDeliveries.$inferSelect, +): number { + return (delivery.attempts ?? 0) + 1; +} + +async function markDead( + db: NodePgDatabase, + deliveryId: string, + attempt: number, + reason: string, +): Promise { + await db + .update(schema.webhookDeliveries) + .set({ + status: "dead", + attempts: attempt, + failureReason: reason, + error: reason, + lockedAt: null, + }) + .where(eq(schema.webhookDeliveries.id, deliveryId)); +} + +async function markDeadWithOutcome( + db: NodePgDatabase, + deliveryId: string, + attempt: number, + reason: string, + outcome: AttemptOutcome, +): Promise { + await db + .update(schema.webhookDeliveries) + .set({ + status: "dead", + attempts: attempt, + failureReason: reason, + error: outcome.error ?? reason, + responseCode: outcome.responseCode ?? null, + responseBody: outcome.responseBody ?? null, + durationMs: outcome.durationMs ?? null, + requestHeaders: outcome.requestHeaders + ? JSON.stringify(outcome.requestHeaders) + : null, + responseHeaders: outcome.responseHeaders + ? JSON.stringify(outcome.responseHeaders) + : null, + lockedAt: null, + }) + .where(eq(schema.webhookDeliveries.id, deliveryId)); +} + +// ── Single-attempt HTTP delivery ────────────────────────────────────────────── + +interface AttemptOutcome { + success: boolean; + responseCode?: number; + responseBody?: string; + responseHeaders?: Record; + requestHeaders?: Record; + durationMs?: number; + error?: string; + /** Receiver returned 4xx — retrying cannot succeed. */ + nonRetryable?: boolean; } /** - * Single delivery attempt. Returns { success } and logs the delivery row. + * One HTTP delivery attempt. Persists nothing except the atomic webhook stat + * bump; the caller owns the delivery-row state transitions. */ -async function deliverOnce( - webhook: typeof schema.webhooks.$inferSelect, +async function attemptDelivery( + webhook: WebhookRow, event: string, payload: WebhookPayload, + deliveryId: string, attempt: number, totalAttempts: number, -): Promise<{ success: boolean; error?: unknown; response?: Response }> { - const db = getDatabase() as NodePgDatabase; - const deliveryId = generateId(); +): Promise { const startTime = Date.now(); try { @@ -207,69 +453,49 @@ async function deliverOnce( clearTimeout(timeout); } - const durationMs = Date.now() - startTime; - const responseBody = await response.text(); const ok = response.ok; + const responseBody = (await response.text()).slice(0, 1000); // Truncate - // Log delivery - await db.insert(schema.webhookDeliveries).values({ - id: deliveryId, - webhookId: webhook.id, - event, - payload: JSON.stringify(payload), - status: ok ? "success" : "failure", + // Update webhook stats (atomic increment to prevent race conditions) + await bumpWebhookStats(webhook.id, ok); + + return { + success: ok, responseCode: response.status, - responseBody: responseBody.slice(0, 1000), // Truncate - durationMs, + responseBody, + responseHeaders: Object.fromEntries(response.headers.entries()), + requestHeaders: headers, + durationMs: Date.now() - startTime, error: ok - ? null + ? undefined : `attempt ${attempt}/${totalAttempts} HTTP ${response.status}`, - requestHeaders: JSON.stringify(headers), - responseHeaders: JSON.stringify( - Object.fromEntries(response.headers.entries()), - ), - }); - - // Update webhook stats (atomic increment to prevent race conditions) - await db - .update(schema.webhooks) - .set({ - deliveryCount: sql`COALESCE(${schema.webhooks.deliveryCount}, 0) + 1`, - lastDeliveryStatus: ok ? "success" : "failure", - lastDeliveryAt: new Date(), - }) - .where(eq(schema.webhooks.id, webhook.id)); - - return { success: ok, response, error: ok ? undefined : new Error(`HTTP ${response.status}`) }; + nonRetryable: !ok && response.status >= 400 && response.status < 500, + }; } catch (error: any) { - const durationMs = Date.now() - startTime; - - // Log failure - await db.insert(schema.webhookDeliveries).values({ - id: deliveryId, - webhookId: webhook.id, - event, - payload: JSON.stringify(payload), - status: "failure", - responseCode: 0, - error: `attempt ${attempt}/${totalAttempts}: ${error.message}`, - durationMs, - }); - // Update webhook stats (atomic increment) - await db - .update(schema.webhooks) - .set({ - deliveryCount: sql`COALESCE(${schema.webhooks.deliveryCount}, 0) + 1`, - lastDeliveryStatus: "failure", - lastDeliveryAt: new Date(), - }) - .where(eq(schema.webhooks.id, webhook.id)); + await bumpWebhookStats(webhook.id, false); - return { success: false, error }; + return { + success: false, + durationMs: Date.now() - startTime, + error: `attempt ${attempt}/${totalAttempts}: ${error?.message ?? error}`, + }; } } +/** Atomic per-attempt stat bump (unchanged semantics from inline dispatch). */ +async function bumpWebhookStats(webhookId: string, ok: boolean): Promise { + const db = getDatabase() as NodePgDatabase; + await db + .update(schema.webhooks) + .set({ + deliveryCount: sql`COALESCE(${schema.webhooks.deliveryCount}, 0) + 1`, + lastDeliveryStatus: ok ? "success" : "failure", + lastDeliveryAt: new Date(), + }) + .where(eq(schema.webhooks.id, webhookId)); +} + /** * Sign payload with secret using HMAC-SHA256 */ diff --git a/src/middleware.ts b/src/middleware.ts index 597a1b95..5785134a 100644 --- a/src/middleware.ts +++ b/src/middleware.ts @@ -26,6 +26,7 @@ const CSRF_EXEMPT_PREFIXES = [ "/api/git/", // Uses Git protocol auth "/api/auth/csrf-token", // The CSRF token endpoint itself "/api/setup", // Setup wizard has no active sessions to protect + "/api/packages/", // Registry push/pull clients (npm, docker) authenticate via PAT Basic/Bearer ]; export const onRequest = defineMiddleware(async (context, next) => { @@ -71,10 +72,17 @@ async function onRequestInner( try { const tokenPayload = await getUserFromRequest(request); if (tokenPayload?.userId) { + // Short-TTL cache absorbs the 2-queries-per-request cost of user+session + // lookups; revocation latency is bounded by AUTH_CACHE_TTL_MS (15s default) + const { cachedUserLookup, cachedSessionLookup } = await import( + "@/lib/auth-cache" + ); const db = getDatabase(); - const user = await db.query.users?.findFirst({ - where: eq(schema.users.id, tokenPayload.userId), - }); + const user = await cachedUserLookup(tokenPayload.userId, () => + db.query.users?.findFirst({ + where: eq(schema.users.id, tokenPayload.userId), + }) ?? Promise.resolve(undefined), + ); if (user) { // Preserve fine-grained PAT scopes on locals.user so permission // checks (`canWriteRepo`, etc.) can enforce them. @@ -82,10 +90,13 @@ async function onRequestInner( context.locals.user = user; } // Populate session for logout and other session-aware handlers - if (tokenPayload.sessionId) { - const session = await db.query.sessions?.findFirst({ - where: eq(schema.sessions.id, tokenPayload.sessionId), - }); + const sessionId = tokenPayload.sessionId; + if (sessionId) { + const session = await cachedSessionLookup(sessionId, () => + db.query.sessions?.findFirst({ + where: eq(schema.sessions.id, sessionId), + }) ?? Promise.resolve(undefined), + ); if (session) { context.locals.session = session; } @@ -132,7 +143,7 @@ async function onRequestInner( // interceptor, etc.) via HTML post-processing below. const cspNonce = randomBytes(16).toString("base64"); context.locals.cspNonce = cspNonce; - const cspHeader = `default-src 'self'; script-src 'self' 'nonce-${cspNonce}' blob:; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' data: https://fonts.gstatic.com; img-src 'self' data: https: blob:; connect-src 'self' https:;`; + const cspHeader = `default-src 'self'; script-src 'self' 'nonce-${cspNonce}' blob:; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' data: https://fonts.gstatic.com; img-src 'self' data: https: blob:; connect-src 'self' https:; object-src 'none'; base-uri 'self'; frame-ancestors 'self'; form-action 'self' https:;`; const response = await next(); const durationMs = performance.now() - startTime; diff --git a/src/pages/[owner]/[repo]/discussions/[id]/index.astro b/src/pages/[owner]/[repo]/discussions/[id]/index.astro new file mode 100644 index 00000000..34e55a28 --- /dev/null +++ b/src/pages/[owner]/[repo]/discussions/[id]/index.astro @@ -0,0 +1,296 @@ +--- +import RepoHeader from "@/components/repo/RepoHeader.astro"; +import { getDatabase } from "@/db"; +import { discussionComments, discussions } from "@/db/schema/discussions"; +import { repositories } from "@/db/schema/repositories"; +import { users } from "@/db/schema/users"; +import BaseLayout from "@/layouts/BaseLayout.astro"; +import { renderMarkdown } from "@/lib/markdown"; +import { canReadRepo, canWriteRepo } from "@/lib/permissions"; +import { and, eq } from "drizzle-orm"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; + +const { owner: ownerName, repo: repoName, id } = Astro.params; +const db = getDatabase() as unknown as NodePgDatabase; + +// 1. Fetch Repo & Owner +const [ownerUser] = await db + .select({ id: users.id, username: users.username }) + .from(users) + .where(eq(users.username, ownerName!)) + .limit(1); + +if (!ownerUser) return Astro.redirect("/404"); + +const [repoData] = await db + .select() + .from(repositories) + .where( + and( + eq(repositories.ownerId, ownerUser.id), + eq(repositories.name, repoName!), + ), + ) + .limit(1); + +if (!repoData) return Astro.redirect("/404"); + +const currentUser = Astro.locals.user; +const hasAccess = await canReadRepo(currentUser?.id, repoData); +if (!hasAccess) { + return Astro.redirect("/404"); +} + +// 2. Fetch Discussion + author +const [discussion] = await db + .select() + .from(discussions) + .where( + and( + eq(discussions.id, id!), + eq(discussions.repositoryId, repoData.id), + ), + ) + .limit(1); + +if (!discussion) return Astro.redirect("/404"); + +const [author] = await db + .select({ username: users.username, avatarUrl: users.avatarUrl }) + .from(users) + .where(eq(users.id, discussion.authorId)) + .limit(1); + +// 3. Fetch Comments + authors (flat in v1; parentId stored for future threading) +const commentRows = await db + .select({ + id: discussionComments.id, + parentId: discussionComments.parentId, + body: discussionComments.body, + createdAt: discussionComments.createdAt, + authorUsername: users.username, + authorAvatarUrl: users.avatarUrl, + }) + .from(discussionComments) + .innerJoin(users, eq(discussionComments.authorId, users.id)) + .where(eq(discussionComments.discussionId, discussion.id)) + .orderBy(discussionComments.createdAt); + +const bodyHtml = await renderMarkdown(discussion.body); +const commentsHtml = await Promise.all( + commentRows.map(async (comment) => ({ + ...comment, + html: await renderMarkdown(comment.body), + })), +); + +const canModerate = currentUser + ? discussion.authorId === currentUser.id || + (await canWriteRepo(currentUser.id, repoData)) + : false; + +function timeAgo(date: Date | null | undefined) { + if (!date) return ""; + const seconds = Math.floor((Date.now() - new Date(date).getTime()) / 1000); + const intervals: [number, string][] = [ + [31536000, "y"], + [2592000, "mo"], + [86400, "d"], + [3600, "h"], + [60, "m"], + ]; + for (const [secs, unit] of intervals) { + if (seconds >= secs) return `${Math.floor(seconds / secs)}${unit} ago`; + } + return `${seconds}s ago`; +} + +const repoForHeader = { + ...repoData, + owner: ownerUser.username, + topics: repoData.topics ? JSON.parse(repoData.topics) : [], +}; +--- + + + + +
+ +
+

{discussion.title}

+
+ + { + canModerate && ( +
+ + +
+ ) + } +
+ + +
+
+ {author?.username ?? "unknown"} + {timeAgo(discussion.createdAt)} +
+
+
+ + +
+

+ {commentRows.length} {commentRows.length === 1 ? "comment" : "comments"} +

+ { + commentsHtml.map((comment) => ( +
+
+ {comment.authorUsername} + {timeAgo(comment.createdAt)} +
+
+
+ )) + } +
+ + + { + currentUser ? ( +
+ + +
+ + {/* Options */} +
+ + +
+ + {/* Actions */} +
+ + + Cancel + +
+ + + + + + diff --git a/src/pages/api/auth/login.ts b/src/pages/api/auth/login.ts index 1f82fa4c..d1c7b6b0 100644 --- a/src/pages/api/auth/login.ts +++ b/src/pages/api/auth/login.ts @@ -3,7 +3,7 @@ */ import { getDatabase, schema } from "@/db"; import { sessions, users } from "@/db/schema"; -import { parseBody, success, unauthorized } from "@/lib/api"; +import { error, parseBody, success, unauthorized } from "@/lib/api"; import { createSession, createToken, @@ -11,6 +11,13 @@ import { verifyPassword, } from "@/lib/auth"; import { withErrorHandler } from "@/lib/errors"; +import { + clearLoginFailures, + ipKey, + isLockedOut, + recordLoginFailure, + userIpKey, +} from "@/lib/login-lockout"; import { logger } from "@/lib/logger"; import { applyRateLimit } from "@/middleware/rate-limit"; import { type APIRoute } from "astro"; @@ -38,6 +45,33 @@ export const POST: APIRoute = withErrorHandler(async ({ request, cookies }) => { const { login, password, totpCode } = parsed.data; const db = getDatabase() as NodePgDatabase; + const ipAddress = + request.headers.get("X-Forwarded-For")?.split(",")[0].trim() || + request.headers.get("X-Real-IP") || + undefined; + + // Brute-force lockout: check per-credentials and per-IP limits + const userLockKey = userIpKey(login, ipAddress); + const ipLockKey = ipKey(ipAddress); + const [userLockout, ipLockout] = await Promise.all([ + isLockedOut(userLockKey), + isLockedOut(ipLockKey), + ]); + + if (userLockout.locked || ipLockout.locked) { + const retryAfter = Math.max( + userLockout.retryAfterSecs ?? 0, + ipLockout.retryAfterSecs ?? 0, + ); + logger.warn( + { login: userLockKey, ip: ipLockKey }, + "Login blocked: too many failed attempts", + ); + const response = error("RATE_LIMITED", "Too many failed attempts", 429); + response.headers.set("Retry-After", String(retryAfter)); + return response; + } + // Find user by username or email const user = await db.query.users.findFirst({ where: (users, { or, eq }) => @@ -45,6 +79,10 @@ export const POST: APIRoute = withErrorHandler(async ({ request, cookies }) => { }); if (!user) { + await Promise.all([ + recordLoginFailure(userLockKey), + recordLoginFailure(ipLockKey), + ]); return unauthorized("Invalid credentials"); } @@ -55,6 +93,10 @@ export const POST: APIRoute = withErrorHandler(async ({ request, cookies }) => { // Verify password if (!user.passwordHash) { + await Promise.all([ + recordLoginFailure(userLockKey), + recordLoginFailure(ipLockKey), + ]); return unauthorized("Invalid credentials"); } @@ -62,6 +104,10 @@ export const POST: APIRoute = withErrorHandler(async ({ request, cookies }) => { if (!isValid) { logger.warn({ user: user.username }, "Invalid password attempt"); + await Promise.all([ + recordLoginFailure(userLockKey), + recordLoginFailure(ipLockKey), + ]); return unauthorized("Invalid credentials"); } @@ -86,10 +132,11 @@ export const POST: APIRoute = withErrorHandler(async ({ request, cookies }) => { // Create session const userAgent = request.headers.get("User-Agent") || undefined; - const ipAddress = - request.headers.get("X-Forwarded-For")?.split(",")[0].trim() || - request.headers.get("X-Real-IP") || - undefined; + + await Promise.all([ + clearLoginFailures(userLockKey), + clearLoginFailures(ipLockKey), + ]); const session = await createSession(user.id, userAgent, ipAddress); diff --git a/src/pages/api/auth/logout.ts b/src/pages/api/auth/logout.ts index 6b9361c5..d460c177 100644 --- a/src/pages/api/auth/logout.ts +++ b/src/pages/api/auth/logout.ts @@ -13,6 +13,10 @@ export const POST: APIRoute = withErrorHandler(async ({ cookies, locals, redirec try { const db = getDatabase() as NodePgDatabase; await db.delete(schema.sessions).where(eq(schema.sessions.id, session.id)); + const { invalidateAuthCache } = await import("@/lib/auth-cache"); + if (session.userId) { + invalidateAuthCache(session.userId, session.id); + } logger.info({ sessionId: session.id }, "Session deleted"); } catch (e) { logger.error({ err: e }, "Failed to delete session from database"); diff --git a/src/pages/api/auth/password.ts b/src/pages/api/auth/password.ts index ae135513..92472385 100644 --- a/src/pages/api/auth/password.ts +++ b/src/pages/api/auth/password.ts @@ -68,7 +68,12 @@ export const PATCH: APIRoute = withErrorHandler(async ({ request }) => { }) .where(eq(users.id, tokenPayload.userId)); - logger.info({ userId: tokenPayload.userId }, "Password updated"); + // Revoke all existing sessions — a password change must force re-login everywhere + await db.delete(schema.sessions).where(eq(schema.sessions.userId, tokenPayload.userId)); + const { revokeUserSessionCache } = await import("@/lib/auth-cache"); + revokeUserSessionCache(tokenPayload.userId); - return success({ message: "Password updated successfully" }); + logger.info({ userId: tokenPayload.userId }, "Password updated, all sessions revoked"); + + return success({ message: "Password updated successfully. Please sign in again." }); }); diff --git a/src/pages/api/auth/reset-password.ts b/src/pages/api/auth/reset-password.ts index 0e15cd99..bd84f778 100644 --- a/src/pages/api/auth/reset-password.ts +++ b/src/pages/api/auth/reset-password.ts @@ -55,6 +55,13 @@ export const POST: APIRoute = withErrorHandler(async ({ request }) => { }) .where(eq(users.id, resetToken.userId)); + // Revoke all sessions — reset tokens prove account compromise recovery path + await db + .delete(schema.sessions) + .where(eq(schema.sessions.userId, resetToken.userId)); + const { revokeUserSessionCache } = await import("@/lib/auth-cache"); + revokeUserSessionCache(resetToken.userId); + // Delete used token await db .delete(passwordResetTokens) diff --git a/src/pages/api/gists/[id]/index.ts b/src/pages/api/gists/[id]/index.ts new file mode 100644 index 00000000..cc6b1909 --- /dev/null +++ b/src/pages/api/gists/[id]/index.ts @@ -0,0 +1,113 @@ +/** + * Single Gist API + * GET /api/gists/[id] — fetch gist (owner or public/secret visibility) + * PATCH /api/gists/[id] — owner only; wholesale replace description/files/public + * DELETE /api/gists/[id] — owner only + */ + +import { getDatabase, schema } from "@/db"; +import { gists } from "@/db/schema/gists"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import { badRequest, noContent, notFound, forbidden, success, unauthorized } from "@/lib/api"; +import { getUserFromRequest } from "@/lib/auth"; +import { withErrorHandler } from "@/lib/errors"; +import { logger } from "@/lib/logger"; +import type { APIRoute } from "astro"; +import { eq } from "drizzle-orm"; +import { z } from "zod"; +import { createGistSchema, gistFilesSchema } from "../index"; + +export const updateGistSchema = z.object({ + description: z.string().max(500).optional(), + public: z.boolean().optional(), + files: gistFilesSchema.optional(), +}); + +async function findGist(db: NodePgDatabase, id: string) { + return db.query.gists.findFirst({ + where: eq(gists.id, id), + }); +} + +export const GET: APIRoute = withErrorHandler(async ({ request, params }) => { + const tokenPayload = await getUserFromRequest(request); + const db = getDatabase() as unknown as NodePgDatabase; + + const gist = await findGist(db, params.id!); + if (!gist) return notFound("Gist not found"); + + // Secret gists are only visible to their owner + if (!gist.public && gist.userId !== tokenPayload?.userId) { + return notFound("Gist not found"); + } + + return success({ + ...gist, + isOwner: gist.userId === tokenPayload?.userId, + }); +}); + +export const PATCH: APIRoute = withErrorHandler(async ({ request, params }) => { + const tokenPayload = await getUserFromRequest(request); + if (!tokenPayload) { + return unauthorized("You must be logged in to edit a gist"); + } + + let raw: unknown; + try { + raw = await request.json(); + } catch { + return badRequest("Invalid JSON body"); + } + + const result = updateGistSchema.safeParse(raw); + if (!result.success) { + return badRequest("Invalid input", result.error.flatten()); + } + + const db = getDatabase() as unknown as NodePgDatabase; + + const gist = await findGist(db, params.id!); + if (!gist) return notFound("Gist not found"); + if (gist.userId !== tokenPayload.userId) { + return forbidden("You do not have permission to edit this gist"); + } + + const updates: Partial = { updatedAt: new Date() }; + if (result.data.description !== undefined) { + updates.description = result.data.description; + } + if (result.data.public !== undefined) { + updates.public = result.data.public; + } + if (result.data.files !== undefined) { + updates.files = result.data.files; + } + + await db.update(gists).set(updates).where(eq(gists.id, gist.id)); + + logger.info({ userId: tokenPayload.userId, gistId: gist.id }, "Gist updated"); + + return success({ ...gist, ...updates }); +}); + +export const DELETE: APIRoute = withErrorHandler(async ({ request, params }) => { + const tokenPayload = await getUserFromRequest(request); + if (!tokenPayload) { + return unauthorized("You must be logged in to delete a gist"); + } + + const db = getDatabase() as unknown as NodePgDatabase; + + const gist = await findGist(db, params.id!); + if (!gist) return notFound("Gist not found"); + if (gist.userId !== tokenPayload.userId) { + return forbidden("You do not have permission to delete this gist"); + } + + await db.delete(gists).where(eq(gists.id, gist.id)); + + logger.info({ userId: tokenPayload.userId, gistId: gist.id }, "Gist deleted"); + + return noContent(); +}); diff --git a/src/pages/api/gists/[id]/raw/[file].ts b/src/pages/api/gists/[id]/raw/[file].ts new file mode 100644 index 00000000..7c701268 --- /dev/null +++ b/src/pages/api/gists/[id]/raw/[file].ts @@ -0,0 +1,50 @@ +/** + * Raw Gist File API + * GET /api/gists/[id]/raw/[file] — raw content of a single file as text/plain + * + * Public gists are readable anonymously; secret gists require the owner. + */ + +import { getDatabase, schema } from "@/db"; +import { gists } from "@/db/schema/gists"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import { notFound, unauthorized } from "@/lib/api"; +import { getUserFromRequest } from "@/lib/auth"; +import { withErrorHandler } from "@/lib/errors"; +import type { APIRoute } from "astro"; +import { eq } from "drizzle-orm"; + +export const GET: APIRoute = withErrorHandler(async ({ request, params }) => { + const tokenPayload = await getUserFromRequest(request); + const db = getDatabase() as unknown as NodePgDatabase; + + const gist = await db.query.gists.findFirst({ + where: eq(gists.id, params.id!), + }); + if (!gist) return notFound("Gist not found"); + + if (!gist.public && gist.userId !== tokenPayload?.userId) { + if (!tokenPayload) { + return unauthorized("You must be logged in to view this gist"); + } + return notFound("Gist not found"); + } + + let filename: string; + try { + filename = decodeURIComponent(params.file!); + } catch { + filename = params.file!; + } + + const file = gist.files.find((f) => f.filename === filename); + if (!file) return notFound("File not found in gist"); + + return new Response(file.content, { + status: 200, + headers: { + "Content-Type": "text/plain; charset=utf-8", + "X-Content-Type-Options": "nosniff", + }, + }); +}); diff --git a/src/pages/api/gists/index.ts b/src/pages/api/gists/index.ts new file mode 100644 index 00000000..3c129bf1 --- /dev/null +++ b/src/pages/api/gists/index.ts @@ -0,0 +1,172 @@ +/** + * Gists API + * GET /api/gists — list caller's gists (?public=true limits to public ones, ?q= substring search) + * POST /api/gists — create a gist (auth required) + */ + +import { getDatabase } from "@/db"; +import { gists, type GistFile } from "@/db/schema/gists"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import { + badRequest, + created, + getPagination, + paginationMeta, + success, + unauthorized, +} from "@/lib/api"; +import { getUserFromRequest } from "@/lib/auth"; +import { withErrorHandler } from "@/lib/errors"; +import { logger } from "@/lib/logger"; +import { generateId } from "@/lib/utils"; +import type { APIRoute } from "astro"; +import { and, desc, eq, ilike, or, sql } from "drizzle-orm"; +import { z } from "zod"; + +/** Max total content size across all files in a gist: 1MB */ +export const MAX_TOTAL_CONTENT_BYTES = 1024 * 1024; +export const MAX_FILES = 10; + +const FILENAME_INVALID_CHARS = /[/\\]|\.\./; + +export const gistFileSchema = z + .object({ + filename: z + .string() + .min(1, "filename is required") + .max(255, "filename must be at most 255 characters") + .refine((v) => !FILENAME_INVALID_CHARS.test(v), { + message: "filename must not contain path separators or '..'", + }), + content: z.string(), + }) + .refine( + (f) => f.filename !== "." && f.filename !== "..", + { message: "filename must not be '.' or '..'" }, + ); + +export const gistFilesSchema = z + .array(gistFileSchema) + .min(1, "at least one file is required") + .max(MAX_FILES); + +export const createGistSchema = z + .object({ + description: z.string().max(500).optional().default(""), + public: z.boolean().optional().default(false), + files: gistFilesSchema, + }) + .refine( + (g) => + g.files.reduce((sum, f) => sum + Buffer.byteLength(f.content, "utf8"), 0) <= + MAX_TOTAL_CONTENT_BYTES, + { message: "total content size exceeds 1MB" }, + ); + +export const listGistsQuerySchema = z.object({ + public: z + .enum(["true", "false"]) + .transform((v) => v === "true") + .optional(), + q: z.string().max(255).optional(), +}); + +function totalContentBytes(files: GistFile[]): number { + return files.reduce((sum, f) => sum + Buffer.byteLength(f.content, "utf8"), 0); +} + +export const GET: APIRoute = withErrorHandler(async ({ request }) => { + const tokenPayload = await getUserFromRequest(request); + if (!tokenPayload) { + return unauthorized("You must be logged in to list your gists"); + } + + const url = new URL(request.url); + const query = listGistsQuerySchema.safeParse(Object.fromEntries(url.searchParams)); + if (!query.success) { + return badRequest("Invalid query parameters", query.error.flatten()); + } + const { public: publicOnly, q } = query.data; + const pagination = getPagination(url); + + const db = getDatabase() as unknown as NodePgDatabase; + + const conditions = [eq(gists.userId, tokenPayload.userId)]; + if (publicOnly) conditions.push(eq(gists.public, true)); + if (q && q.length > 0) { + conditions.push( + or( + ilike(gists.description, `%${q}%`), + sql`CAST(${gists.files} AS TEXT) ILIKE ${`%${q}%`}`, + )!, + ); + } + + const rows = await db + .select() + .from(gists) + .where(and(...conditions)) + .orderBy(desc(gists.updatedAt)) + .limit(pagination.perPage) + .offset(pagination.offset); + + const [countRow] = await db + .select({ total: sql`count(*)::int` }) + .from(gists) + .where(and(...conditions)); + + return success( + rows.map((row) => ({ + ...row, + fileCount: row.files.length, + totalBytes: totalContentBytes(row.files), + })), + paginationMeta(countRow?.total ?? 0, pagination), + ); +}); + +export const POST: APIRoute = withErrorHandler(async ({ request }) => { + const tokenPayload = await getUserFromRequest(request); + if (!tokenPayload) { + return unauthorized("You must be logged in to create a gist"); + } + + let raw: unknown; + try { + raw = await request.json(); + } catch { + return badRequest("Invalid JSON body"); + } + + const result = createGistSchema.safeParse(raw); + if (!result.success) { + return badRequest("Invalid input", result.error.flatten()); + } + const { description, public: isPublic, files } = result.data; + + const db = getDatabase() as unknown as NodePgDatabase; + + const now = new Date(); + const newGist = { + id: generateId("gist"), + userId: tokenPayload.userId, + description, + public: isPublic, + files, + createdAt: now, + updatedAt: now, + }; + + await db.insert(gists).values(newGist); + + logger.info( + { userId: tokenPayload.userId, gistId: newGist.id }, + "Gist created", + ); + + return created({ + ...newGist, + fileCount: files.length, + totalBytes: totalContentBytes(files), + }); +}); diff --git a/src/pages/api/metrics.ts b/src/pages/api/metrics.ts index 2c366348..e1899a1e 100644 --- a/src/pages/api/metrics.ts +++ b/src/pages/api/metrics.ts @@ -5,8 +5,8 @@ import { withErrorHandler } from "@/lib/errors"; function isAuthorized(request: Request): boolean { const expected = process.env.METRICS_TOKEN; - // When no token is configured the endpoint stays public (e.g. air-gapped - // single-host installs). For production, set METRICS_TOKEN and restrict + // When unset, the endpoint is public only outside production (handled in + // GET below). In production METRICS_TOKEN must be configured; also restrict // network access in the reverse proxy. if (!expected) return true; @@ -21,6 +21,18 @@ function isAuthorized(request: Request): boolean { } export const GET: APIRoute = withErrorHandler(async ({ request }) => { + // Deny by default in production: scraping metrics requires METRICS_TOKEN. + // Non-production stays public when unset (dev convenience). + if (!process.env.METRICS_TOKEN && process.env.NODE_ENV === "production") { + return new Response( + JSON.stringify({ error: "METRICS_TOKEN must be set in production" }), + { + status: 403, + headers: { "Content-Type": "application/json" }, + }, + ); + } + if (!isAuthorized(request)) { return new Response("Unauthorized", { status: 401 }); } diff --git a/src/pages/api/packages/docker/v2/[...path].ts b/src/pages/api/packages/docker/v2/[...path].ts index 6a49c4d3..934dbbaa 100644 --- a/src/pages/api/packages/docker/v2/[...path].ts +++ b/src/pages/api/packages/docker/v2/[...path].ts @@ -1,6 +1,10 @@ /** * Docker/OCI Registry v2 API - * Implements the OCI Distribution Spec endpoints + * Implements the OCI Distribution Spec endpoints including the full push flow: + * POST (initiate) -> PATCH (chunks) -> PUT ?digest= (finalize), plus blob GET. + * + * Auth: PAT via Basic auth or Bearer token (same model as the npm registry + * routes). Pulls are anonymous; pushes require authentication. */ import { logger } from "@/lib/logger"; @@ -12,13 +16,69 @@ import { listDockerTags, publishVersion, } from "@/lib/packages"; +import { + appendToUpload, + cancelUpload, + createUploadSession, + finalizeUpload, + getUploadSession, +} from "@/lib/docker-registry-upload"; import { getStorage } from "@/lib/storage"; import type { APIRoute } from "astro"; import crypto from "node:crypto"; -/** GET /api/packages/docker/v2 — Registry ping */ +const V2_PREFIX = "/api/packages/docker/v2"; + +function ociError( + errors: Array<{ code: string; message?: string }>, + status: number, + headers?: Record, +) { + return new Response(JSON.stringify({ errors }), { + status, + headers: { + "Content-Type": "application/json", + "Docker-Distribution-API-Version": "registry/2.0", + ...headers, + }, + }); +} + +function blobStorageKey(imageName: string, digest: string): string { + return `packages/docker/${imageName}/blobs/${digest}`; +} + +/** Authenticate a registry client: Basic (username + PAT/password) or Bearer. */ +async function authenticate(request: Request): Promise { + const authHeader = request.headers.get("authorization") || ""; + try { + if (authHeader.startsWith("Basic ")) { + const { validateBasicAuth } = await import("@/lib/auth-basic"); + return await validateBasicAuth(authHeader); + } + if (authHeader.startsWith("Bearer ")) { + const { getUserFromRequest } = await import("@/lib/auth"); + const payload = await getUserFromRequest(request); + return payload?.userId ?? null; + } + } catch (error) { + logger.debug( + { error: error instanceof Error ? error.message : "unknown" }, + "Docker registry auth failed", + ); + } + return null; +} + +function unauthorized() { + return ociError([{ code: "UNAUTHORIZED", message: "Authentication required" }], 401, { + "WWW-Authenticate": 'Basic realm="opencodehub"', + }); +} + +/** GET /api/packages/docker/v2[...] */ export const GET: APIRoute = async ({ url }) => { - const path = url.pathname.replace("/api/packages/docker/v2", ""); + const path = url.pathname.replace(V2_PREFIX, ""); // Base ping: GET /v2/ if (!path || path === "/") { @@ -38,10 +98,7 @@ export const GET: APIRoute = async ({ url }) => { const imageName = tagsMatch[1]; const result = await listDockerTags(orgId, imageName); if (!result) { - return new Response( - JSON.stringify({ errors: [{ code: "NAME_UNKNOWN" }] }), - { status: 404 }, - ); + return ociError([{ code: "NAME_UNKNOWN" }], 404); } return new Response(JSON.stringify(result), { headers: { "Content-Type": "application/json" }, @@ -54,10 +111,7 @@ export const GET: APIRoute = async ({ url }) => { const [, imageName, reference] = manifestMatch; const result = await getDockerManifest(orgId, imageName, reference); if (!result) { - return new Response( - JSON.stringify({ errors: [{ code: "MANIFEST_UNKNOWN" }] }), - { status: 404 }, - ); + return ociError([{ code: "MANIFEST_UNKNOWN" }], 404); } return new Response(JSON.stringify(result.manifest), { headers: { @@ -67,49 +121,213 @@ export const GET: APIRoute = async ({ url }) => { }); } - // HEAD /v2/:name/blobs/:digest - const blobMatch = path.match(/^\/([^/]+(?:\/[^/]+)*)\/blobs\/(.+)$/); - if (blobMatch) { - const [, imageName, digest] = blobMatch; - const result = await checkDockerBlobExists(orgId, imageName, digest); - if (!result.exists) { - return new Response(null, { status: 404 }); + // GET /v2/:name/blobs/:digest — stream blob content from storage + const blobGetMatch = path.match( + /^\/([^/]+(?:\/[^/]+)*)\/blobs\/(sha256:[a-f0-9]{64})$/, + ); + if (blobGetMatch) { + const [, imageName, digest] = blobGetMatch; + const storage = await getStorage(); + const key = blobStorageKey(imageName, digest); + if (!(await storage.exists(key))) { + return ociError([{ code: "BLOB_UNKNOWN" }], 404); } + const obj = await storage.stat(key); + const stream = await storage.getStream(key); + return new Response(stream as unknown as ReadableStream, { + status: 200, + headers: { + "Content-Type": "application/octet-stream", + "Content-Length": String(obj.size ?? 0), + "Docker-Content-Digest": digest, + "Cache-Control": "private, max-age=31536000, immutable", + }, + }); + } + + return ociError([{ code: "UNSUPPORTED" }], 404); +}; + +/** HEAD /api/packages/docker/v2/:name/blobs/:digest */ +export const HEAD: APIRoute = async ({ url }) => { + const path = url.pathname.replace(V2_PREFIX, ""); + const blobMatch = path.match(/^\/([^/]+(?:\/[^/]+)*)\/blobs\/(.+)$/); + if (!blobMatch) { + return new Response(null, { status: 404 }); + } + const [, imageName, digest] = blobMatch; + + // Fast path: content-addressed canonical storage (written by the push flow) + const storage = await getStorage(); + const key = blobStorageKey(imageName, digest); + if (await storage.exists(key)) { + const obj = await storage.stat(key); return new Response(null, { status: 200, headers: { "Docker-Content-Digest": digest, - "Content-Length": String(result.size || 0), + "Content-Length": String(obj.size ?? 0), + }, + }); + } + + // Fallback: legacy metadata lookup (pushes made before blob storage existed) + const orgId = "default"; + const result = await checkDockerBlobExists(orgId, imageName, digest); + if (!result.exists) { + return new Response(null, { status: 404 }); + } + return new Response(null, { + status: 200, + headers: { + "Docker-Content-Digest": digest, + "Content-Length": String(result.size || 0), + }, + }); +}; + +/** POST — initiate chunked upload, or monolithic push with ?digest= */ +export const POST: APIRoute = async ({ url, request }) => { + const path = url.pathname.replace(V2_PREFIX, ""); + const userId = await authenticate(request); + if (!userId) return unauthorized(); + + const initMatch = path.match( + /^\/([^/]+(?:\/[^/]+)*)\/blobs\/uploads\/$/, + ); + if (!initMatch) { + return ociError([{ code: "UNSUPPORTED" }], 404); + } + const imageName = initMatch[1]; + + // Monolithic single-request upload: POST /v2/:name/blobs/uploads/?digest=sha256:... + const monolithicDigest = url.searchParams.get("digest"); + if (monolithicDigest) { + const body = Buffer.from(await request.arrayBuffer()); + const computed = `sha256:${crypto.createHash("sha256").update(body).digest("hex")}`; + if (computed !== monolithicDigest) { + return ociError([{ code: "DIGEST_INVALID" }], 400); + } + const storage = await getStorage(); + const key = blobStorageKey(imageName, computed); + await storage.put(key, body, { contentType: "application/octet-stream" }); + return new Response(null, { + status: 201, + headers: { + "Docker-Content-Digest": computed, + Location: `${V2_PREFIX}/${imageName}/blobs/${computed}`, }, }); } - return new Response(JSON.stringify({ errors: [{ code: "UNSUPPORTED" }] }), { - status: 404, + // Chunked upload initiation + const session = await createUploadSession(imageName); + return new Response(null, { + status: 202, + headers: { + Location: `${V2_PREFIX}/${imageName}/blobs/uploads/${session.id}`, + "Docker-Upload-UUID": session.id, + Range: "0-0", + }, }); }; -/** PUT /v2/:name/manifests/:reference — Push manifest */ +/** PATCH — append a chunk to an in-flight upload */ +export const PATCH: APIRoute = async ({ url, request }) => { + const path = url.pathname.replace(V2_PREFIX, ""); + const userId = await authenticate(request); + if (!userId) return unauthorized(); + + const patchMatch = path.match( + /^\/([^/]+(?:\/[^/]+)*)\/blobs\/uploads\/([a-f0-9-]{36})$/, + ); + if (!patchMatch) { + return ociError([{ code: "UNSUPPORTED" }], 404); + } + const [, imageName, uploadId] = patchMatch; + + const session = getUploadSession(uploadId); + if (!session || session.imageName !== imageName) { + return ociError([{ code: "BLOB_UPLOAD_UNKNOWN" }], 404); + } + + const chunk = Buffer.from(await request.arrayBuffer()); + const result = await appendToUpload(uploadId, chunk); + if (!result.ok) { + return ociError([{ code: result.reason }], 404); + } + + const end = result.session.size; + return new Response(null, { + status: 202, + headers: { + Location: `${V2_PREFIX}/${imageName}/blobs/uploads/${uploadId}`, + "Docker-Upload-UUID": uploadId, + Range: `0-${end > 0 ? end - 1 : 0}`, + }, + }); +}; + +/** PUT — finalize an upload with ?digest=, or push a manifest */ export const PUT: APIRoute = async ({ url, request }) => { - const path = url.pathname.replace("/api/packages/docker/v2", ""); - const orgId = "default"; - const userId = request.headers.get("x-user-id"); + const path = url.pathname.replace(V2_PREFIX, ""); + const userId = await authenticate(request); + if (!userId) return unauthorized(); + + // PUT /v2/:name/blobs/uploads/:uuid?digest=sha256:... — finalize blob upload + const finalizeMatch = path.match( + /^\/([^/]+(?:\/[^/]+)*)\/blobs\/uploads\/([a-f0-9-]{36})$/, + ); + if (finalizeMatch && url.searchParams.has("digest")) { + const [, imageName, uploadId] = finalizeMatch; + const digest = url.searchParams.get("digest") || ""; + if (!/^sha256:[a-f0-9]{64}$/.test(digest)) { + return ociError([{ code: "DIGEST_INVALID", message: "malformed digest" }], 400); + } + + // Support a final chunk carried on the PUT itself + const hasBody = + request.headers.get("content-length") && + request.headers.get("content-length") !== "0"; + if (hasBody) { + const chunk = Buffer.from(await request.arrayBuffer()); + const appended = await appendToUpload(uploadId, chunk); + if (!appended.ok) { + return ociError([{ code: appended.reason }], 404); + } + } + + const result = await finalizeUpload(uploadId, digest, blobStorageKey(imageName, digest)); + if (!result.ok) { + return ociError([{ code: result.reason }], result.status); + } - if (!userId) { - return new Response( - JSON.stringify({ errors: [{ code: "UNAUTHORIZED" }] }), - { status: 401 }, + logger.info( + { image: imageName, digest: result.digest, size: result.size }, + "Docker blob stored", ); + return new Response(null, { + status: 201, + headers: { + "Docker-Content-Digest": result.digest, + Location: `${V2_PREFIX}/${imageName}/blobs/${result.digest}`, + }, + }); + } + if (finalizeMatch) { + // Upload exists but no digest supplied — cancel per spec ambiguity + await cancelUpload(finalizeMatch[2]); + return ociError([{ code: "DIGEST_INVALID", message: "missing digest" }], 400); } + // PUT /v2/:name/manifests/:reference — push manifest const manifestMatch = path.match(/^\/([^/]+(?:\/[^/]+)*)\/manifests\/(.+)$/); if (!manifestMatch) { - return new Response(JSON.stringify({ errors: [{ code: "UNSUPPORTED" }] }), { - status: 404, - }); + return ociError([{ code: "UNSUPPORTED" }], 404); } const [, imageName, reference] = manifestMatch; + const orgId = "default"; try { const body = await request.text(); @@ -170,9 +388,30 @@ export const PUT: APIRoute = async ({ url, request }) => { { error: msg, image: imageName }, "Docker manifest push failed", ); - return new Response( - JSON.stringify({ errors: [{ code: "MANIFEST_INVALID", message: msg }] }), - { status: 400 }, - ); + return ociError([{ code: "MANIFEST_INVALID", message: msg }], 400); + } +}; + +/** DELETE — abort an in-flight upload */ +export const DELETE: APIRoute = async ({ url, request }) => { + const path = url.pathname.replace(V2_PREFIX, ""); + const userId = await authenticate(request); + if (!userId) return unauthorized(); + + const cancelMatch = path.match( + /^\/([^/]+(?:\/[^/]+)*)\/blobs\/uploads\/([a-f0-9-]{36})$/, + ); + if (!cancelMatch) { + return ociError([{ code: "UNSUPPORTED" }], 404); } + + const session = getUploadSession(cancelMatch[2]); + if (!session || session.imageName !== cancelMatch[1]) { + return ociError([{ code: "BLOB_UPLOAD_UNKNOWN" }], 404); + } + + await cancelUpload(cancelMatch[2]); + return new Response(null, { status: 204 }); }; + + diff --git a/src/pages/api/repos/[owner]/[repo]/compare/index.ts b/src/pages/api/repos/[owner]/[repo]/compare/index.ts index 15c3af0c..1950df64 100644 --- a/src/pages/api/repos/[owner]/[repo]/compare/index.ts +++ b/src/pages/api/repos/[owner]/[repo]/compare/index.ts @@ -1,5 +1,6 @@ import { getDatabase, schema } from "@/db"; import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import { verifyCommitsSignatures } from "@/lib/commit-signature"; import { compareBranches, getMergeBase } from "@/lib/git"; import { resolveRepoPath } from "@/lib/git-storage"; import { canReadRepo } from "@/lib/permissions"; @@ -53,5 +54,25 @@ export const GET: APIRoute = withErrorHandler(async ({ params, request, locals } const { commits, diffs } = await compareBranches(repoPath, base, head); const mergeBase = await getMergeBase(repoPath, base, head); - return success({ commits, diffs, mergeBase }); + // Commit signature verification (GitHub-style "Verified") + let commitList = commits; + if ( + process.env.COMMIT_SIGNATURE_VERIFICATION !== "false" && + commits.length > 0 && + commits.some((c) => c.verification && c.verification.status !== "N") + ) { + const verifications = await verifyCommitsSignatures( + ownerName, + repoName, + commits.map((c) => c.sha), + ); + if (verifications.size > 0) { + commitList = commits.map((c) => ({ + ...c, + signatureVerification: verifications.get(c.sha), + })); + } + } + + return success({ commits: commitList, diffs, mergeBase }); }); diff --git a/src/pages/api/repos/[owner]/[repo]/discussions/[id]/comments.ts b/src/pages/api/repos/[owner]/[repo]/discussions/[id]/comments.ts new file mode 100644 index 00000000..89b5785d --- /dev/null +++ b/src/pages/api/repos/[owner]/[repo]/discussions/[id]/comments.ts @@ -0,0 +1,217 @@ +import { getDatabase } from "@/db"; +import { discussionComments, discussions } from "@/db/schema/discussions"; +import { repositories } from "@/db/schema/repositories"; +import { users } from "@/db/schema/users"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import { + badRequest, + created, + getPagination, + notFound, + paginationMeta, + success, + unauthorized, +} from "@/lib/api"; +import { getUserFromRequest } from "@/lib/auth"; +import { withErrorHandler } from "@/lib/errors"; +import { canReadRepo } from "@/lib/permissions"; +import { generateId } from "@/lib/utils"; +import type { APIRoute } from "astro"; +import { and, eq, sql } from "drizzle-orm"; +import { z } from "zod"; + +export const createCommentSchema = z.object({ + body: z.string().min(1).max(65535), + parentId: z.string().optional(), +}); + +export const GET: APIRoute = withErrorHandler(async ({ request, params }) => { + const { owner: ownerName, repo: repoName, id } = params; + const db = getDatabase() as unknown as NodePgDatabase; + + // 1. Resolve owner & repository & discussion + const [owner] = await db + .select({ id: users.id }) + .from(users) + .where(eq(users.username, ownerName!)) + .limit(1); + + if (!owner) return notFound("Owner not found"); + + const [repo] = await db + .select() + .from(repositories) + .where( + and(eq(repositories.ownerId, owner.id), eq(repositories.name, repoName!)), + ) + .limit(1); + + if (!repo) return notFound("Repository not found"); + + // Read permission (anonymous OK for public repos) + const tokenPayload = await getUserFromRequest(request); + const hasAccess = await canReadRepo(tokenPayload?.userId, repo as any); + if (!hasAccess) return notFound("Repository not found"); + + const [discussion] = await db + .select() + .from(discussions) + .where( + and( + eq(discussions.id, id!), + eq(discussions.repositoryId, repo.id), + ), + ) + .limit(1); + + if (!discussion) return notFound("Discussion not found"); + + // 2. Paginated comments (oldest first, thread order) + const pagination = getPagination(new URL(request.url)); + + const comments = await db + .select({ + id: discussionComments.id, + parentId: discussionComments.parentId, + body: discussionComments.body, + createdAt: discussionComments.createdAt, + updatedAt: discussionComments.updatedAt, + author: { + id: users.id, + username: users.username, + avatarUrl: users.avatarUrl, + }, + }) + .from(discussionComments) + .innerJoin(users, eq(discussionComments.authorId, users.id)) + .where(eq(discussionComments.discussionId, discussion.id)) + .orderBy(discussionComments.createdAt) + .limit(pagination.perPage) + .offset(pagination.offset); + + const [countRow] = await db + .select({ total: sql`count(*)::int` }) + .from(discussionComments) + .where(eq(discussionComments.discussionId, discussion.id)); + + return success(comments, paginationMeta(countRow?.total ?? 0, pagination)); +}); + +export const POST: APIRoute = withErrorHandler(async ({ request, params }) => { + const { owner: ownerName, repo: repoName, id } = params; + + // 1. Authenticate + const tokenPayload = await getUserFromRequest(request); + if (!tokenPayload) { + return unauthorized("You must be logged in to comment"); + } + const userId = tokenPayload.userId; + + // 2. Parse body + let raw: unknown; + try { + raw = await request.json(); + } catch { + return badRequest("Invalid JSON body"); + } + const result = createCommentSchema.safeParse(raw); + if (!result.success) { + return badRequest("Invalid input", result.error.flatten()); + } + const { body, parentId } = result.data; + + const db = getDatabase() as unknown as NodePgDatabase; + + // 3. Resolve repository & discussion + const [owner] = await db + .select({ id: users.id }) + .from(users) + .where(eq(users.username, ownerName!)) + .limit(1); + + if (!owner) return notFound("Owner not found"); + + const [repo] = await db + .select() + .from(repositories) + .where( + and(eq(repositories.ownerId, owner.id), eq(repositories.name, repoName!)), + ) + .limit(1); + + if (!repo) return notFound("Repository not found"); + + // GitHub model: anyone who can read the repository can comment + const hasPermission = await canReadRepo(userId, repo as any); + if (!hasPermission) return notFound("Repository not found"); + + const [discussion] = await db + .select() + .from(discussions) + .where( + and( + eq(discussions.id, id!), + eq(discussions.repositoryId, repo.id), + ), + ) + .limit(1); + + if (!discussion) return notFound("Discussion not found"); + + // 4. Validate parent comment belongs to this discussion (one-level threading) + if (parentId) { + const [parent] = await db + .select({ id: discussionComments.id, parentId: discussionComments.parentId }) + .from(discussionComments) + .where( + and( + eq(discussionComments.id, parentId), + eq(discussionComments.discussionId, discussion.id), + ), + ) + .limit(1); + + if (!parent) { + return badRequest("Parent comment not found on this discussion"); + } + if (parent.parentId) { + return badRequest("Replies beyond one level are not supported"); + } + } + + // 5. Insert comment + update counters atomically + const commentId = generateId("dcomment"); + const createdAt = new Date(); + + await db.transaction(async (tx) => { + await tx.insert(discussionComments).values({ + id: commentId, + discussionId: discussion.id, + parentId: parentId ?? null, + authorId: userId, + body, + createdAt, + updatedAt: createdAt, + }); + + await tx + .update(discussions) + .set({ + commentCount: sql`${discussions.commentCount} + 1`, + lastActivityAt: createdAt, + updatedAt: createdAt, + } as any) + .where(eq(discussions.id, discussion.id)); + }); + + return created({ + id: commentId, + discussionId: discussion.id, + parentId: parentId ?? null, + authorId: userId, + body, + createdAt, + updatedAt: createdAt, + author: { id: userId, username: tokenPayload.username }, + }); +}); diff --git a/src/pages/api/repos/[owner]/[repo]/discussions/[id]/index.ts b/src/pages/api/repos/[owner]/[repo]/discussions/[id]/index.ts new file mode 100644 index 00000000..a30c49a4 --- /dev/null +++ b/src/pages/api/repos/[owner]/[repo]/discussions/[id]/index.ts @@ -0,0 +1,237 @@ +import { getDatabase } from "@/db"; +import { + discussionComments, + discussions, + DISCUSSION_CATEGORIES, +} from "@/db/schema/discussions"; +import { repositories } from "@/db/schema/repositories"; +import { users } from "@/db/schema/users"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import { + badRequest, + forbidden, + getPagination, + noContent, + notFound, + paginationMeta, + success, + unauthorized, +} from "@/lib/api"; +import { getUserFromRequest } from "@/lib/auth"; +import { withErrorHandler } from "@/lib/errors"; +import { canAdminRepo, canReadRepo, canWriteRepo } from "@/lib/permissions"; +import type { APIRoute } from "astro"; +import { and, eq, sql } from "drizzle-orm"; +import { z } from "zod"; + +export const updateDiscussionSchema = z + .object({ + title: z.string().min(1).max(300), + body: z.string().min(1).max(65535), + category: z.enum(DISCUSSION_CATEGORIES), + closed: z.boolean(), + pinned: z.boolean(), + }) + .partial() + .refine((data) => Object.keys(data).length > 0, { + message: "At least one field must be provided", + }); + +async function resolveDiscussion( + db: NodePgDatabase, + ownerName: string, + repoName: string, + discussionId: string, +): Promise<{ repo?: any; discussion?: any }> { + const [owner] = await db + .select({ id: users.id }) + .from(users) + .where(eq(users.username, ownerName)) + .limit(1); + + if (!owner) return {}; + + const [repo] = await db + .select() + .from(repositories) + .where( + and(eq(repositories.ownerId, owner.id), eq(repositories.name, repoName)), + ) + .limit(1); + + if (!repo) return {}; + + const [discussion] = await db + .select() + .from(discussions) + .where( + and( + eq(discussions.id, discussionId), + eq(discussions.repositoryId, repo.id), + ), + ) + .limit(1); + + if (!discussion) return {}; + + return { repo, discussion }; +} + +export const GET: APIRoute = withErrorHandler(async ({ request, params }) => { + const { owner: ownerName, repo: repoName, id } = params; + const db = getDatabase() as unknown as NodePgDatabase; + + const { repo, discussion } = await resolveDiscussion( + db, + ownerName!, + repoName!, + id!, + ); + if (!repo || !discussion) return notFound("Discussion not found"); + + // Read permission (anonymous OK for public repos) + const tokenPayload = await getUserFromRequest(request); + const hasAccess = await canReadRepo(tokenPayload?.userId, repo as any); + if (!hasAccess) return notFound("Discussion not found"); + + const pagination = getPagination(new URL(request.url)); + + const comments = await db + .select({ + id: discussionComments.id, + parentId: discussionComments.parentId, + body: discussionComments.body, + createdAt: discussionComments.createdAt, + updatedAt: discussionComments.updatedAt, + author: { + id: users.id, + username: users.username, + avatarUrl: users.avatarUrl, + }, + }) + .from(discussionComments) + .innerJoin(users, eq(discussionComments.authorId, users.id)) + .where(eq(discussionComments.discussionId, discussion.id)) + .orderBy(discussionComments.createdAt) + .limit(pagination.perPage) + .offset(pagination.offset); + + const [countRow] = await db + .select({ total: sql`count(*)::int` }) + .from(discussionComments) + .where(eq(discussionComments.discussionId, discussion.id)); + + const [author] = await db + .select({ + id: users.id, + username: users.username, + avatarUrl: users.avatarUrl, + }) + .from(users) + .where(eq(users.id, discussion.authorId)) + .limit(1); + + return success( + { ...discussion, author: author ?? null, comments }, + paginationMeta(countRow?.total ?? 0, pagination), + ); +}); + +export const PATCH: APIRoute = withErrorHandler(async ({ request, params }) => { + const { owner: ownerName, repo: repoName, id } = params; + + // 1. Authenticate + const tokenPayload = await getUserFromRequest(request); + if (!tokenPayload) { + return unauthorized("You must be logged in to edit a discussion"); + } + const userId = tokenPayload.userId; + + // 2. Parse body + let raw: unknown; + try { + raw = await request.json(); + } catch { + return badRequest("Invalid JSON body"); + } + const result = updateDiscussionSchema.safeParse(raw); + if (!result.success) { + return badRequest("Invalid input", result.error.flatten()); + } + + const db = getDatabase() as unknown as NodePgDatabase; + + // 3. Resolve discussion + const { repo, discussion } = await resolveDiscussion( + db, + ownerName!, + repoName!, + id!, + ); + if (!repo || !discussion) return notFound("Discussion not found"); + + // 4. Permission: author or repo write access + const isAuthor = discussion.authorId === userId; + const canWrite = await canWriteRepo(userId, repo as any); + if (!isAuthor && !canWrite) { + return forbidden("You do not have permission to edit this discussion"); + } + + // 5. Apply updates (title/body/category/close/reopen/pin/unpin) + const updates: Record = {}; + if (result.data.title !== undefined) updates.title = result.data.title; + if (result.data.body !== undefined) updates.body = result.data.body; + if (result.data.category !== undefined) { + updates.category = result.data.category; + } + if (result.data.closed !== undefined) updates.closed = result.data.closed; + if (result.data.pinned !== undefined) updates.pinned = result.data.pinned; + updates.updatedAt = new Date(); + + await db + .update(discussions) + .set(updates as any) + .where(eq(discussions.id, discussion.id)); + + const [updated] = await db + .select() + .from(discussions) + .where(eq(discussions.id, discussion.id)) + .limit(1); + + return success(updated); +}); + +export const DELETE: APIRoute = withErrorHandler(async ({ request, params }) => { + const { owner: ownerName, repo: repoName, id } = params; + + // 1. Authenticate + const tokenPayload = await getUserFromRequest(request); + if (!tokenPayload) { + return unauthorized("You must be logged in to delete a discussion"); + } + const userId = tokenPayload.userId; + + const db = getDatabase() as unknown as NodePgDatabase; + + // 2. Resolve discussion + const { repo, discussion } = await resolveDiscussion( + db, + ownerName!, + repoName!, + id!, + ); + if (!repo || !discussion) return notFound("Discussion not found"); + + // 3. Permission: author or repo admin + const isAuthor = discussion.authorId === userId; + const isAdmin = await canAdminRepo(userId, repo as any); + if (!isAuthor && !isAdmin) { + return forbidden("You do not have permission to delete this discussion"); + } + + // Comments are removed via ON DELETE CASCADE + await db.delete(discussions).where(eq(discussions.id, discussion.id)); + + return noContent(); +}); diff --git a/src/pages/api/repos/[owner]/[repo]/discussions/index.ts b/src/pages/api/repos/[owner]/[repo]/discussions/index.ts new file mode 100644 index 00000000..a721bc1c --- /dev/null +++ b/src/pages/api/repos/[owner]/[repo]/discussions/index.ts @@ -0,0 +1,192 @@ +import { getDatabase } from "@/db"; +import { discussions, DISCUSSION_CATEGORIES } from "@/db/schema/discussions"; +import { repositories } from "@/db/schema/repositories"; +import { users } from "@/db/schema/users"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; +import { + badRequest, + created, + getPagination, + notFound, + paginationMeta, + success, + unauthorized, +} from "@/lib/api"; +import { getUserFromRequest } from "@/lib/auth"; +import { withErrorHandler } from "@/lib/errors"; +import { logger } from "@/lib/logger"; +import { canReadRepo } from "@/lib/permissions"; +import { generateId } from "@/lib/utils"; +import type { APIRoute } from "astro"; +import { and, desc, eq, sql } from "drizzle-orm"; +import { z } from "zod"; + +export const createDiscussionSchema = z.object({ + title: z.string().min(1).max(300), + body: z.string().min(1).max(65535), + category: z.enum(DISCUSSION_CATEGORIES).optional().default("General"), +}); + +export const listDiscussionsQuerySchema = z.object({ + category: z.enum(DISCUSSION_CATEGORIES).optional(), + closed: z + .string() + .refine((v): v is "true" | "false" => v === "true" || v === "false", { + message: "closed must be 'true' or 'false'", + }) + .transform((v) => v === "true") + .optional(), + sort: z.enum(["lastActivity", "newest"]).optional().default("lastActivity"), +}); + +async function resolveRepo(db: NodePgDatabase, ownerName: string, repoName: string) { + const [owner] = await db + .select({ id: users.id }) + .from(users) + .where(eq(users.username, ownerName)) + .limit(1); + + if (!owner) return null; + + const [repo] = await db + .select() + .from(repositories) + .where( + and(eq(repositories.ownerId, owner.id), eq(repositories.name, repoName)), + ) + .limit(1); + + return repo ?? null; +} + +export const GET: APIRoute = withErrorHandler(async ({ request, params }) => { + const { owner: ownerName, repo: repoName } = params; + const db = getDatabase() as unknown as NodePgDatabase; + + // 1. Resolve owner & repository + const repo = await resolveRepo(db, ownerName!, repoName!); + if (!repo) return notFound("Repository not found"); + + // 2. Read permission (anonymous OK for public repos) + const tokenPayload = await getUserFromRequest(request); + const hasAccess = await canReadRepo(tokenPayload?.userId, repo as any); + if (!hasAccess) return notFound("Repository not found"); + + // 3. Filters + pagination + const url = new URL(request.url); + const query = listDiscussionsQuerySchema.safeParse( + Object.fromEntries(url.searchParams), + ); + if (!query.success) { + return badRequest("Invalid query parameters", query.error.flatten()); + } + const { category, closed, sort } = query.data; + const pagination = getPagination(url); + + const conditions = [eq(discussions.repositoryId, repo.id)]; + if (category) conditions.push(eq(discussions.category, category)); + if (closed !== undefined) conditions.push(eq(discussions.closed, closed)); + + const rows = await db + .select({ + id: discussions.id, + title: discussions.title, + body: discussions.body, + category: discussions.category, + pinned: discussions.pinned, + closed: discussions.closed, + commentCount: discussions.commentCount, + lastActivityAt: discussions.lastActivityAt, + createdAt: discussions.createdAt, + updatedAt: discussions.updatedAt, + author: { + id: users.id, + username: users.username, + avatarUrl: users.avatarUrl, + }, + }) + .from(discussions) + .innerJoin(users, eq(discussions.authorId, users.id)) + .where(and(...conditions)) + .orderBy( + desc(discussions.pinned), + sort === "newest" + ? desc(discussions.createdAt) + : desc(discussions.lastActivityAt), + ) + .limit(pagination.perPage) + .offset(pagination.offset); + + const [countRow] = await db + .select({ total: sql`count(*)::int` }) + .from(discussions) + .where(and(...conditions)); + + return success(rows, paginationMeta(countRow?.total ?? 0, pagination)); +}); + +export const POST: APIRoute = withErrorHandler(async ({ request, params }) => { + const { owner: ownerName, repo: repoName } = params; + + // 1. Authenticate + const tokenPayload = await getUserFromRequest(request); + if (!tokenPayload) { + return unauthorized("You must be logged in to create a discussion"); + } + const userId = tokenPayload.userId; + + // 2. Parse body + let raw: unknown; + try { + raw = await request.json(); + } catch { + return badRequest("Invalid JSON body"); + } + const result = createDiscussionSchema.safeParse(raw); + if (!result.success) { + return badRequest("Invalid input", result.error.flatten()); + } + const { title, body, category } = result.data; + + const db = getDatabase() as unknown as NodePgDatabase; + + // 3. Resolve repository + const repo = await resolveRepo(db, ownerName!, repoName!); + if (!repo) return notFound("Repository not found"); + + // GitHub model: anyone who can read the repository can start a discussion + const hasPermission = await canReadRepo(userId, repo as any); + if (!hasPermission) return notFound("Repository not found"); + + // 4. Create discussion + const discussionId = generateId("discussion"); + const createdAt = new Date(); + + const newDiscussion = { + id: discussionId, + repositoryId: repo.id, + authorId: userId, + title, + body, + category, + pinned: false, + closed: false, + commentCount: 0, + lastActivityAt: createdAt, + createdAt, + updatedAt: createdAt, + }; + + await db.insert(discussions).values(newDiscussion); + + logger.info( + { userId, repoId: repo.id, discussionId }, + "Discussion created", + ); + + return created({ + ...newDiscussion, + author: { id: userId, username: tokenPayload.username }, + url: `/${ownerName}/${repoName}/discussions/${discussionId}`, + }); +}); diff --git a/src/pages/api/repos/[owner]/[repo]/releases/index.ts b/src/pages/api/repos/[owner]/[repo]/releases/index.ts index dcd756de..fd9c7936 100644 --- a/src/pages/api/repos/[owner]/[repo]/releases/index.ts +++ b/src/pages/api/repos/[owner]/[repo]/releases/index.ts @@ -16,6 +16,7 @@ const createReleaseSchema = z.object({ body: z.string().optional(), isDraft: z.boolean().optional().default(false), isPrerelease: z.boolean().optional().default(false), + targetCommitish: z.string().min(1).optional(), }); export const GET: APIRoute = withErrorHandler(async ({ params, request }) => { @@ -101,16 +102,78 @@ export const POST: APIRoute = withErrorHandler(async ({ params, request }) => { body: releaseBody, isDraft, isPrerelease, + targetCommitish, } = parsed.data; // Find tag if exists - const tag = await db.query.tags?.findFirst?.({ + let tag = await db.query.tags?.findFirst?.({ where: and( eq(schema.tags.repositoryId, repoData.id), eq(schema.tags.name, tagName), ), }); + // Ensure the git tag exists — create it on the target commitish if missing. + // All git work happens before any DB writes so a failure doesn't half-create. + const { simpleGit } = await import("simple-git"); + const { acquireRepo, releaseRepo } = await import("@/lib/git-storage"); + + let tagSha: string | null = null; + try { + const repoPath = await acquireRepo(owner, repo); + const git = simpleGit(repoPath); + + // Resolve the target commit: optional commitish or the repo default branch HEAD + const targetRef = targetCommitish || repoData.defaultBranch || "main"; + let targetSha: string; + try { + targetSha = (await git.revparse([targetRef])).trim(); + } catch { + return badRequest( + `Target "${targetRef}" not found in repository. Provide a valid branch, tag or commit SHA.`, + ); + } + + // Idempotency check: skip creation if the tag already exists + let tagExists = false; + try { + await git.raw(["rev-parse", "--verify", "--quiet", `refs/tags/${tagName}`]); + tagExists = true; + } catch { + tagExists = false; + } + + if (!tagExists) { + const message = releaseBody?.trim() || name; + await git.tag(["-a", tagName, "-m", message, targetSha]); + } + + // Resolve the commit the tag points at (annotated tags point to a tag object) + tagSha = (await git.revparse([`${tagName}^{commit}`])).trim(); + + // Tag was created locally — sync back to storage (no-op for local storage) + await releaseRepo(owner, repo, true); + } catch (err) { + return badRequest( + `Failed to create tag "${tagName}": ${err instanceof Error ? err.message : String(err)}`, + ); + } + + // Link or create the DB tag row + if (!tag && tagSha) { + const newTagId = crypto.randomUUID(); + await (db as any).insert(schema.tags).values({ + id: newTagId, + repositoryId: repoData.id, + name: tagName, + commitSha: tagSha, + message: releaseBody?.trim() || name, + isRelease: true, + taggedAt: new Date(), + }); + tag = { id: newTagId } as typeof schema.tags.$inferSelect; + } + const releaseId = crypto.randomUUID(); await (db as any).insert(schema.releases).values({ id: releaseId, diff --git a/src/pages/api/repos/[owner]/[repo]/settings/mirror.ts b/src/pages/api/repos/[owner]/[repo]/settings/mirror.ts index 1d35fa63..e8592255 100644 --- a/src/pages/api/repos/[owner]/[repo]/settings/mirror.ts +++ b/src/pages/api/repos/[owner]/[repo]/settings/mirror.ts @@ -8,11 +8,23 @@ import { canAdminRepo, canReadRepo } from "@/lib/permissions"; import { badRequest, forbidden, notFound, parseBody, success, unauthorized } from "@/lib/api"; import { withErrorHandler } from "@/lib/errors"; import { disableMirror, initializeMirror } from "@/lib/mirror-sync"; +import { configurePushMirror, removePushMirror } from "@/lib/push-mirror"; -const configureMirrorSchema = z.object({ - mirrorUrl: z.string().url(), +const pushMirrorSchema = z.object({ + enabled: z.boolean(), + url: z.string().url().optional(), + authToken: z.string().optional(), }); +const configureMirrorSchema = z + .object({ + mirrorUrl: z.string().url().optional(), + push: pushMirrorSchema.optional(), + }) + .refine((data) => data.mirrorUrl !== undefined || data.push !== undefined, { + message: "Provide mirrorUrl and/or push configuration", + }); + async function resolveRepository(owner: string, repoName: string) { const db = getDatabase() as NodePgDatabase; const ownerUser = await db.query.users.findFirst({ @@ -78,6 +90,13 @@ export const GET: APIRoute = withErrorHandler(async ({ params, request }) => { mirrorSyncStatus: repository.mirrorSyncStatus, lastMirrorSyncAt: repository.lastMirrorSyncAt, ...health, + push: { + enabled: repository.pushMirrorEnabled, + url: repository.pushMirrorUrl, + status: repository.pushMirrorStatus, + lastPushMirrorAt: repository.lastPushMirrorAt, + hasToken: repository.pushMirrorToken !== null && repository.pushMirrorToken !== undefined, + }, }); }); @@ -99,15 +118,37 @@ export const POST: APIRoute = withErrorHandler(async ({ params, request }) => { const parsed = await parseBody(request, configureMirrorSchema); if ("error" in parsed) return parsed.error; - const result = await initializeMirror(repository.id, parsed.data.mirrorUrl); - if (!result.success) { - return badRequest(result.error || "Failed to initialize mirror"); + const payload: Record = {}; + + if (parsed.data.mirrorUrl !== undefined) { + const result = await initializeMirror(repository.id, parsed.data.mirrorUrl); + if (!result.success) { + return badRequest(result.error || "Failed to initialize mirror"); + } + payload.configured = true; + payload.refsUpdated = result.refsUpdated; } - return success({ - configured: true, - refsUpdated: result.refsUpdated, - }); + if (parsed.data.push) { + const { enabled, url, authToken } = parsed.data.push; + if (enabled && url) { + const result = await configurePushMirror(repository.id, { url, authToken }); + if (!result.success) { + return badRequest(result.error || "Failed to configure push mirror"); + } + payload.push = { configured: true }; + } else if (!enabled) { + const result = await removePushMirror(repository.id); + if (!result.success) { + return badRequest(result.error || "Failed to remove push mirror"); + } + payload.push = { configured: false }; + } else { + return badRequest("push.url is required when enabling push mirroring"); + } + } + + return success(payload); }); export const DELETE: APIRoute = withErrorHandler(async ({ params, request }) => { @@ -130,5 +171,8 @@ export const DELETE: APIRoute = withErrorHandler(async ({ params, request }) => return badRequest(result.error || "Failed to disable mirror"); } - return success({ configured: false }); + // Also remove any push mirror configuration (best-effort). + await removePushMirror(repository.id); + + return success({ configured: false, push: { configured: false } }); }); diff --git a/src/pages/api/repos/[owner]/[repo]/settings/mirror/sync.ts b/src/pages/api/repos/[owner]/[repo]/settings/mirror/sync.ts index dc5bacb6..1892f25c 100644 --- a/src/pages/api/repos/[owner]/[repo]/settings/mirror/sync.ts +++ b/src/pages/api/repos/[owner]/[repo]/settings/mirror/sync.ts @@ -7,6 +7,7 @@ import { canWriteRepo } from "@/lib/permissions"; import { badRequest, forbidden, notFound, success, unauthorized } from "@/lib/api"; import { withErrorHandler } from "@/lib/errors"; import { syncMirrorRepository } from "@/lib/mirror-sync"; +import { pushMirrorNow } from "@/lib/push-mirror"; async function resolveRepository(owner: string, repoName: string) { const db = getDatabase() as NodePgDatabase; @@ -37,11 +38,33 @@ export const POST: APIRoute = withErrorHandler(async ({ params, request }) => { return forbidden(); } + const direction = new URL(request.url).searchParams.get("direction") ?? "pull"; + if (!["pull", "push", "both"].includes(direction)) { + return badRequest("direction must be one of: pull, push, both"); + } + + if (direction === "push") { + const pushResult = await pushMirrorNow(repository.id); + if (!pushResult.success) { + return badRequest(pushResult.error || "Push mirror failed"); + } + return success({ direction, ...pushResult }); + } + const result = await syncMirrorRepository(repository.id); if (!result.success) { return badRequest(result.error || "Mirror sync failed"); } + if (direction === "both") { + const pushResult = await pushMirrorNow(repository.id); + return success({ + direction, + pull: result, + push: pushResult, + }); + } + return success(result); }); diff --git a/src/pages/gists/[id]/index.astro b/src/pages/gists/[id]/index.astro new file mode 100644 index 00000000..167db347 --- /dev/null +++ b/src/pages/gists/[id]/index.astro @@ -0,0 +1,297 @@ +--- +import BaseLayout from "@/layouts/BaseLayout.astro"; +import { getDatabase } from "@/db"; +import { gists } from "@/db/schema/gists"; +import { Code, Globe, Lock, Pencil, Plus, Trash2, X } from "lucide-react"; +import { eq } from "drizzle-orm"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; + +const user = Astro.locals.user; +const gistId = Astro.params.id ?? ""; + +const db = getDatabase() as unknown as NodePgDatabase; +const rows = await db + .select() + .from(gists) + .where(eq(gists.id, gistId)) + .limit(1); +const found = rows[0] ?? null; + +const isOwner = !!found && !!user && user.id === found.userId; + +// Secret gists are only visible to their owner +if (!found || (!found.public && !isOwner)) { + return Astro.redirect("/gists"); +} + +const files = Array.isArray(found.files) ? found.files : []; +const editMode = Astro.url.searchParams.get("edit") === "1" && isOwner; +--- + + +
+
+
+ {found.description ? ( +

{found.description}

+ ) : ( +

Untitled gist

+ )} +
+ + {found.public ? : } + {found.public ? "Public" : "Secret"} + + Updated {new Date(found.updatedAt).toLocaleString()} +
+
+ + {isOwner && !editMode && ( + + )} +
+ + + {editMode ? ( +
+
+

Edit gist

+ + + +
+
+
+ + +
+ + + +
+
+ Files + +
+
+
+ diff --git a/src/pages/gists/index.astro b/src/pages/gists/index.astro new file mode 100644 index 00000000..766ac49c --- /dev/null +++ b/src/pages/gists/index.astro @@ -0,0 +1,297 @@ +--- +import BaseLayout from "@/layouts/BaseLayout.astro"; +import { getDatabase } from "@/db"; +import { gists } from "@/db/schema/gists"; +import { Code, Globe, Lock, Plus, X } from "lucide-react"; +import { desc, eq } from "drizzle-orm"; +import type { NodePgDatabase } from "drizzle-orm/node-postgres"; + +const user = Astro.locals.user; +if (!user) { + return Astro.redirect("/login"); +} + +const db = getDatabase() as unknown as NodePgDatabase; + +// Only the caller's own gists are listed here (public + secret) +let gistsList: any[] = []; +try { + gistsList = await db + .select() + .from(gists) + .where(eq(gists.userId, user.id)) + .orderBy(desc(gists.updatedAt)) + .limit(100); +} catch (e) { + console.error("Failed to fetch gists:", e); +} + +function gistTitle(gist: any): string { + if (gist.description && gist.description.trim().length > 0) { + return gist.description; + } + const first = Array.isArray(gist.files) ? gist.files[0] : null; + return first?.filename || "Untitled gist"; +} + +function fileNames(gist: any): string[] { + return Array.isArray(gist.files) ? gist.files.map((f: any) => f.filename) : []; +} + +function timeAgo(date: Date | string) { + const seconds = Math.floor((Date.now() - new Date(date).getTime()) / 1000); + if (seconds < 60) return "just now"; + if (seconds < 3600) return Math.floor(seconds / 60) + "m ago"; + if (seconds < 86400) return Math.floor(seconds / 3600) + "h ago"; + if (seconds < 604800) return Math.floor(seconds / 86400) + "d ago"; + return new Date(date).toLocaleDateString(); +} +--- + + +
+ +
+
+

+ + Your gists +

+

Standalone code snippets, quick notes, and scripts

+
+ +
+ + + + + + +
+
+ + diff --git a/src/pages/inbox.astro b/src/pages/inbox.astro index cc1e38f8..6cc30e8c 100644 --- a/src/pages/inbox.astro +++ b/src/pages/inbox.astro @@ -121,7 +121,7 @@ const prsWithStacks = await Promise.all( if (stackEntry && stackEntry.stack) { stackInfo = { name: stackEntry.stack.name, - position: stackEntry.position + 1, + position: stackEntry.stackOrder + 1, count: stackEntry.stack.entries.length, }; } diff --git a/src/pages/merge-queue.astro b/src/pages/merge-queue.astro index 66fed98b..0876e112 100644 --- a/src/pages/merge-queue.astro +++ b/src/pages/merge-queue.astro @@ -62,7 +62,7 @@ interface QueueItemWithDetails { } | null; stackInfo?: { id: string; - name: string; + name: string | null; position: number; total: number; }; @@ -116,7 +116,7 @@ try { stackInfo = { id: stackEntry.stack.id, name: stackEntry.stack.name, - position: stackEntry.position + 1, + position: stackEntry.stackOrder + 1, total: stackEntry.stack.entries.length, }; } @@ -130,9 +130,9 @@ try { id: item.id, position: currentPosition, status: item.status, - priority: item.priority, + priority: item.priority ?? 0, mergeMethod: item.mergeMethod || "merge", - addedAt: item.createdAt ? new Date(item.createdAt).toISOString() : new Date().toISOString(), + addedAt: item.addedAt ? new Date(item.addedAt).toISOString() : new Date().toISOString(), pr: item.pullRequest ? { number: item.pullRequest.number, diff --git a/tests/integration/discussions-route.test.ts b/tests/integration/discussions-route.test.ts new file mode 100644 index 00000000..39f5b080 --- /dev/null +++ b/tests/integration/discussions-route.test.ts @@ -0,0 +1,527 @@ +/** + * Integration tests for Discussions routes: + * GET/POST /api/repos/[owner]/[repo]/discussions + * GET/PATCH/DELETE /api/repos/[owner]/[repo]/discussions/[id] + * GET/POST /api/repos/[owner]/[repo]/discussions/[id]/comments + */ +import { beforeEach, describe, expect, it, vi } from "vitest"; + +/* ─── hoisted mocks ─── */ +const mocks = vi.hoisted(() => ({ + getUserFromRequestMock: vi.fn().mockResolvedValue({ + userId: "usr_1", + username: "alice", + email: "alice@example.com", + }), + canReadRepoMock: vi.fn().mockResolvedValue(true), + canWriteRepoMock: vi.fn().mockResolvedValue(false), + canAdminRepoMock: vi.fn().mockResolvedValue(false), + generateIdMock: vi.fn(), +})); + +vi.mock("@/db", () => ({ + getDatabase: () => mockDb, +})); + +vi.mock("@/lib/auth", () => ({ + getUserFromRequest: mocks.getUserFromRequestMock, +})); + +vi.mock("@/lib/permissions", () => ({ + canReadRepo: mocks.canReadRepoMock, + canWriteRepo: mocks.canWriteRepoMock, + canAdminRepo: mocks.canAdminRepoMock, +})); + +vi.mock("@/lib/errors", () => ({ + withErrorHandler: (fn: any) => fn, +})); + +vi.mock("@/lib/logger", () => ({ + logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() }, +})); + +vi.mock("@/lib/utils", () => ({ + generateId: mocks.generateIdMock, + now: () => new Date().toISOString(), +})); + +/* ─── fixtures ─── */ +import { discussions, discussionComments } from "@/db/schema/discussions"; +import { users } from "@/db/schema/users"; +import { repositories } from "@/db/schema/repositories"; + +const ownerRow = { id: "usr_owner", username: "owner", avatarUrl: null }; +const repoRow = { + id: "repo_1", + name: "my-repo", + ownerId: "usr_owner", + visibility: "public", +}; + +const discussionRow = { + id: "discussion_1", + repositoryId: "repo_1", + authorId: "usr_1", + title: "Hello world", + body: "First discussion", + category: "General", + pinned: false, + closed: false, + commentCount: 0, + lastActivityAt: new Date(), + createdAt: new Date(), + updatedAt: new Date(), +}; + +/* ─── fake db ─── */ +function thenable(result: unknown) { + const p = Promise.resolve(result); + const step = () => api; + const api: any = { + where: step, + orderBy: step, + limit: step, + offset: step, + innerJoin: step, + leftJoin: step, + then: p.then.bind(p), + catch: p.catch.bind(p), + finally: p.finally.bind(p), + }; + return api; +} + +function makeDb( + opts: { + users?: any[]; + repositories?: any[]; + discussions?: any[]; + comments?: any[]; + } = {}, +) { + const rowsByTable = new Map([ + [users, opts.users ?? [ownerRow]], + [repositories, opts.repositories ?? [repoRow]], + [discussions, opts.discussions ?? [discussionRow]], + [discussionComments, opts.comments ?? []], + ]); + + // Count queries project a single { total } field — resolve a count row + const select = vi.fn((fields?: any) => { + const isCount = + fields && + typeof fields === "object" && + Object.keys(fields).length === 1 && + "total" in fields; + return { + from: (table: any) => + thenable( + isCount + ? [{ total: (rowsByTable.get(table) ?? []).length }] + : (rowsByTable.get(table) ?? []), + ), + }; + }); + + const insertValues = vi.fn().mockResolvedValue([]); + const insert = vi.fn(() => ({ values: insertValues })); + + const updateSet = vi.fn().mockReturnValue({ + where: vi.fn().mockResolvedValue([]), + }); + const update = vi.fn(() => ({ set: updateSet })); + + const deleteWhere = vi.fn().mockResolvedValue([]); + const del = vi.fn(() => ({ where: deleteWhere })); + + // Transactions execute inline against dedicated mock writers + const txSet = vi.fn().mockReturnValue({ + where: vi.fn().mockResolvedValue([]), + }); + const txUpdate = vi.fn(() => ({ set: txSet })); + const transaction = vi.fn(async (cb: any) => + cb({ insert, update: txUpdate }), + ); + + return { + select, + insert, + insertValues, + update, + updateSet, + delete: del, + deleteWhere, + transaction, + txSet, + }; +} + +let mockDb: ReturnType; +const readJson = (r: Response) => r.json(); + +/* ─── import routes ─── */ +import { + GET as listDiscussions, + POST as createDiscussion, +} from "@/pages/api/repos/[owner]/[repo]/discussions/index"; +import { + DELETE as deleteDiscussion, + GET as getDiscussion, + PATCH as patchDiscussion, +} from "@/pages/api/repos/[owner]/[repo]/discussions/[id]/index"; +import { POST as createComment } from "@/pages/api/repos/[owner]/[repo]/discussions/[id]/comments"; + +const baseCtx = (request: Request, params: Record = {}) => ({ + params: { + owner: "owner", + repo: "my-repo", + id: "discussion_1", + ...params, + }, + request, +}); + +const jsonRequest = (body: unknown, method = "POST") => + new Request( + "http://localhost/api/repos/owner/my-repo/discussions/discussion_1", + { + method, + headers: { "Content-Type": "application/json" }, + body: JSON.stringify(body), + }, + ); + +beforeEach(() => { + vi.clearAllMocks(); + mockDb = makeDb(); + mocks.generateIdMock.mockImplementation( + (prefix?: string) => `${prefix ?? "id"}_new`, + ); + mocks.getUserFromRequestMock.mockResolvedValue({ + userId: "usr_1", + username: "alice", + email: "alice@example.com", + }); + mocks.canReadRepoMock.mockResolvedValue(true); + mocks.canWriteRepoMock.mockResolvedValue(false); + mocks.canAdminRepoMock.mockResolvedValue(false); +}); + +/* ─── POST /discussions (create) ─── */ +describe("POST /api/repos/[owner]/[repo]/discussions", () => { + it("returns 201 and creates a discussion with default category", async () => { + const res = await createDiscussion( + baseCtx( + new Request("http://localhost/api/repos/owner/my-repo/discussions", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ title: "Hi", body: "Content" }), + }), + ) as any, + ); + expect(res.status).toBe(201); + const json = await readJson(res); + expect(json.data.title).toBe("Hi"); + expect(json.data.id).toBe("discussion_new"); + expect(json.data.category).toBe("General"); + expect(mocks.canReadRepoMock).toHaveBeenCalled(); + }); + + it("returns 401 when not authenticated", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await createDiscussion( + baseCtx(jsonRequest({ title: "Hi", body: "Content" })) as any, + ); + expect(res.status).toBe(401); + }); + + it("returns 404 when user has no read permission (GitHub model)", async () => { + mocks.canReadRepoMock.mockResolvedValue(false); + const res = await createDiscussion( + baseCtx(jsonRequest({ title: "Hi", body: "Content" })) as any, + ); + expect(res.status).toBe(404); + }); + + it("returns 400 on invalid category or missing title", async () => { + const badCategory = await createDiscussion( + baseCtx( + jsonRequest({ title: "Hi", body: "Content", category: "Memes" }), + ) as any, + ); + expect(badCategory.status).toBe(400); + + const noTitle = await createDiscussion( + baseCtx(jsonRequest({ body: "Content" })) as any, + ); + expect(noTitle.status).toBe(400); + }); + + it("returns 404 when repository does not exist", async () => { + mockDb = makeDb({ repositories: [] }); + const res = await createDiscussion( + baseCtx(jsonRequest({ title: "Hi", body: "Content" })) as any, + ); + expect(res.status).toBe(404); + }); +}); + +/* ─── GET /discussions (list) ─── */ +describe("GET /api/repos/[owner]/[repo]/discussions", () => { + it("returns 200 with discussions and pagination meta", async () => { + const res = await listDiscussions( + baseCtx( + new Request("http://localhost/api/repos/owner/my-repo/discussions"), + ) as any, + ); + expect(res.status).toBe(200); + const json = await readJson(res); + expect(Array.isArray(json.data)).toBe(true); + expect(json.data.length).toBe(1); + expect(json.meta.total).toBe(1); + }); + + it("allows anonymous listing of public repos", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await listDiscussions( + baseCtx( + new Request("http://localhost/api/repos/owner/my-repo/discussions"), + ) as any, + ); + expect(res.status).toBe(200); + }); + + it("hides the repo from anonymous users without read access", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + mocks.canReadRepoMock.mockResolvedValue(false); + const res = await listDiscussions( + baseCtx( + new Request("http://localhost/api/repos/owner/my-repo/discussions"), + ) as any, + ); + expect(res.status).toBe(404); + }); + + it("rejects invalid filters with 400", async () => { + const res = await listDiscussions( + baseCtx( + new Request( + "http://localhost/api/repos/owner/my-repo/discussions?closed=maybe", + ), + ) as any, + ); + expect(res.status).toBe(400); + }); +}); + +/* ─── GET /discussions/[id] ─── */ +describe("GET /api/repos/[owner]/[repo]/discussions/[id]", () => { + it("returns 200 with discussion, author and comments", async () => { + const res = await getDiscussion( + baseCtx( + new Request( + "http://localhost/api/repos/owner/my-repo/discussions/discussion_1", + ), + ) as any, + ); + expect(res.status).toBe(200); + const json = await readJson(res); + expect(json.data.id).toBe("discussion_1"); + expect(json.data.author.username).toBeDefined(); + expect(Array.isArray(json.data.comments)).toBe(true); + }); +}); + +/* ─── PATCH /discussions/[id] — permission matrix ─── */ +describe("PATCH /api/repos/[owner]/[repo]/discussions/[id]", () => { + it("author can edit even without repo write access", async () => { + const res = await patchDiscussion( + baseCtx(jsonRequest({ title: "Updated" }, "PATCH")) as any, + ); + expect(res.status).toBe(200); + const json = await readJson(res); + expect(json.data.title).toBe("Hello world"); // re-select resolves fixture + expect(mockDb.updateSet).toHaveBeenCalled(); + }); + + it("non-author without write access gets 403", async () => { + mockDb = makeDb({ + discussions: [{ ...discussionRow, authorId: "usr_other" }], + }); + const res = await patchDiscussion( + baseCtx(jsonRequest({ title: "Hacked" }, "PATCH")) as any, + ); + expect(res.status).toBe(403); + }); + + it("non-author with repo write access can edit (close)", async () => { + mocks.canWriteRepoMock.mockResolvedValue(true); + const db = makeDb({ + discussions: [{ ...discussionRow, authorId: "usr_other" }], + }); + mockDb = db; + const res = await patchDiscussion( + baseCtx(jsonRequest({ closed: true }, "PATCH")) as any, + ); + expect(res.status).toBe(200); + expect(db.updateSet).toHaveBeenCalledWith( + expect.objectContaining({ closed: true }), + ); + }); + + it("supports pin/unpin via booleans", async () => { + const res = await patchDiscussion( + baseCtx(jsonRequest({ pinned: true }, "PATCH")) as any, + ); + expect(res.status).toBe(200); + expect(mockDb.updateSet).toHaveBeenCalledWith( + expect.objectContaining({ pinned: true }), + ); + }); + + it("returns 400 for invalid payloads", async () => { + const res = await patchDiscussion( + baseCtx(jsonRequest({ closed: "yes" }, "PATCH")) as any, + ); + expect(res.status).toBe(400); + }); + + it("returns 401 when not authenticated", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await patchDiscussion( + baseCtx(jsonRequest({ closed: true }, "PATCH")) as any, + ); + expect(res.status).toBe(401); + }); + + it("returns 404 when discussion is missing", async () => { + mockDb = makeDb({ discussions: [] }); + const res = await patchDiscussion( + baseCtx(jsonRequest({ closed: true }, "PATCH")) as any, + ); + expect(res.status).toBe(404); + }); +}); + +/* ─── DELETE /discussions/[id] — permission matrix ─── */ +describe("DELETE /api/repos/[owner]/[repo]/discussions/[id]", () => { + it("author can delete even without admin rights", async () => { + const res = await deleteDiscussion( + baseCtx( + new Request( + "http://localhost/api/repos/owner/my-repo/discussions/discussion_1", + { method: "DELETE" }, + ), + ) as any, + ); + expect(res.status).toBe(204); + }); + + it("non-author without admin rights gets 403", async () => { + mocks.canAdminRepoMock.mockResolvedValue(false); + mockDb = makeDb({ + discussions: [{ ...discussionRow, authorId: "usr_other" }], + }); + const res = await deleteDiscussion( + baseCtx( + new Request( + "http://localhost/api/repos/owner/my-repo/discussions/discussion_1", + { method: "DELETE" }, + ), + ) as any, + ); + expect(res.status).toBe(403); + }); + + it("repo admin can delete someone else's discussion", async () => { + mocks.canAdminRepoMock.mockResolvedValue(true); + mockDb = makeDb({ + discussions: [{ ...discussionRow, authorId: "usr_other" }], + }); + const res = await deleteDiscussion( + baseCtx( + new Request( + "http://localhost/api/repos/owner/my-repo/discussions/discussion_1", + { method: "DELETE" }, + ), + ) as any, + ); + expect(res.status).toBe(204); + }); + + it("returns 401 when not authenticated", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await deleteDiscussion( + baseCtx( + new Request( + "http://localhost/api/repos/owner/my-repo/discussions/discussion_1", + { method: "DELETE" }, + ), + ) as any, + ); + expect(res.status).toBe(401); + }); +}); + +/* ─── POST /discussions/[id]/comments ─── */ +describe("POST /api/repos/[owner]/[repo]/discussions/[id]/comments", () => { + it("returns 201 and updates counters transactionally", async () => { + const res = await createComment( + baseCtx( + jsonRequest({ body: "A thoughtful reply" }), + { id: "discussion_1" }, + ) as any, + ); + + expect(res.status).toBe(201); + const json = await readJson(res); + expect(json.data.id).toBe("dcomment_new"); + expect(json.data.body).toBe("A thoughtful reply"); + + // insert + counter update happen inside a single transaction + expect(mockDb.transaction).toHaveBeenCalledTimes(1); + expect(mockDb.insertValues).toHaveBeenCalledWith( + expect.objectContaining({ + discussionId: "discussion_1", + authorId: "usr_1", + }), + ); + expect(mockDb.txSet).toHaveBeenCalledTimes(1); + const setArg = mockDb.txSet.mock.calls[0][0]; + expect(setArg.commentCount).toBeDefined(); // sql`comment_count + 1` + expect(setArg.lastActivityAt).toBeInstanceOf(Date); + }); + + it("returns 401 when not authenticated", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await createComment( + baseCtx(jsonRequest({ body: "hi" }), { id: "discussion_1" }) as any, + ); + expect(res.status).toBe(401); + }); + + it("returns 404 without read permission on the repository", async () => { + mocks.canReadRepoMock.mockResolvedValue(false); + const res = await createComment( + baseCtx(jsonRequest({ body: "hi" }), { id: "discussion_1" }) as any, + ); + expect(res.status).toBe(404); + }); + + it("returns 404 when discussion does not exist in this repo", async () => { + mockDb = makeDb({ discussions: [] }); + const res = await createComment( + baseCtx(jsonRequest({ body: "hi" }), { id: "missing" }) as any, + ); + expect(res.status).toBe(404); + }); + + it("returns 400 for empty comment bodies", async () => { + const res = await createComment( + baseCtx(jsonRequest({ body: "" }), { id: "discussion_1" }) as any, + ); + expect(res.status).toBe(400); + }); +}); + diff --git a/tests/integration/gists-route.test.ts b/tests/integration/gists-route.test.ts new file mode 100644 index 00000000..b1b757fe --- /dev/null +++ b/tests/integration/gists-route.test.ts @@ -0,0 +1,458 @@ +/** + * Integration tests for Gists routes: + * GET/POST /api/gists + * GET/PATCH/DELETE /api/gists/[id] + * GET /api/gists/[id]/raw/[file] + */ +import { beforeEach, describe, expect, it, vi } from "vitest"; + +/* ─── hoisted mocks ─── */ +const mocks = vi.hoisted(() => ({ + getUserFromRequestMock: vi.fn().mockResolvedValue({ + userId: "usr_1", + username: "alice", + email: "alice@example.com", + }), + generateIdMock: vi.fn(), +})); + +vi.mock("@/db", () => ({ + getDatabase: () => mockDb, +})); + +vi.mock("@/lib/auth", () => ({ + getUserFromRequest: mocks.getUserFromRequestMock, +})); + +vi.mock("@/lib/errors", () => ({ + withErrorHandler: (fn: any) => fn, +})); + +vi.mock("@/lib/logger", () => ({ + logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() }, +})); + +vi.mock("@/lib/utils", () => ({ + generateId: mocks.generateIdMock, +})); + +/* ─── fixtures ─── */ +import { gists } from "@/db/schema/gists"; + +const ownerGist = { + id: "gist_1", + userId: "usr_1", + description: "my secret snippet", + public: false, + files: [{ filename: "a.txt", content: "hello world" }], + createdAt: new Date("2026-01-01"), + updatedAt: new Date("2026-01-02"), +}; + +/* ─── fake db ─── */ +function thenable(result: unknown) { + const p = Promise.resolve(result); + const step = () => api; + const api: any = { + where: step, + orderBy: step, + limit: step, + offset: step, + innerJoin: step, + leftJoin: step, + then: p.then.bind(p), + catch: p.catch.bind(p), + finally: p.finally.bind(p), + }; + return api; +} + +function makeDb(opts: { gists?: any[] } = {}) { + const rowsByTable = new Map([[gists, opts.gists ?? [ownerGist]]]); + + const state = { + inserted: null as any, + updates: [] as any[], + deletedWhere: null as any, + /** Result returned by query.gists.findFirst */ + found: ownerGist as any, + }; + + const select = vi.fn((fields?: any) => { + const isCount = + fields && + typeof fields === "object" && + Object.keys(fields).length === 1 && + "total" in fields; + return { + from: (table: any) => + thenable( + isCount + ? [{ total: (rowsByTable.get(table) ?? []).length }] + : (rowsByTable.get(table) ?? []), + ), + }; + }); + + const insertValues = vi.fn((values: any) => { + state.inserted = values; + return Promise.resolve([]); + }); + const insert = vi.fn(() => ({ values: insertValues })); + + const updateSet = vi.fn((values: any) => { + state.updates.push(values); + return { + where: vi.fn().mockResolvedValue([]), + }; + }); + const update = vi.fn(() => ({ set: updateSet })); + + const deleteWhere = vi.fn((where: any) => { + state.deletedWhere = where; + return Promise.resolve([]); + }); + const del = vi.fn(() => ({ where: deleteWhere })); + + const query = { + gists: { + findFirst: vi.fn(async () => state.found), + }, + }; + + return { + select, + insert, + insertValues, + update, + updateSet, + delete: del, + deleteWhere, + query, + state, + }; +} + +let mockDb: ReturnType; + +/* ─── import routes ─── */ +import { GET as listGists, POST as createGist } from "@/pages/api/gists/index"; +import { + DELETE as deleteGist, + GET as getGist, + PATCH as patchGist, +} from "@/pages/api/gists/[id]/index"; +import { GET as getRawFile } from "@/pages/api/gists/[id]/raw/[file]"; + +const readJson = (r: Response) => r.json(); + +function listCtx(query = "") { + const reqUrl = `http://localhost/api/gists${query}`; + return { request: new Request(reqUrl), url: new URL(reqUrl) } as any; +} + +function idCtx( + request: Request, + params: Record = {}, +) { + return { + request, + url: new URL(request.url), + params: { id: "gist_1", ...params }, + } as any; +} + +const gistRequest = ( + body: unknown, + method = "POST", + path = "/api/gists", +) => + new Request(`http://localhost${path}`, { + method, + headers: { "Content-Type": "application/json" }, + ...(body !== undefined ? { body: JSON.stringify(body) } : {}), + }); + +beforeEach(() => { + vi.clearAllMocks(); + mockDb = makeDb(); + mocks.generateIdMock.mockImplementation( + (prefix?: string) => `${prefix ?? "id"}_new`, + ); + mocks.getUserFromRequestMock.mockResolvedValue({ + userId: "usr_1", + username: "alice", + email: "alice@example.com", + }); +}); + +/* ─── POST /api/gists (create) ─── */ +describe("POST /api/gists", () => { + it("returns 401 when not authenticated", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await createGist({ + request: gistRequest({ files: [{ filename: "a.txt", content: "x" }] }), + url: new URL("http://localhost/api/gists"), + } as any); + expect(res.status).toBe(401); + }); + + it("returns 201 and creates a gist", async () => { + const res = await createGist({ + request: gistRequest({ + description: "demo", + public: true, + files: [{ filename: "a.txt", content: "hello" }], + }), + url: new URL("http://localhost/api/gists"), + } as any); + expect(res.status).toBe(201); + const json = await readJson(res); + expect(json.data.id).toBe("gist_new"); + expect(json.data.description).toBe("demo"); + expect(json.data.public).toBe(true); + expect(json.data.fileCount).toBe(1); + expect(mockDb.state.inserted.userId).toBe("usr_1"); + expect(mocks.generateIdMock).toHaveBeenCalledWith("gist"); + }); + + it("defaults description to '' and visibility to secret", async () => { + const res = await createGist({ + request: gistRequest({ files: [{ filename: "a.txt", content: "x" }] }), + url: new URL("http://localhost/api/gists"), + } as any); + const json = await readJson(res); + expect(json.data.description).toBe(""); + expect(json.data.public).toBe(false); + }); + + it("returns 400 for path-traversal filenames", async () => { + for (const filename of ["../escape.txt", "/etc/passwd", "a/b.txt"]) { + const res = await createGist({ + request: gistRequest({ files: [{ filename, content: "x" }] }), + url: new URL("http://localhost/api/gists"), + } as any); + expect(res.status).toBe(400); + const json = await readJson(res); + expect(json.error.code).toBe("BAD_REQUEST"); + } + }); + + it("returns 400 when files array is empty or exceeds 10 files", async () => { + const empty = await createGist({ + request: gistRequest({ files: [] }), + url: new URL("http://localhost/api/gists"), + } as any); + expect(empty.status).toBe(400); + + const tooMany = Array.from({ length: 11 }, (_, i) => ({ + filename: `f${i}.txt`, + content: "x", + })); + const res = await createGist({ + request: gistRequest({ files: tooMany }), + url: new URL("http://localhost/api/gists"), + } as any); + expect(res.status).toBe(400); + }); + + it("returns 400 when total content exceeds 1MB", async () => { + const half = "a".repeat(512 * 1024); + const res = await createGist({ + request: gistRequest({ + files: [ + { filename: "one.txt", content: half }, + { filename: "two.txt", content: half + "overflow" }, + ], + }), + url: new URL("http://localhost/api/gists"), + } as any); + expect(res.status).toBe(400); + }); +}); + +/* ─── GET /api/gists (list) ─── */ +describe("GET /api/gists", () => { + it("returns 401 when not authenticated", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await listGists(listCtx()); + expect(res.status).toBe(401); + }); + + it("returns 200 with pagination meta for authenticated user", async () => { + const res = await listGists(listCtx()); + expect(res.status).toBe(200); + const json = await readJson(res); + expect(json.success).toBe(true); + expect(Array.isArray(json.data)).toBe(true); + expect(json.data[0].fileCount).toBe(1); + expect(json.meta.page).toBe(1); + expect(json.meta.total).toBe(1); + }); +}); + +/* ─── GET /api/gists/[id] ─── */ +describe("GET /api/gists/[id]", () => { + it("lets the owner read a secret gist", async () => { + const res = await getGist(idCtx(gistRequest(undefined, "GET"))); + expect(res.status).toBe(200); + const json = await readJson(res); + expect(json.data.isOwner).toBe(true); + }); + + it("hides secret gists from non-owners (404)", async () => { + mocks.getUserFromRequestMock.mockResolvedValue({ + userId: "usr_other", + username: "bob", + }); + const res = await getGist(idCtx(gistRequest(undefined, "GET"))); + expect(res.status).toBe(404); + }); + + it("serves public gists to anonymous users without isOwner", async () => { + mockDb.state.found = { ...ownerGist, public: true }; + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await getGist(idCtx(gistRequest(undefined, "GET"))); + expect(res.status).toBe(200); + const json = await readJson(res); + expect(json.data.isOwner).toBe(false); + }); + + it("returns 404 for unknown ids", async () => { + mockDb.state.found = undefined; + const res = await getGist(idCtx(gistRequest(undefined, "GET"))); + expect(res.status).toBe(404); + }); +}); + +/* ─── PATCH /api/gists/[id] ─── */ +describe("PATCH /api/gists/[id]", () => { + it("is owner-only: returns 403 for authenticated non-owner", async () => { + mocks.getUserFromRequestMock.mockResolvedValue({ + userId: "usr_other", + username: "bob", + }); + const res = await patchGist( + idCtx(gistRequest({ description: "hack" }, "PATCH")), + ); + expect(res.status).toBe(403); + expect(mockDb.state.updates).toHaveLength(0); + }); + + it("returns 401 when unauthenticated", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await patchGist( + idCtx(gistRequest({ description: "x" }, "PATCH")), + ); + expect(res.status).toBe(401); + }); + + it("replaces description, visibility and files wholesale for the owner", async () => { + const payload = { + description: "updated", + public: true, + files: [{ filename: "new.txt", content: "new content" }], + }; + const res = await patchGist(idCtx(gistRequest(payload, "PATCH"))); + expect(res.status).toBe(200); + const applied = mockDb.state.updates[0]; + expect(applied.description).toBe("updated"); + expect(applied.public).toBe(true); + expect(applied.files).toEqual(payload.files); + expect(applied.updatedAt).toBeInstanceOf(Date); + }); + + it("returns 400 for invalid replacement files", async () => { + const res = await patchGist( + idCtx(gistRequest({ files: [{ filename: "../x", content: "" }] }, "PATCH")), + ); + expect(res.status).toBe(400); + }); + + it("returns 404 when gist does not exist", async () => { + mockDb.state.found = undefined; + const res = await patchGist( + idCtx(gistRequest({ description: "x" }, "PATCH")), + ); + expect(res.status).toBe(404); + }); +}); + +/* ─── DELETE /api/gists/[id] ─── */ +describe("DELETE /api/gists/[id]", () => { + it("deletes for the owner and returns 204", async () => { + const res = await deleteGist(idCtx(new Request("http://localhost/api/gists/gist_1", { method: "DELETE" }))); + expect(res.status).toBe(204); + expect(mockDb.state.deletedWhere).toBeDefined(); + }); + + it("is owner-only: returns 403 for authenticated non-owner", async () => { + mocks.getUserFromRequestMock.mockResolvedValue({ + userId: "usr_other", + username: "bob", + }); + const res = await deleteGist(idCtx(new Request("http://localhost/api/gists/gist_1", { method: "DELETE" }))); + expect(res.status).toBe(403); + expect(mockDb.state.deletedWhere).toBeNull(); + }); + + it("returns 404 for unknown ids", async () => { + mockDb.state.found = undefined; + const res = await deleteGist(idCtx(new Request("http://localhost/api/gists/gist_1", { method: "DELETE" }))); + expect(res.status).toBe(404); + }); +}); + +/* ─── GET /api/gists/[id]/raw/[file] ─── */ +describe("GET /api/gists/[id]/raw/[file]", () => { + const rawCtx = (file: string, params = {}) => + idCtx(new Request(`http://localhost/api/gists/gist_1/raw/${file}`), { + file, + ...params, + }); + + it("serves file content as text/plain charset=utf-8", async () => { + const res = await getRawFile(rawCtx("a.txt")); + expect(res.status).toBe(200); + expect(res.headers.get("Content-Type")).toBe("text/plain; charset=utf-8"); + expect(await res.text()).toBe("hello world"); + }); + + it("allows anonymous access to public gists", async () => { + mockDb.state.found = { ...ownerGist, public: true }; + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await getRawFile(rawCtx("a.txt")); + expect(res.status).toBe(200); + expect(res.headers.get("Content-Type")).toContain("text/plain"); + }); + + it("requires auth for secret gists (401 anonymous)", async () => { + mocks.getUserFromRequestMock.mockResolvedValue(null); + const res = await getRawFile(rawCtx("a.txt")); + expect(res.status).toBe(401); + }); + + it("hides secret gists from non-owners (404)", async () => { + mocks.getUserFromRequestMock.mockResolvedValue({ + userId: "usr_other", + username: "bob", + }); + const res = await getRawFile(rawCtx("a.txt")); + expect(res.status).toBe(404); + }); + + it("returns 404 for filenames not present in the gist", async () => { + const res = await getRawFile(rawCtx("missing.txt")); + expect(res.status).toBe(404); + }); + + it("decodes URL-encoded filenames", async () => { + mockDb.state.found = { + ...ownerGist, + files: [{ filename: "my file.txt", content: "spaces are ok" }], + }; + const res = await getRawFile(rawCtx("my%20file.txt")); + expect(res.status).toBe(200); + expect(await res.text()).toBe("spaces are ok"); + }); +}); diff --git a/tests/integration/issues-releases-route.test.ts b/tests/integration/issues-releases-route.test.ts index 8ce16a98..68a20be3 100644 --- a/tests/integration/issues-releases-route.test.ts +++ b/tests/integration/issues-releases-route.test.ts @@ -24,6 +24,13 @@ const mocks = vi.hoisted(() => ({ }), }), selectMock: vi.fn(), + acquireRepoMock: vi.fn().mockResolvedValue("/tmp/fake-repo.git"), + releaseRepoMock: vi.fn().mockResolvedValue(undefined), + simpleGitMock: vi.fn(() => ({ + revparse: vi.fn().mockResolvedValue("abc123def456\n"), + tag: vi.fn().mockResolvedValue(undefined), + raw: vi.fn().mockResolvedValue(""), + })), fakeSchema: { users: { username: "username" }, repositories: { @@ -82,6 +89,15 @@ vi.mock("@/lib/logger", () => ({ logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() }, })); +vi.mock("@/lib/git-storage", () => ({ + acquireRepo: mocks.acquireRepoMock, + releaseRepo: mocks.releaseRepoMock, +})); + +vi.mock("simple-git", () => ({ + simpleGit: mocks.simpleGitMock, +})); + vi.mock("@/lib/email", () => ({ sendIssueEmail: vi.fn().mockResolvedValue(undefined), })); diff --git a/tests/integration/realtime-bridge.test.ts b/tests/integration/realtime-bridge.test.ts new file mode 100644 index 00000000..71eec548 --- /dev/null +++ b/tests/integration/realtime-bridge.test.ts @@ -0,0 +1,150 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import Redis from "ioredis"; + +/** + * Cross-process realtime bridge tests. + * + * Two "processes" are simulated with fresh module registries + * (vi.resetModules + dynamic import): each import gets its own connections + * Map and originId, so messages between them genuinely travel through Redis + * pub/sub on the och:realtime channel. + * + * Requires a live Redis (see TEST_REALTIME_REDIS_URL). When unreachable the + * suite skips — the bridge degrades to local-only delivery in that case, + * which is covered by the local-delivery test running without redis. + */ + +const TEST_REDIS_URL = process.env.TEST_REALTIME_REDIS_URL || "redis://127.0.0.1:16379"; + +async function probeRedis(url: string, timeoutMs = 1000): Promise { + try { + const client = new Redis(url, { lazyConnect: true, connectTimeout: timeoutMs }); + await client.connect(); + client.disconnect(); + return true; + } catch { + return false; + } +} + +const redisAvailable = await probeRedis(TEST_REDIS_URL); + +interface ReceivedEvent { + type: string; + timestamp: unknown; + data: unknown; +} + +function makeFakeController(received: ReceivedEvent[]) { + return { + enqueue(chunk: Uint8Array) { + const text = new TextDecoder().decode(chunk); + const match = text.match(/^data: (.*)\n\n$/); + if (match) received.push(JSON.parse(match[1])); + }, + close() {}, + } as unknown as ReadableStreamDefaultController; +} + +/** Drop the initial connection-confirmation event emitted by registerConnection */ +function dataEvents(received: ReceivedEvent[]): ReceivedEvent[] { + return received.filter((e) => !(e.type === "inbox:refresh" && (e.data as { connected?: boolean })?.connected)); +} + +async function loadRealtime() { + return import("@/lib/realtime"); +} + +describe("realtime redis bridge", () => { + let warnCalls = 0; + + beforeEach(() => { + vi.resetModules(); + vi.doMock("@/lib/logger", () => ({ + logger: { + info: vi.fn(), + warn: vi.fn((msg: unknown) => { + if (String(msg).includes("Realtime Redis bridge")) warnCalls++; + }), + error: vi.fn(), + debug: vi.fn(), + }, + })); + warnCalls = 0; + process.env.NODE_ENV = "development"; + process.env.REDIS_URL = TEST_REDIS_URL; + }); + + afterEach(() => { + vi.doUnmock("@/lib/logger"); + vi.resetModules(); + }); + + it("delivers locally exactly once when broadcasting (no self-loop double delivery)", async () => { + const rt = await loadRealtime(); + const received: ReceivedEvent[] = []; + rt.registerConnection("user-1", makeFakeController(received), ["repo-1"]); + // allow the fire-and-forget subscription to settle before publishing + await new Promise((r) => setTimeout(r, 300)); + + const sent = rt.broadcastToRepository("repo-1", { + type: "pr:opened", + timestamp: new Date(), + data: { prId: "p1" }, + }); + + expect(sent).toBe(1); + await new Promise((r) => setTimeout(r, 400)); + expect(dataEvents(received)).toHaveLength(1); + expect(dataEvents(received)[0].type).toBe("pr:opened"); + }, 15000); + + it("degrades to local-only without throwing when redis is unreachable", async () => { + process.env.REDIS_URL = "redis://127.0.0.1:59999"; + const rt = await loadRealtime(); + const received: ReceivedEvent[] = []; + rt.registerConnection("user-3", makeFakeController(received), []); + + for (let i = 0; i < 5; i++) { + const sent = rt.broadcastToAll({ + type: "pr:updated", + timestamp: new Date(), + data: { i }, + }); + expect(sent).toBe(1); + } + + await new Promise((r) => setTimeout(r, 500)); + expect(dataEvents(received)).toHaveLength(5); + expect(warnCalls).toBeLessThanOrEqual(1); + }, 15000); + + describe.skipIf(!redisAvailable)("with redis reachable", () => { + it("relays events from a second instance through redis", async () => { + const receiverRt = await loadRealtime(); + const received: ReceivedEvent[] = []; + receiverRt.registerConnection("user-2", makeFakeController(received), ["repo-9"]); + await new Promise((r) => setTimeout(r, 300)); + + // simulate the worker process: fresh registry, no local connections + vi.resetModules(); + const workerRt = await loadRealtime(); + + const publishedAt = new Date("2026-01-02T03:04:05.000Z"); + const published = await workerRt.publishRealtimeEvent( + { kind: "repository", repositoryId: "repo-9" }, + { type: "queue:position_changed", timestamp: publishedAt, data: { repositoryId: "repo-9" } } + ); + expect(published).toBe(true); + + await new Promise((r) => setTimeout(r, 600)); + const match = dataEvents(received).find((e) => e.type === "queue:position_changed"); + expect(match).toBeDefined(); + expect(match!.data).toEqual({ repositoryId: "repo-9" }); + // timestamp must survive the JSON round-trip preserving the instant + expect(new Date(match!.timestamp as string).toISOString()).toBe(publishedAt.toISOString()); + }, 15000); + }); +}); + + diff --git a/tests/integration/repo-mirror-settings-route.test.ts b/tests/integration/repo-mirror-settings-route.test.ts index 25b5a746..90996638 100644 --- a/tests/integration/repo-mirror-settings-route.test.ts +++ b/tests/integration/repo-mirror-settings-route.test.ts @@ -8,6 +8,9 @@ const { initializeMirrorMock, disableMirrorMock, syncMirrorRepositoryMock, + configurePushMirrorMock, + removePushMirrorMock, + pushMirrorNowMock, fakeSchema, } = vi.hoisted(() => ({ getUserFromRequestMock: vi.fn(async () => ({ userId: "user-1", isAdmin: false })), @@ -17,6 +20,12 @@ const { initializeMirrorMock: vi.fn(async () => ({ success: true, refsUpdated: 3 })), disableMirrorMock: vi.fn(async () => ({ success: true })), syncMirrorRepositoryMock: vi.fn(async () => ({ success: true, refsUpdated: 2 })), + configurePushMirrorMock: vi.fn(async () => ({ + success: true, + config: { enabled: true, url: "https://example.com/target.git", hasToken: false, status: "pending", lastPushMirrorAt: null }, + })), + removePushMirrorMock: vi.fn(async () => ({ success: true })), + pushMirrorNowMock: vi.fn(async () => ({ success: true, refsUpdated: 4, durationMs: 12 })), fakeSchema: { users: { username: {} }, repositories: { ownerId: {}, name: {}, id: {} }, @@ -46,6 +55,12 @@ vi.mock("@/lib/mirror-sync", () => ({ syncMirrorRepository: syncMirrorRepositoryMock, })); +vi.mock("@/lib/push-mirror", () => ({ + configurePushMirror: configurePushMirrorMock, + removePushMirror: removePushMirrorMock, + pushMirrorNow: pushMirrorNowMock, +})); + import { GET as mirrorGet, POST as mirrorPost, DELETE as mirrorDelete } from "@/pages/api/repos/[owner]/[repo]/settings/mirror"; import { POST as mirrorSyncPost } from "@/pages/api/repos/[owner]/[repo]/settings/mirror/sync"; @@ -64,6 +79,11 @@ function makeDb() { mirrorUrl: "https://example.com/upstream.git", mirrorSyncStatus: "success", lastMirrorSyncAt: new Date("2026-02-19T00:00:00Z"), + pushMirrorEnabled: true, + pushMirrorUrl: "https://example.com/target.git", + pushMirrorToken: "encrypted-token", + pushMirrorStatus: "success", + lastPushMirrorAt: new Date("2026-02-20T00:00:00Z"), })), }, }, @@ -76,6 +96,7 @@ async function readJson(response: Response): Promise { describe("repository mirror settings routes", () => { beforeEach(() => { + vi.clearAllMocks(); mockDb = makeDb(); getUserFromRequestMock.mockResolvedValue({ userId: "user-1", isAdmin: false }); canReadRepoMock.mockResolvedValue(true); @@ -84,6 +105,12 @@ describe("repository mirror settings routes", () => { initializeMirrorMock.mockResolvedValue({ success: true, refsUpdated: 3 }); disableMirrorMock.mockResolvedValue({ success: true }); syncMirrorRepositoryMock.mockResolvedValue({ success: true, refsUpdated: 2 }); + configurePushMirrorMock.mockResolvedValue({ + success: true, + config: { enabled: true, url: "https://example.com/target.git", hasToken: false, status: "pending", lastPushMirrorAt: null }, + }); + removePushMirrorMock.mockResolvedValue({ success: true }); + pushMirrorNowMock.mockResolvedValue({ success: true, refsUpdated: 4, durationMs: 12 }); }); it("returns mirror settings for readers", async () => { @@ -97,6 +124,12 @@ describe("repository mirror settings routes", () => { expect(body?.data?.isMirror).toBe(true); expect(typeof body?.data?.isHealthy).toBe("boolean"); expect(typeof body?.data?.isStale).toBe("boolean"); + expect(body?.data?.push).toMatchObject({ + enabled: true, + url: "https://example.com/target.git", + status: "success", + hasToken: true, + }); }); it("configures mirror for repo admins", async () => { @@ -139,5 +172,118 @@ describe("repository mirror settings routes", () => { expect(response.status).toBe(200); expect(body?.data?.configured).toBe(false); expect(disableMirrorMock).toHaveBeenCalledWith("repo-1"); + expect(removePushMirrorMock).toHaveBeenCalledWith("repo-1"); + }); + + it("configures push mirror for repo admins", async () => { + const response = await mirrorPost({ + params: { owner: "owner-1", repo: "demo" }, + request: new Request("http://localhost/api/repos/owner-1/demo/settings/mirror", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + push: { enabled: true, url: "https://example.com/target.git", authToken: "tok-123" }, + }), + }), + } as any); + + const body = await readJson(response); + expect(response.status).toBe(200); + expect(body?.data?.push).toEqual({ configured: true }); + expect(configurePushMirrorMock).toHaveBeenCalledWith("repo-1", { + url: "https://example.com/target.git", + authToken: "tok-123", + }); + }); + + it("rejects push config without url when enabling", async () => { + const response = await mirrorPost({ + params: { owner: "owner-1", repo: "demo" }, + request: new Request("http://localhost/api/repos/owner-1/demo/settings/mirror", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ push: { enabled: true } }), + }), + } as any); + + expect(response.status).toBe(400); + }); + + it("removes push mirror when disabled via POST", async () => { + const response = await mirrorPost({ + params: { owner: "owner-1", repo: "demo" }, + request: new Request("http://localhost/api/repos/owner-1/demo/settings/mirror", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ push: { enabled: false } }), + }), + } as any); + + const body = await readJson(response); + expect(response.status).toBe(200); + expect(body?.data?.push).toEqual({ configured: false }); + expect(removePushMirrorMock).toHaveBeenCalledWith("repo-1"); + }); + + it("runs manual push-only sync when direction=push", async () => { + const response = await mirrorSyncPost({ + params: { owner: "owner-1", repo: "demo" }, + request: new Request( + "http://localhost/api/repos/owner-1/demo/settings/mirror/sync?direction=push", + { method: "POST" } + ), + } as any); + + const body = await readJson(response); + expect(response.status).toBe(200); + expect(pushMirrorNowMock).toHaveBeenCalledWith("repo-1"); + expect(syncMirrorRepositoryMock).not.toHaveBeenCalled(); + expect(body?.data?.direction).toBe("push"); + expect(body?.data?.refsUpdated).toBe(4); + }); + + it("syncs both directions when direction=both", async () => { + const response = await mirrorSyncPost({ + params: { owner: "owner-1", repo: "demo" }, + request: new Request( + "http://localhost/api/repos/owner-1/demo/settings/mirror/sync?direction=both", + { method: "POST" } + ), + } as any); + + const body = await readJson(response); + expect(response.status).toBe(200); + expect(syncMirrorRepositoryMock).toHaveBeenCalledWith("repo-1"); + expect(pushMirrorNowMock).toHaveBeenCalledWith("repo-1"); + expect(body?.data?.pull).toMatchObject({ success: true }); + expect(body?.data?.push).toMatchObject({ success: true }); + }); + + it("defaults to pull sync preserving legacy behavior", async () => { + const response = await mirrorSyncPost({ + params: { owner: "owner-1", repo: "demo" }, + request: new Request( + "http://localhost/api/repos/owner-1/demo/settings/mirror/sync", + { method: "POST" } + ), + } as any); + + const body = await readJson(response); + expect(response.status).toBe(200); + expect(syncMirrorRepositoryMock).toHaveBeenCalledWith("repo-1"); + expect(pushMirrorNowMock).not.toHaveBeenCalled(); + expect(body?.data?.refsUpdated).toBe(2); + }); + + it("rejects invalid direction values", async () => { + const response = await mirrorSyncPost({ + params: { owner: "owner-1", repo: "demo" }, + request: new Request( + "http://localhost/api/repos/owner-1/demo/settings/mirror/sync?direction=sideways", + { method: "POST" } + ), + } as any); + + expect(response.status).toBe(400); }); }); diff --git a/tests/unit/discussions.test.ts b/tests/unit/discussions.test.ts new file mode 100644 index 00000000..338fd371 --- /dev/null +++ b/tests/unit/discussions.test.ts @@ -0,0 +1,242 @@ +/** + * Unit tests for Discussions schema + zod validation schemas + */ +import { describe, expect, it, vi } from "vitest"; +import { getTableConfig } from "drizzle-orm/pg-core"; +import { + discussions, + discussionComments, + DISCUSSION_CATEGORIES, +} from "@/db/schema/discussions"; + +/* Route modules pull infra deps at import time — stub them */ +vi.mock("@/db", () => ({ getDatabase: () => ({}) })); +vi.mock("@/lib/auth", () => ({ getUserFromRequest: vi.fn() })); +vi.mock("@/lib/permissions", () => ({ + canReadRepo: vi.fn(), + canWriteRepo: vi.fn(), + canAdminRepo: vi.fn(), +})); +vi.mock("@/lib/errors", () => ({ withErrorHandler: (fn: any) => fn })); +vi.mock("@/lib/logger", () => ({ + logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() }, +})); + +import { + createDiscussionSchema, + listDiscussionsQuerySchema, +} from "@/pages/api/repos/[owner]/[repo]/discussions/index"; +import { updateDiscussionSchema } from "@/pages/api/repos/[owner]/[repo]/discussions/[id]/index"; +import { createCommentSchema } from "@/pages/api/repos/[owner]/[repo]/discussions/[id]/comments"; + +function columnMap(table: any) { + const { columns } = getTableConfig(table); + return new Map(columns.map((c: any) => [c.name, c])); +} + +describe("discussions schema", () => { + it("defines the discussions table with expected columns", () => { + const config = getTableConfig(discussions); + expect(config.name).toBe("discussions"); + + const cols = columnMap(discussions); + for (const name of [ + "id", + "repository_id", + "author_id", + "title", + "body", + "category", + "pinned", + "closed", + "comment_count", + "last_activity_at", + "created_at", + "updated_at", + ]) { + expect(cols.has(name)).toBe(true); + } + }); + + it("uses text primary keys and required core fields", () => { + const cols = columnMap(discussions); + expect(cols.get("id")!.dataType).toBe("string"); + expect(cols.get("title")!.notNull).toBe(true); + expect(cols.get("body")!.notNull).toBe(true); + expect(cols.get("repository_id")!.notNull).toBe(true); + expect(cols.get("author_id")!.notNull).toBe(true); + }); + + it("defaults category to General, flags to false and counters to 0", () => { + const cols = columnMap(discussions); + expect(cols.get("category")!.hasDefault).toBe(true); + expect(cols.get("pinned")!.hasDefault).toBe(true); + expect(cols.get("closed")!.hasDefault).toBe(true); + expect(cols.get("comment_count")!.hasDefault).toBe(true); + }); + + it("declares repo+closed and repo+lastActivityAt indexes", () => { + const { indexes } = getTableConfig(discussions); + const names = indexes.map((i: any) => i.config.name); + expect(names).toContain("discussions_repo_closed_idx"); + expect(names).toContain("discussions_repo_activity_idx"); + expect(names).toContain("discussions_author_idx"); + }); + + it("defines threaded comments stored flat-ready", () => { + const config = getTableConfig(discussionComments); + expect(config.name).toBe("discussion_comments"); + + const cols = columnMap(discussionComments); + expect(cols.get("parent_id")!.notNull).toBe(false); + + const indexNames = config.indexes.map((i: any) => i.config.name); + expect(indexNames).toContain("discussion_comments_discussion_idx"); + expect(indexNames).toContain("discussion_comments_parent_idx"); + }); + + it("exposes the four v1 categories", () => { + expect([...DISCUSSION_CATEGORIES]).toEqual([ + "General", + "Ideas", + "Q&A", + "Show and tell", + ]); + }); +}); + +describe("createDiscussionSchema", () => { + it("accepts a valid payload and defaults the category", () => { + const parsed = createDiscussionSchema.parse({ + title: "Hello", + body: "World", + }); + expect(parsed.category).toBe("General"); + }); + + it("rejects titles beyond 300 characters", () => { + const result = createDiscussionSchema.safeParse({ + title: "x".repeat(301), + body: "World", + }); + expect(result.success).toBe(false); + }); + + it("rejects empty titles", () => { + const result = createDiscussionSchema.safeParse({ + title: "", + body: "World", + }); + expect(result.success).toBe(false); + }); + + it("rejects bodies over 64k", () => { + const result = createDiscussionSchema.safeParse({ + title: "Hello", + body: "x".repeat(65536), + }); + expect(result.success).toBe(false); + }); + + it("requires a body", () => { + const result = createDiscussionSchema.safeParse({ title: "Hello" }); + expect(result.success).toBe(false); + }); + + it("rejects unknown categories", () => { + const result = createDiscussionSchema.safeParse({ + title: "Hello", + body: "World", + category: "Memes", + }); + expect(result.success).toBe(false); + }); +}); + +describe("listDiscussionsQuerySchema", () => { + it("applies default sort", () => { + expect(listDiscussionsQuerySchema.parse({}).sort).toBe("lastActivity"); + }); + + it("parses closed filter strings into booleans", () => { + expect(listDiscussionsQuerySchema.parse({ closed: "true" }).closed).toBe( + true, + ); + expect(listDiscussionsQuerySchema.parse({ closed: "false" }).closed).toBe( + false, + ); + expect(listDiscussionsQuerySchema.parse({}).closed).toBeUndefined(); + }); + + it("rejects malformed closed filter", () => { + expect( + listDiscussionsQuerySchema.safeParse({ closed: "yes" }).success, + ).toBe(false); + }); + + it("accepts newest sort and known categories", () => { + const parsed = listDiscussionsQuerySchema.parse({ + sort: "newest", + category: "Q&A", + }); + expect(parsed.sort).toBe("newest"); + expect(parsed.category).toBe("Q&A"); + }); + + it("rejects unknown sort values", () => { + expect( + listDiscussionsQuerySchema.safeParse({ sort: "bogus" }).success, + ).toBe(false); + }); +}); + +describe("updateDiscussionSchema", () => { + it("accepts individual partial fields", () => { + expect(updateDiscussionSchema.parse({ pinned: true })).toEqual({ + pinned: true, + }); + expect(updateDiscussionSchema.parse({ closed: true })).toEqual({ + closed: true, + }); + expect( + updateDiscussionSchema.parse({ title: "New title", body: "New body" }), + ).toEqual({ title: "New title", body: "New body" }); + expect( + updateDiscussionSchema.parse({ category: "Show and tell" }).category, + ).toBe("Show and tell"); + }); + + it("rejects an empty patch", () => { + expect(updateDiscussionSchema.safeParse({}).success).toBe(false); + }); + + it("rejects non-boolean close/pin values", () => { + expect(updateDiscussionSchema.safeParse({ closed: "yes" }).success).toBe( + false, + ); + expect(updateDiscussionSchema.safeParse({ pinned: 1 }).success).toBe(false); + }); + + it("rejects oversized bodies", () => { + expect( + updateDiscussionSchema.safeParse({ body: "x".repeat(65536) }).success, + ).toBe(false); + }); +}); + +describe("createCommentSchema", () => { + it("accepts a body and optional parentId", () => { + expect(createCommentSchema.parse({ body: "hi" }).parentId).toBeUndefined(); + expect( + createCommentSchema.parse({ body: "hi", parentId: "dcomment_1" }) + .parentId, + ).toBe("dcomment_1"); + }); + + it("rejects empty or oversized bodies", () => { + expect(createCommentSchema.safeParse({ body: "" }).success).toBe(false); + expect( + createCommentSchema.safeParse({ body: "x".repeat(65536) }).success, + ).toBe(false); + }); +}); diff --git a/tests/unit/gists.test.ts b/tests/unit/gists.test.ts new file mode 100644 index 00000000..7df03a53 --- /dev/null +++ b/tests/unit/gists.test.ts @@ -0,0 +1,270 @@ +/** + * Unit tests for Gists schema + zod validation schemas + */ +import { describe, expect, it, vi } from "vitest"; +import { getTableConfig } from "drizzle-orm/pg-core"; +import { gists } from "@/db/schema/gists"; + +/* Route modules pull infra deps at import time — stub them */ +vi.mock("@/db", () => ({ getDatabase: () => ({}) })); +vi.mock("@/lib/auth", () => ({ getUserFromRequest: vi.fn() })); +vi.mock("@/lib/errors", () => ({ withErrorHandler: (fn: any) => fn })); +vi.mock("@/lib/logger", () => ({ + logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn() }, +})); +vi.mock("@/lib/utils", () => ({ + generateId: (prefix?: string) => `${prefix ?? "id"}_test`, +})); + +import { + createGistSchema, + gistFileSchema, + listGistsQuerySchema, + MAX_FILES, + MAX_TOTAL_CONTENT_BYTES, +} from "@/pages/api/gists/index"; +import { updateGistSchema } from "@/pages/api/gists/[id]/index"; + +function columnMap(table: any) { + const { columns } = getTableConfig(table); + return new Map(columns.map((c: any) => [c.name, c])); +} + +describe("gists schema", () => { + it("defines the gists table with expected columns", () => { + const config = getTableConfig(gists); + expect(config.name).toBe("gists"); + + const cols = columnMap(gists); + for (const name of [ + "id", + "user_id", + "description", + "public", + "files", + "created_at", + "updated_at", + ]) { + expect(cols.has(name)).toBe(true); + } + }); + + it("uses a text primary key and requires core fields", () => { + const cols = columnMap(gists); + expect(cols.get("id")!.dataType).toBe("string"); + expect(cols.get("user_id")!.notNull).toBe(true); + expect(cols.get("files")!.notNull).toBe(true); + expect(cols.get("public")!.notNull).toBe(true); + }); + + it("defaults description to empty string and public to false", () => { + const cols = columnMap(gists); + expect(cols.get("description")!.hasDefault).toBe(true); + expect(cols.get("public")!.hasDefault).toBe(true); + }); + + it("declares user+updatedAt and public indexes", () => { + const { indexes } = getTableConfig(gists); + const names = indexes.map((i: any) => i.config.name); + expect(names).toContain("gists_user_updated_idx"); + expect(names).toContain("gists_public_idx"); + }); + + it("references users with cascade delete", () => { + const config = getTableConfig(gists); + expect(config.foreignKeys).toHaveLength(1); + }); +}); + +describe("gistFileSchema filename validation", () => { + it("accepts simple filenames with extensions", () => { + const ok = gistFileSchema.safeParse({ + filename: "hello.py", + content: "print('hi')", + }); + expect(ok.success).toBe(true); + }); + + it("rejects empty filenames", () => { + const res = gistFileSchema.safeParse({ filename: "", content: "x" }); + expect(res.success).toBe(false); + }); + + it("rejects filenames longer than 255 chars", () => { + const res = gistFileSchema.safeParse({ + filename: "a".repeat(256), + content: "x", + }); + expect(res.success).toBe(false); + }); + + it("rejects path separators '/' in filenames", () => { + const res = gistFileSchema.safeParse({ + filename: "a/b.txt", + content: "x", + }); + expect(res.success).toBe(false); + }); + + it("rejects backslash separators in filenames", () => { + const res = gistFileSchema.safeParse({ + filename: "a\\b.txt", + content: "x", + }); + expect(res.success).toBe(false); + }); + + it("rejects traversal like '../secret'", () => { + const res = gistFileSchema.safeParse({ filename: "../secret", content: "x" }); + expect(res.success).toBe(false); + }); + + it("rejects bare '..' and '.' filenames", () => { + expect(gistFileSchema.safeParse({ filename: "..", content: "x" }).success).toBe( + false, + ); + expect(gistFileSchema.safeParse({ filename: ".", content: "x" }).success).toBe( + false, + ); + }); + + it("rejects embedded '..' segments like 'a..b.txt'", () => { + // '..' anywhere is treated as traversal-prone and rejected + const res = gistFileSchema.safeParse({ filename: "a..b.txt", content: "x" }); + expect(res.success).toBe(false); + }); +}); + +describe("createGistSchema", () => { + const validFile = { filename: "a.txt", content: "hello" }; + + it("accepts a minimal valid payload", () => { + const res = createGistSchema.safeParse({ + description: "my snippet", + public: true, + files: [validFile], + }); + expect(res.success).toBe(true); + if (res.success) { + expect(res.data.public).toBe(true); + expect(res.data.description).toBe("my snippet"); + } + }); + + it("defaults description to '' and public to false", () => { + const res = createGistSchema.safeParse({ files: [validFile] }); + expect(res.success).toBe(true); + if (res.success) { + expect(res.data.description).toBe(""); + expect(res.data.public).toBe(false); + } + }); + + it("rejects descriptions over 500 chars", () => { + const res = createGistSchema.safeParse({ + description: "x".repeat(501), + files: [validFile], + }); + expect(res.success).toBe(false); + }); + + it("requires at least one file", () => { + const res = createGistSchema.safeParse({ files: [] }); + expect(res.success).toBe(false); + }); + + it(`allows at most ${MAX_FILES} files`, () => { + const manyFiles = Array.from({ length: MAX_FILES }, (_, i) => ({ + filename: `f${i}.txt`, + content: "x", + })); + expect(createGistSchema.safeParse({ files: manyFiles }).success).toBe(true); + + const tooMany = Array.from({ length: MAX_FILES + 1 }, (_, i) => ({ + filename: `f${i}.txt`, + content: "x", + })); + expect(createGistSchema.safeParse({ files: tooMany }).success).toBe(false); + }); + + it("enforces the 1MB total content limit across all files", async () => { + const half = "a".repeat(MAX_TOTAL_CONTENT_BYTES / 2); + const atLimit = createGistSchema.safeParse({ + files: [ + { filename: "one.txt", content: half }, + { filename: "two.txt", content: half }, + ], + }); + expect(atLimit.success).toBe(true); + + const overLimit = createGistSchema.safeParse({ + files: [ + { filename: "one.txt", content: half }, + { filename: "two.txt", content: half + "x" }, + ], + }); + expect(overLimit.success).toBe(false); + + // Multi-byte characters count towards the byte budget + // ("é" is 2 bytes in UTF-8; 525000 chars = 1050000 bytes > 1MiB) + const multibyteOverLimit = createGistSchema.safeParse({ + files: [{ filename: "u.txt", content: "é".repeat(525000) }], + }).success; + expect(multibyteOverLimit).toBe(false); + }); + + it("propagates per-file validation to nested files", () => { + const res = createGistSchema.safeParse({ + files: [validFile, { filename: "../escape.txt", content: "nope" }], + }); + expect(res.success).toBe(false); + }); +}); + +describe("listGistsQuerySchema", () => { + it("parses public=true into boolean true", () => { + const res = listGistsQuerySchema.safeParse({ public: "true" }); + expect(res.success).toBe(true); + if (res.success) expect(res.data.public).toBe(true); + }); + + it("rejects invalid public values", () => { + expect(listGistsQuerySchema.safeParse({ public: "yes" }).success).toBe(false); + }); + + it("passes through q search strings", () => { + const res = listGistsQuerySchema.safeParse({ q: "snippet" }); + expect(res.success).toBe(true); + if (res.success) expect(res.data.q).toBe("snippet"); + }); + + it("allows an empty query", () => { + expect(listGistsQuerySchema.safeParse({}).success).toBe(true); + }); +}); + +describe("updateGistSchema", () => { + it("accepts partial updates", () => { + expect(updateGistSchema.safeParse({ description: "new" }).success).toBe(true); + expect(updateGistSchema.safeParse({ public: true }).success).toBe(true); + expect( + updateGistSchema.safeParse({ + files: [{ filename: "n.txt", content: "x" }], + }).success, + ).toBe(true); + }); + + it("rejects invalid replacement files", () => { + expect(updateGistSchema.safeParse({ files: [] }).success).toBe(false); + expect( + updateGistSchema.safeParse({ files: [{ filename: "/etc/x", content: "y" }] }) + .success, + ).toBe(false); + }); + + it("rejects unknown oversized descriptions", () => { + expect( + updateGistSchema.safeParse({ description: "x".repeat(501) }).success, + ).toBe(false); + }); +}); diff --git a/tests/unit/push-mirror.test.ts b/tests/unit/push-mirror.test.ts new file mode 100644 index 00000000..d7d64cc7 --- /dev/null +++ b/tests/unit/push-mirror.test.ts @@ -0,0 +1,459 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +const { + resolveRepoPathMock, + validateGitCloneUrlMock, + createSimpleGitMock, + rawMock, +} = vi.hoisted(() => ({ + resolveRepoPathMock: vi.fn(async () => "/data/repos/demo.git"), + validateGitCloneUrlMock: vi.fn(async (): Promise<{ valid: true } | { valid: false; reason: string }> => ({ valid: true })), + createSimpleGitMock: vi.fn(), + rawMock: vi.fn(async () => ""), +})); + +vi.mock("@/db", () => ({ + getDatabase: () => mockDb, + schema: {}, +})); + +vi.mock("@/lib/git-storage", () => ({ + resolveRepoPath: resolveRepoPathMock, +})); + +vi.mock("@/lib/url-validator", () => ({ + validateGitCloneUrl: validateGitCloneUrlMock, +})); + +vi.mock("@/lib/workflow-secret-crypto", () => ({ + encryptWorkflowSecret: (value: string) => `enc:${value}`, +})); + +vi.mock("@/lib/mirror-sync", () => ({ + // Mirrors pull-side behavior: token embedded transiently, never persisted. + buildFetchUrl: (url: string, token: string | null | undefined) => + token ? `https://oauth2:${token}@dest.example.com/target.git` : url, +})); + +vi.mock("@/lib/git", () => ({ + createSimpleGit: createSimpleGitMock, +})); + +import { + configurePushMirror, + getPushMirror, + processDuePushMirrors, + pushMirrorNow, + redactCredentials, + removePushMirror, +} from "@/lib/push-mirror"; + +function makeDb() { + const batches: any[][] = []; + const sets: any[] = []; + const takeBatch = () => (batches.length > 0 ? batches.shift()! : []); + const db = { + select: () => ({ + from: () => ({ + where: () => ({ + limit: async () => takeBatch(), + orderBy: () => ({ limit: async () => takeBatch() }), + }), + orderBy: () => ({ limit: async () => takeBatch() }), + }), + }), + update: () => ({ + set: (values: Record) => { + sets.push(values); + return { where: async () => ({}) }; + }, + }), + }; + return { db, batches, sets }; +} + +let mockDb: any; + +const repoRow = { + id: "repo-1", + diskPath: "/data/repos/demo.git", + pushMirrorEnabled: true, + pushMirrorUrl: "https://dest.example.com/target.git", + pushMirrorToken: "enc:tok-123", +}; + +describe("push-mirror library", () => { + beforeEach(() => { + vi.clearAllMocks(); + createSimpleGitMock.mockReturnValue({ raw: rawMock }); + rawMock.mockResolvedValue(""); + validateGitCloneUrlMock.mockResolvedValue({ valid: true }); + mockDb = makeDb().db; + }); + + afterEach(() => { + vi.unstubAllEnvs(); + }); + + describe("configurePushMirror", () => { + it("rejects URLs failing SSRF validation without touching the database", async () => { + const state = makeDb(); + mockDb = state.db; + validateGitCloneUrlMock.mockResolvedValue({ + valid: false, + reason: "Scheme \"file:\" is not allowed.", + }); + + const result = await configurePushMirror("repo-1", { url: "file:///tmp/repo.git" }); + + expect(result.success).toBe(false); + expect(result.error).toContain("not allowed"); + expect(state.sets).toHaveLength(0); + expect(validateGitCloneUrlMock).toHaveBeenCalledWith( + "file:///tmp/repo.git", + false + ); + }); + + it("permits private targets when PUSH_MIRROR_ALLOW_PRIVATE=true", async () => { + const state = makeDb(); + mockDb = state.db; + vi.stubEnv("PUSH_MIRROR_ALLOW_PRIVATE", "true"); + state.batches.push([{ id: "repo-1" }]); + + await configurePushMirror("repo-1", { url: "http://localhost:3000/repo.git" }); + + expect(validateGitCloneUrlMock).toHaveBeenCalledWith( + "http://localhost:3000/repo.git", + true + ); + }); + + it("encrypts the auth token and stores pending status", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push( + [{ id: "repo-1" }], + [ + { + enabled: true, + url: "https://dest.example.com/target.git", + token: "enc:tok-123", + status: "pending", + lastPushMirrorAt: null, + }, + ] + ); + + const result = await configurePushMirror("repo-1", { + url: "https://dest.example.com/target.git", + authToken: "tok-123", + }); + + expect(result.success).toBe(true); + expect(state.sets[0]).toMatchObject({ + pushMirrorEnabled: true, + pushMirrorUrl: "https://dest.example.com/target.git", + pushMirrorToken: "enc:tok-123", + pushMirrorStatus: "pending", + }); + expect(result.config?.hasToken).toBe(true); + }); + + it("keeps an existing token when authToken is omitted", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([{ id: "repo-1" }], []); + + const result = await configurePushMirror("repo-1", { + url: "https://dest.example.com/target.git", + }); + + expect(result.success).toBe(true); + expect(state.sets[0].pushMirrorToken).toBeUndefined(); + }); + + it("fails for unknown repository", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([]); + + const result = await configurePushMirror("missing", { + url: "https://dest.example.com/target.git", + }); + + expect(result.success).toBe(false); + expect(result.error).toBe("Repository not found"); + }); + }); + + describe("getPushMirror", () => { + it("never exposes the stored token", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([ + { + enabled: true, + url: "https://dest.example.com/target.git", + token: "enc:tok-123", + status: "success", + lastPushMirrorAt: new Date("2026-08-01T00:00:00Z"), + }, + ]); + + const config = await getPushMirror("repo-1"); + + expect(config).toMatchObject({ + enabled: true, + url: "https://dest.example.com/target.git", + hasToken: true, + status: "success", + }); + expect(config && "token" in config).toBe(false); + }); + + it("reports hasToken=false without a stored token", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([ + { enabled: true, url: "https://dest.example.com/target.git", token: null, status: null, lastPushMirrorAt: null }, + ]); + + const config = await getPushMirror("repo-1"); + expect(config?.hasToken).toBe(false); + }); + + it("returns null when repository missing", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([]); + + expect(await getPushMirror("missing")).toBeNull(); + }); + }); + + describe("removePushMirror", () => { + it("clears all push mirror fields", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([{ id: "repo-1" }]); + + const result = await removePushMirror("repo-1"); + + expect(result.success).toBe(true); + expect(state.sets[0]).toMatchObject({ + pushMirrorEnabled: false, + pushMirrorUrl: null, + pushMirrorToken: null, + pushMirrorStatus: null, + }); + }); + + it("fails when repository is missing", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([]); + + const result = await removePushMirror("missing"); + expect(result.success).toBe(false); + expect(result.error).toBe("Repository not found"); + }); + }); + + describe("pushMirrorNow", () => { + it("pushes forced heads/tags refspecs without --mirror and records success", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([repoRow]); + rawMock.mockResolvedValue( + "To https://dest.example.com/target.git\n * [new branch] main -> main\n v1 -> v1\n" + ); + + const result = await pushMirrorNow("repo-1"); + + expect(result.success).toBe(true); + expect(result.refsUpdated).toBe(2); + + const args = rawMock.mock.calls[0][0]; + expect(args[0]).toBe("push"); + expect(args).toContain("+refs/heads/*:refs/heads/*"); + expect(args).toContain("+refs/tags/*:refs/tags/*"); + expect(args).not.toContain("--mirror"); + // Transient credential injection per attempt + expect(args).toContain("https://oauth2:enc:tok-123@dest.example.com/target.git"); + + // simple-git block timeout kills the git process (timeout guard) + expect(createSimpleGitMock).toHaveBeenCalledWith( + expect.objectContaining({ + baseDir: "/data/repos/demo.git", + timeout: { block: 300_000 }, + }) + ); + + // Status transitions: pushing -> success with timestamp + expect(state.sets[0]).toMatchObject({ pushMirrorStatus: "pushing" }); + expect(state.sets[1]).toMatchObject({ + pushMirrorStatus: "success", + lastPushMirrorAt: expect.any(Date), + }); + }); + + it("does nothing when push mirror is not configured", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([ + { id: "repo-1", diskPath: "/data/repos/demo.git", pushMirrorEnabled: false, pushMirrorUrl: null }, + ]); + + const result = await pushMirrorNow("repo-1"); + + expect(result.success).toBe(false); + expect(result.error).toBe("Push mirror not configured"); + expect(rawMock).not.toHaveBeenCalled(); + expect(state.sets).toHaveLength(0); + }); + + it("marks failed and redacts credentials from git errors", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([repoRow]); + rawMock.mockRejectedValue( + new Error( + "fatal: unable to access 'https://oauth2:enc:tok-123@dest.example.com/target.git/': The requested URL returned error: 403" + ) + ); + + const result = await pushMirrorNow("repo-1"); + + expect(result.success).toBe(false); + expect(result.error).not.toContain("enc:tok-123"); + expect(result.error).toContain("***"); + expect(state.sets.at(-1)).toMatchObject({ pushMirrorStatus: "failed" }); + }); + + it("honours PUSH_MIRROR_TIMEOUT_SECS for the process kill guard", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([repoRow]); + vi.stubEnv("PUSH_MIRROR_TIMEOUT_SECS", "42"); + + await pushMirrorNow("repo-1"); + + expect(createSimpleGitMock).toHaveBeenCalledWith( + expect.objectContaining({ timeout: { block: 42_000 } }) + ); + }); + + it("survives repository lookup failures", async () => { + mockDb = { + select: () => { + throw new Error("db down"); + }, + update: () => ({ set: () => ({ where: async () => ({}) }) }), + }; + + const result = await pushMirrorNow("repo-1"); + expect(result.success).toBe(false); + expect(result.error).toBe("db down"); + }); + }); + + describe("processDuePushMirrors", () => { + it("processes never-pushed repos first, then stale ones, oldest first", async () => { + const state = makeDb(); + mockDb = state.db; + // Selection: never-pushed [a,b], stale [c]; then one repo lookup per id. + state.batches.push( + [{ id: "a" }, { id: "b" }], + [{ id: "c" }], + [repoRow], + [repoRow], + [repoRow] + ); + rawMock.mockResolvedValue("x -> y\n"); + + const result = await processDuePushMirrors({}); + + expect(result.total).toBe(3); + expect(result.pushed).toBe(3); + expect(result.failed).toBe(0); + expect(rawMock).toHaveBeenCalledTimes(3); + }); + + it("continues after individual failures and reports them", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push( + [{ id: "ok-1" }, { id: "bad-1" }, { id: "ok-2" }], + [], + [repoRow], + [repoRow], + [repoRow] + ); + let call = 0; + rawMock.mockImplementation(async () => { + call += 1; + if (call === 2) throw new Error("remote rejected"); + return ""; + }); + + const result = await processDuePushMirrors({}); + + expect(result.pushed).toBe(2); + expect(result.failed).toBe(1); + expect(result.failedRepoIds).toEqual(["bad-1"]); + }); + + it("respects the limit across both queries", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([{ id: "a" }], [{ id: "c" }]); + rawMock.mockResolvedValue(""); + + const result = await processDuePushMirrors({ limit: 2 }); + + expect(result.total).toBe(2); + }); + + it("uses PUSH_MIRROR_MIN_INTERVAL_SECS default of 300 for staleness cutoff", async () => { + const state = makeDb(); + mockDb = state.db; + state.batches.push([], []); + vi.stubEnv("PUSH_MIRROR_MIN_INTERVAL_SECS", "600"); + + await processDuePushMirrors({}); + // No crash + empty selection is enough here; cutoff math is internal. + expect(true).toBe(true); + }); + + it("never throws when due-selection query fails", async () => { + mockDb = { + select: () => { + throw new Error("db down"); + }, + }; + + const result = await processDuePushMirrors({ limit: 5 }); + + expect(result).toEqual({ + total: 0, + eligible: 0, + pushed: 0, + failed: 0, + failedRepoIds: [], + durationMs: expect.any(Number), + }); + }); + }); + + describe("redactCredentials", () => { + it("strips userinfo passwords from embedded URLs", () => { + const input = "error fetching https://user:s3cret@host/path and https://oauth2:t0k@other.host/x"; + const output = redactCredentials(input); + expect(output).not.toContain("s3cret"); + expect(output).not.toContain("t0k"); + expect(output).toContain("https://user:***@host/path"); + expect(output).toContain("https://oauth2:***@other.host/x"); + }); + }); +}); diff --git a/tests/unit/webhook-queue.test.ts b/tests/unit/webhook-queue.test.ts new file mode 100644 index 00000000..5b507f5d --- /dev/null +++ b/tests/unit/webhook-queue.test.ts @@ -0,0 +1,414 @@ +/** + * Unit: Webhook delivery queue (src/lib/webhooks.ts) + * + * Contract after queue decoupling: + * - triggerWebhooks only ENQUEUES pending delivery rows (no HTTP dispatch). + * - deliverWebhookDelivery performs exactly one attempt and transitions the + * row: delivered / pending-with-backoff / dead. + * - Retries follow the 1s→16s exponential backoff schedule and die after + * WEBHOOK_MAX_RETRIES retries (+initial attempt). + * - 4xx responses are non-retryable and dead-letter immediately. + * - processWebhookQueue claims rows atomically (FOR UPDATE SKIP LOCKED) and + * requeues locks stuck beyond STALE_WEBHOOK_LOCK_SECS. + */ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +const { validateWebhookUrlMock, isOfflineModeMock } = vi.hoisted(() => ({ + validateWebhookUrlMock: vi.fn(async (): Promise<{ valid: boolean; reason?: string }> => ({ valid: true })), + isOfflineModeMock: vi.fn(() => false), +})); + +vi.mock("@/db", () => ({ + getDatabase: () => mockDb.db, + schema: { + webhooks: { deliveryCount: "__col_delivery_count__" }, + webhookDeliveries: {}, + }, +})); + +vi.mock("@/lib/url-validator", () => ({ + validateWebhookUrl: validateWebhookUrlMock, +})); + +vi.mock("@/lib/config", () => ({ + isOfflineMode: isOfflineModeMock, +})); + +import { schema } from "@/db"; +import { PgDialect } from "drizzle-orm/pg-core"; +import { + deliverWebhookDelivery, + processWebhookQueue, + reclaimStaleLocks, + triggerWebhooks, +} from "@/lib/webhooks"; + +const WEBHOOKS_TABLE = Symbol("webhooks"); +const DELIVERIES_TABLE = Symbol("webhookDeliveries"); +// The mocked schema above hands the lib marker objects; mirror their identity +// so the fake db can tell updates apart. +(schema as any).webhooks = { ...(schema as any).webhooks, __table: WEBHOOKS_TABLE }; +(schema as any).webhookDeliveries = { __table: DELIVERIES_TABLE }; + +interface UpdateRecord { + table: symbol; + values: Record; +} + +function makeDb(options: { + hooks?: any[]; + /** FIFO results served by webhookDeliveries.findFirst */ + deliveries?: any[]; + /** ids returned by the atomic claim statement */ + claimIds?: string[]; + /** ids returned by the stale-sweep .returning() */ + reclaimedIds?: string[]; +} = {}) { + const state = { + insertedDeliveries: [] as any[], + deliveryUpdates: [] as UpdateRecord[], + webhookUpdates: [] as UpdateRecord[], + claimStatements: [] as string[], + }; + + function recordUpdate(table: any, values: Record) { + const record = { table, values }; + if (table === (schema as any).webhookDeliveries) state.deliveryUpdates.push(record); + else if (table === (schema as any).webhooks) state.webhookUpdates.push(record); + return record; + } + + const db = { + query: { + webhooks: { + findMany: async () => options.hooks ?? [], + findFirst: async () => options.hooks?.[0] ?? null, + }, + webhookDeliveries: { + findFirst: async () => { + const next = options.deliveries?.shift(); + if (next instanceof Error) throw next; // simulate a worker crash + return next ?? null; + }, + }, + }, + insert: (_table: any) => ({ + values: async (rows: any) => { + for (const row of Array.isArray(rows) ? rows : [rows]) { + state.insertedDeliveries.push(row); + } + return {}; + }, + }), + update: (table: any) => ({ + set: (values: Record) => ({ + where: () => { + const record = recordUpdate(table, values); + return { + returning: async () => { + if (table === (schema as any).webhookDeliveries && values.lockedAt === null && values.status === "pending") { + return (options.reclaimedIds ?? []).map((id) => ({ id })); + } + return []; + }, + then: (resolve: (v: any) => void, reject: (e: any) => void) => + Promise.resolve({}).then(() => resolve({}), reject), + } as any; + }, + }), + }), + execute: async (query: any) => { + state.claimStatements.push(new PgDialect().sqlToQuery(query).sql); + return { rows: (options.claimIds ?? []).map((id) => ({ id })) }; + }, + }; + + void recordUpdate; + return { db, state }; +} + +let mockDb: ReturnType; + +const hookRow = { + id: "hook-1", + repositoryId: "repo-1", + url: "https://receiver.example.com/hook", + secret: "topsecret", + events: ["push"], + active: true, +}; + +const starHookRow = { + ...hookRow, + id: "hook-star", + events: ["*"], +}; + +const otherHookRow = { + ...hookRow, + id: "hook-other", + events: ["issues"], +}; + +function pendingDelivery(overrides: Partial = {}) { + return { + id: "delivery-1", + webhookId: "hook-1", + event: "push", + payload: JSON.stringify({ ref: "refs/heads/main" }), + status: "pending", + attempts: 0, + nextAttemptAt: new Date(Date.now() - 1000), + lockedAt: null, + failureReason: null, + responseCode: null, + responseBody: null, + durationMs: null, + error: null, + requestHeaders: null, + responseHeaders: null, + ...overrides, + }; +} + +function stubFetch(status: number, body = "") { + const fetchMock = vi.fn(async () => + new Response(body, { status, headers: { "x-reply": "yes" } }), + ); + vi.stubGlobal("fetch", fetchMock); + return fetchMock; +} + +beforeEach(() => { + mockDb = makeDb(); +}); + +afterEach(() => { + vi.unstubAllGlobals(); + vi.unstubAllEnvs(); + vi.clearAllMocks(); +}); + +describe("triggerWebhooks (enqueue-only)", () => { + it("enqueues one pending delivery per matching hook and performs no HTTP dispatch", async () => { + const fetchMock = stubFetch(200); + mockDb = makeDb({ hooks: [hookRow, starHookRow, otherHookRow] }); + + const enqueued = await triggerWebhooks("repo-1", "push", { after: "abc" }); + + expect(enqueued).toBe(2); // push-subscriber + wildcard; 'issues'-only hook excluded + expect(mockDb.state.insertedDeliveries).toHaveLength(2); + + for (const row of mockDb.state.insertedDeliveries) { + expect(row.status).toBe("pending"); + expect(row.attempts).toBe(0); + expect(row.event).toBe("push"); + expect(row.nextAttemptAt).toBeInstanceOf(Date); + expect(row.webhookId).not.toBe("hook-other"); + expect(JSON.parse(row.payload)).toEqual({ after: "abc" }); + expect(row.id).toBeTruthy(); + } + expect(fetchMock).not.toHaveBeenCalled(); + }); + + it("returns 0 and inserts nothing when no hooks subscribe to the event", async () => { + mockDb = makeDb({ hooks: [otherHookRow] }); + const enqueued = await triggerWebhooks("repo-1", "push", {}); + expect(enqueued).toBe(0); + expect(mockDb.state.insertedDeliveries).toHaveLength(0); + }); +}); + +describe("deliverWebhookDelivery", () => { + it("marks the row delivered on 2xx, records response fields and bumps stats", async () => { + const fetchMock = stubFetch(200, "ok"); + mockDb = makeDb({ + hooks: [hookRow], + deliveries: [pendingDelivery()], + }); + + const outcome = await deliverWebhookDelivery("delivery-1"); + + expect(outcome).toBe("delivered"); + expect(fetchMock).toHaveBeenCalledTimes(1); + + const update = mockDb.state.deliveryUpdates[0]; + expect(update.values.status).toBe("delivered"); + expect(update.values.attempts).toBe(1); + expect(update.values.responseCode).toBe(200); + expect(update.values.responseBody).toBe("ok"); + expect(update.values.error).toBeNull(); + expect(update.values.lockedAt).toBeNull(); + + const headers = JSON.parse(update.values.requestHeaders); + expect(headers["X-Hub-Signature-256"]).toMatch(/^sha256=[0-9a-f]{64}$/); + expect(headers["X-OpenCodeHub-Delivery"]).toBe("delivery-1"); + expect(headers["X-OpenCodeHub-Event"]).toBe("push"); + + const stats = mockDb.state.webhookUpdates[0]; + expect(stats.values.lastDeliveryStatus).toBe("success"); + expect(stats.values.lastDeliveryAt).toBeInstanceOf(Date); + // Atomic COALESCE increment referencing the deliveryCount column + expect(JSON.stringify(stats.values.deliveryCount)).toContain( + "__col_delivery_count__", + ); + }); + + it("signs with X-Hub-Signature-256 only when a secret is configured", async () => { + stubFetch(200); + mockDb = makeDb({ + hooks: [{ ...hookRow, secret: null }], + deliveries: [pendingDelivery()], + }); + + await deliverWebhookDelivery("delivery-1"); + + const headers = JSON.parse(mockDb.state.deliveryUpdates[0].values.requestHeaders); + expect(headers["X-Hub-Signature-256"]).toBeUndefined(); + }); + + it("retries failures with exponential backoff, then dies after max attempts", async () => { + vi.stubEnv("WEBHOOK_MAX_RETRIES", "1"); // total attempts = 2 + const fetchMock = stubFetch(500); + const delivery = pendingDelivery(); + mockDb = makeDb({ hooks: [hookRow], deliveries: [delivery] }); + + const first = await deliverWebhookDelivery("delivery-1"); + expect(first).toBe("retrying"); + expect(fetchMock).toHaveBeenCalledTimes(1); + + const retryValues = mockDb.state.deliveryUpdates[0].values; + expect(retryValues.status).toBe("pending"); + expect(retryValues.attempts).toBe(1); + expect(retryValues.lockedAt).toBeNull(); + expect(retryValues.failureReason).toBeNull(); + const delay = retryValues.nextAttemptAt.getTime() - Date.now(); + expect(delay).toBeGreaterThanOrEqual(900); // 1s backoff after 1st failure + expect(delay).toBeLessThanOrEqual(2500); + expect(retryValues.error).toContain("HTTP 500"); + + // Second (final allowed) attempt fails → dead letter with reason + mockDb = makeDb({ + hooks: [hookRow], + deliveries: [pendingDelivery({ attempts: 1 })], + }); + const second = await deliverWebhookDelivery("delivery-1"); + + expect(second).toBe("dead"); + const deadValues = mockDb.state.deliveryUpdates[0].values; + expect(deadValues.status).toBe("dead"); + expect(deadValues.attempts).toBe(2); + expect(deadValues.failureReason).toBe("Failed after 2/2 attempts"); + expect(deadValues.lockedAt).toBeNull(); + }); + + it("dead-letters 4xx responses immediately (non-retryable)", async () => { + const fetchMock = stubFetch(422); + mockDb = makeDb({ + hooks: [hookRow], + deliveries: [pendingDelivery()], + }); + + const outcome = await deliverWebhookDelivery("delivery-1"); + + expect(outcome).toBe("dead"); + expect(fetchMock).toHaveBeenCalledTimes(1); // no retry scheduled + const deadValues = mockDb.state.deliveryUpdates[0].values; + expect(deadValues.status).toBe("dead"); + expect(deadValues.attempts).toBe(1); + expect(deadValues.failureReason).toContain("Not retryable"); + expect(deadValues.failureReason).toContain("422"); + expect(deadValues.responseCode).toBe(422); + expect(deadValues.nextAttemptAt).toBeUndefined(); // no reschedule + }); + + it("dead-letters rows whose webhook was deleted", async () => { + stubFetch(200); + mockDb = makeDb({ hooks: [], deliveries: [pendingDelivery()] }); + + const outcome = await deliverWebhookDelivery("delivery-1"); + + expect(outcome).toBe("dead"); + expect(mockDb.state.deliveryUpdates[0].values.failureReason).toContain("no longer exist"); + }); + + it("skips terminal rows without touching them", async () => { + stubFetch(200); + mockDb = makeDb({ + hooks: [hookRow], + deliveries: [pendingDelivery({ status: "delivered" })], + }); + + const outcome = await deliverWebhookDelivery("delivery-1"); + + expect(outcome).toBe("skipped"); + expect(mockDb.state.deliveryUpdates).toHaveLength(0); + expect(mockDb.state.webhookUpdates).toHaveLength(0); + }); +}); + +describe("processWebhookQueue", () => { + it("claims due rows atomically with FOR UPDATE SKIP LOCKED and delivers them", async () => { + stubFetch(200); + mockDb = makeDb({ + hooks: [hookRow], + deliveries: [pendingDelivery({ id: "d1" }), pendingDelivery({ id: "d2" })], + claimIds: ["d1", "d2"], + }); + + const result = await processWebhookQueue(20); + + expect(result.claimed).toBe(2); + expect(result.delivered).toBe(2); + expect(result.dead).toBe(0); + expect(mockDb.state.claimStatements).toHaveLength(1); + expect(mockDb.state.claimStatements[0]).toContain("FOR UPDATE SKIP LOCKED"); + expect(mockDb.state.claimStatements[0]).toContain("status = 'pending'"); + expect(mockDb.state.claimStatements[0]).toContain("next_attempt_at <= now()"); + const deliveredUpdates = mockDb.state.deliveryUpdates.filter( + (u) => u.values.status === "delivered", + ); + expect(deliveredUpdates).toHaveLength(2); + // The sweep ran first and requeued nothing (no reclaimedIds configured) + expect(mockDb.state.deliveryUpdates[0].values.status).toBe("pending"); + expect(mockDb.state.deliveryUpdates[0].values.lockedAt).toBeNull(); + }); + + it("keeps delivering remaining claimed rows when one delivery crashes", async () => { + const fetchMock = stubFetch(200); + mockDb = makeDb({ + hooks: [hookRow], + deliveries: [ + new Error("simulated worker crash"), // d-crash: findFirst explodes → caught by queue loop + pendingDelivery({ id: "d-ok" }), + ], + claimIds: ["d-crash", "d-ok"], + }); + + const result = await processWebhookQueue(10); + + // The crashed row is not counted anywhere — it stays 'processing' until + // the stale-lock sweep requeues it; the healthy row still gets delivered. + expect(result.claimed).toBe(2); + expect(result.delivered).toBe(1); + expect(result.dead).toBe(0); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it("requeues stale processing locks before claiming", async () => { + stubFetch(200); + mockDb = makeDb({ + hooks: [hookRow], + deliveries: [pendingDelivery()], + claimIds: ["delivery-1"], + reclaimedIds: ["stale-1", "stale-2"], + }); + + const swept = await reclaimStaleLocks(); + + expect(swept).toBe(2); + expect(mockDb.state.deliveryUpdates).toHaveLength(1); + expect(mockDb.state.deliveryUpdates[0].values.status).toBe("pending"); + expect(mockDb.state.deliveryUpdates[0].values.lockedAt).toBeNull(); + }); +});