Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 42 additions & 3 deletions .agents/rules/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,22 +27,61 @@ Read before touching entities, DbContexts, migrations, or query filters.

A child entity reached **only** through a parent's navigation collection needs `Property(x => x.Id).ValueGeneratedNever()` in its EF config — otherwise EF treats it as `Modified` instead of `Added` and the insert silently misbehaves.

## Database providers (PostgreSQL + SQL Server)

- The provider is **config-selected**, never hardcoded: `DatabaseOptions:Provider` (`POSTGRESQL` | `MSSQL`) plus a matching `DatabaseOptions:MigrationsAssembly`. Both migrations projects ship in every build, so switching is a config change — no rescaffold, no file surgery.
- **MSSQL requires SQL Server 2025 (17.x) or Azure SQL.** JSON columns map to the native `json` type (compatibility level 170); it does not exist on 2019/2022 and those migrations will not apply there.
- **Never write provider SQL in an entity configuration.** A literal `HasColumnType("jsonb")` or `HasFilter("\"IsDeleted\" = FALSE")` locks the model to one provider and fails model-build on the other. Declare intent instead — `HeroProviderConventions` resolves it per provider:

| Instead of | Write | PostgreSQL | SQL Server |
|---|---|---|---|
| `HasColumnType("jsonb")` | `.HasJsonColumn()` | `jsonb` | `json` |
| `HasColumnType("text")` | `.HasUnboundedTextColumn()` | `text` | `nvarchar(max)` |
| `HasDefaultValueSql("'{}'::jsonb")` | `.HasJsonDefaultEmptyObject()` | `'{}'::jsonb` | `N'{}'` |
| `HasDefaultValueSql("CURRENT_TIMESTAMP")` | `.HasUtcNowDefault()` | `CURRENT_TIMESTAMP` | `SYSUTCDATETIME()` |
| `HasFilter("\"IsDeleted\" = FALSE")` | `.HasNotDeletedFilter()` | `"IsDeleted" = FALSE` | `[IsDeleted] = 0` |
| `HasFilter("\"Status\" = 1")` | `.HasEqualsFilter("Status", 1)` | `"Status" = 1` | `[Status] = 1` |
| `.HasMethod("gin").HasOperators("gin_trgm_ops")` | `.AsTrigramSearchIndex()` | trigram GIN | *removed from model* |
| `.HasMethod("gin").HasOperators("jsonb_path_ops")` | `.AsJsonContainmentIndex()` | `jsonb_path_ops` GIN | *removed; `CREATE JSON INDEX` in the migration* |

- A DbContext that does **not** derive from `BaseDbContext` must register the conventions itself in a `ConfigureConventions` override (`IdentityDbContext`, `TenantDbContext`, `BillingDbContext` do). `ProviderConventionRegistrationTests` fails the build if a new one forgets — the symptom otherwise is silent: JSON columns become `text`, every partial-index filter vanishes, and `Fsh:*` annotations leak into the snapshot and break the scaffolder.
- **Queries:** use `.WhereSearch(db.Database, term, x => x.Name, …)` rather than `EF.Functions.ILike` — `ILike` is Npgsql-only. For matching a JSON property as text, build the pattern with `ProviderQueryExtensions.JsonTextPropertyPattern`: PostgreSQL renders `jsonb::text` as `{"k": "v"}` (space after the colon), SQL Server's `json` casts to `{"k":"v"}` (no space), so a hard-coded pattern silently matches nothing on the other provider.
- **Capability differences on SQL Server:**
- Audit `Source`/`UserName` substring search scans — no trigram equivalent for `%term%` on `nvarchar(max)`.
- Chat message search uses `FREETEXTTABLE` when the instance has Full-Text Search, and falls back to a `LIKE` scan when it does not (the official `mcr.microsoft.com/mssql/server` container has no FTS; Azure SQL and full installs do). SQL Server populates a full-text index **asynchronously**, unlike PostgreSQL's generated `tsvector` column, so the FTS path is UNIONed with a `LIKE` pass over the last few minutes — otherwise a just-sent message would be silently unsearchable. Those pending matches are returned first, newest-first, ahead of the ranked ones.
- Guid v7 ids do **not** sort chronologically on SQL Server: `uniqueidentifier` compares the last six bytes first. Message paging therefore sorts on a persisted `char(36)` sort key (`ChatDbContext.MessageSortKey`) rather than on `Id`. Any new cursor pagination keyed on a Guid needs the same treatment — see `MessageOrdering`.
- **Raw SQL + `Include` on SQL Server:** EF wraps a `FromSql` query in a subselect to resolve the includes, and `SELECT … FROM (WITH … SELECT …) t` is not valid T-SQL. Use derived tables, never a CTE, in a `FromSql` that gets composed with `Include`.
- `EnableRetryOnFailure` is deliberately **off** for MSSQL: its execution strategy refuses user-initiated transactions, which the outbox requires.

## Migrations

All migrations live in **one** project, `src/Host/FSH.Starter.Migrations.PostgreSQL`, organized **per-module by folder** (`Identity/`, `Catalog/`, `Chat/`, …), each with its own `{Module}DbContextModelSnapshot`.
Migrations live in **one project per provider** `src/Host/FSH.Starter.Migrations.PostgreSQL` and `src/Host/FSH.Starter.Migrations.MSSQL` — each organized **per-module by folder** (`Identity/`, `Catalog/`, `Chat/`, …) with its own `{Module}DbContextModelSnapshot`. **An entity change needs a migration in BOTH.**

```bash
# PostgreSQL (default provider)
dotnet ef migrations add {Name} \
--project src/Host/FSH.Starter.Migrations.PostgreSQL \
--startup-project src/Host/FSH.Starter.Api \
--context {Module}DbContext

# SQL Server — the env vars pick the provider the design-time model is built against
DatabaseOptions__Provider=MSSQL \
DatabaseOptions__MigrationsAssembly=FSH.Starter.Migrations.MSSQL \
DatabaseOptions__ConnectionString='Server=localhost,1433;Database=fsh;User Id=sa;Password=…;TrustServerCertificate=True' \
dotnet ef migrations add {Name} \
--project src/Host/FSH.Starter.Migrations.MSSQL \
--startup-project src/Host/FSH.Starter.Api \
--context {Module}DbContext
```

- **Verify with the `verify-migrations` skill** (`MigrationDriftTests` in `Architecture.Tests`): it compares every context's live model against that provider's own snapshot — the in-process equivalent of `has-pending-model-changes` — with no database and no Docker, and prints the exact `migrations add` command for whatever is missing. It is what catches "generated the PostgreSQL migration, forgot the SQL Server one", and also a portable-intent helper resolving differently than expected. The CLI command remains useful for one-off diagnosis.
- **Maintaining both providers is not mandatory.** `FshMaintainedDbProviders` in `src/Directory.Build.props` (`both` | `POSTGRESQL` | `MSSQL`, override per run with `FSH_MIGRATIONS_PROVIDERS`) decides which providers *fail* the build on drift; the rest only report it. Ships as `both` because the bundled modules are kept in sync for both engines — narrow it once a project settles on one, so nobody is blocked by migrations for an engine they do not use.

- **`migrations remove` operates on the snapshot** — run a full build *before* `migrations add` so the snapshot is current, or you can lose the previous migration.
- The DB is **not** migrated at API startup. The `DbMigrator` host is a separate step: `apply` (default), `seed`, `seed-demo` (dev only), `list-pending`; flags `--tenant <id>`, `--catalog-only`, `--seed`. It migrates the tenant catalog first, then each tenant's per-module schema, serialized by a Postgres advisory lock.
- The DB is **not** migrated at API startup. The `DbMigrator` host is a separate step: `apply` (default), `seed`, `seed-demo` (dev only), `list-pending`; flags `--tenant <id>`, `--catalog-only`, `--seed`. It migrates the tenant catalog first, then each tenant's per-module schema, serialized by a database-held migrator lock (a Postgres advisory lock, or `sp_getapplock` on SQL Server).
- `dotnet-ef` is pinned in `.config/dotnet-tools.json` — run `dotnet tool restore` first.

## Tests + EF

- Integration tests use Testcontainers (real PostgreSQL) — **Docker must be running**.
- Integration tests use Testcontainers (real PostgreSQL by default) — **Docker must be running**. Set `FSH_TEST_DB_PROVIDER=MSSQL` to run the same suite against SQL Server 2025.
- In integration tests, set the Finbuckle tenant context **inline in the same method** as the `UserManager`/`DbContext` call; an awaited-helper set is lost (AsyncLocal) and the tenant query filter NREs.
2 changes: 1 addition & 1 deletion .agents/rules/integration-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

## Harness

`WebApplicationFactory` over **real** infra via Testcontainers — PostgreSQL + Redis + MinIO. **Docker must be running**; if it isn't, tests fail fast with `DockerUnavailableException` (environmental, not a regression — run the unit projects instead).
`WebApplicationFactory` over **real** infra via Testcontainers — PostgreSQL + Redis + MinIO. Set `FSH_TEST_DB_PROVIDER=MSSQL` to run the same suite against SQL Server 2025 instead of PostgreSQL; the default is unchanged. **Docker must be running**; if it isn't, tests fail fast with `DockerUnavailableException` (environmental, not a regression — run the unit projects instead).

`FshWebApplicationFactory` (`Integration.Tests/Infrastructure/`) boots the containers, overlays in-memory config, swaps `IMailService` → `NoOpMailService`, and rewires storage to MinIO.

Expand Down
2 changes: 1 addition & 1 deletion .agents/rules/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ xUnit · Shouldly (`result.ShouldBe(...)`) · NSubstitute (`Substitute.For<IServ
| `{Module}.Tests` | Unit: handlers, services, domain | no |
| `Framework.Tests`, `Generic.Tests`, `Caching.Tests` | BuildingBlocks units | no |
| `Architecture.Tests` | NetArchTest: module boundaries + tenant-isolation rules + handler↔validator pairing | no |
| `Integration.Tests` | `WebApplicationFactory` over real PostgreSQL/Redis/MinIO | **yes** |
| `Integration.Tests` | `WebApplicationFactory` over real PostgreSQL (or SQL Server 2025 via `FSH_TEST_DB_PROVIDER=MSSQL`)/Redis/MinIO | **yes** |
| `Integration.Middleware.Tests` | Real middleware wiring | **yes** |

```bash
Expand Down
42 changes: 37 additions & 5 deletions .agents/skills/create-migration/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
---
name: create-migration
description: Create and apply an EF Core migration for a module's DbContext the FSH way (central Migrations project, per-module folder, correct --context). Use after changing entities/EF config. See .agents/rules/database.md.
description: Create and apply an EF Core migration for a module's DbContext the FSH way (one Migrations project per provider, per-module folder, correct --context). Use after changing entities/EF config — every change needs a migration for BOTH PostgreSQL and SQL Server. See .agents/rules/database.md.
argument-hint: "[ModuleName] [MigrationName]"
---

# Create Migration

All migrations live in **one** project — `src/Host/FSH.Starter.Migrations.PostgreSQL` — but are foldered
**per module/context** (`Catalog/`, `Identity/`, …), each with its own `{X}DbContextModelSnapshot`. The DB
is **not** migrated at API startup; the `DbMigrator` host applies it.
Migrations live in **one project per provider** — `src/Host/FSH.Starter.Migrations.PostgreSQL` and
`src/Host/FSH.Starter.Migrations.MSSQL` — each foldered **per module/context** (`Catalog/`, `Identity/`, …)
with its own `{X}DbContextModelSnapshot`. The DB is **not** migrated at API startup; the `DbMigrator`
host applies it.

> **Both providers, every time.** An entity or EF-config change that produces a PostgreSQL migration
> must produce the matching SQL Server one, or the MSSQL deployment silently drifts. Step 2 runs twice.

## Step 0 — restore the pinned tool (first time)

Expand All @@ -33,13 +37,37 @@ Specify **all three** of `--project` (the Migrations project), `--startup-projec
folder for the context).

```bash
# 2a — PostgreSQL
dotnet ef migrations add {MigrationName} \
--project src/Host/FSH.Starter.Migrations.PostgreSQL \
--startup-project src/Host/FSH.Starter.Api \
--context {X}DbContext \
--output-dir {X}
```

```bash
# 2b — SQL Server. The env vars decide which provider the design-time model is built for; without
# them you would scaffold PostgreSQL DDL into the MSSQL project. Needs a reachable SQL Server 2025.
DatabaseOptions__Provider=MSSQL \
DatabaseOptions__MigrationsAssembly=FSH.Starter.Migrations.MSSQL \
DatabaseOptions__ConnectionString='Server=localhost,1433;Database=fsh;User Id=sa;Password=…;TrustServerCertificate=True' \
dotnet ef migrations add {MigrationName} \
--project src/Host/FSH.Starter.Migrations.MSSQL \
--startup-project src/Host/FSH.Starter.Api \
--context {X}DbContext \
--output-dir {X}
```

Then confirm neither provider has drifted. The fastest check is the **`verify-migrations`** skill
(`MigrationDriftTests`, no database, under a second), which names the exact command for anything
missing. The CLI equivalent, for one-off diagnosis:

```bash
dotnet ef migrations has-pending-model-changes --context {X}DbContext \
--project src/Host/FSH.Starter.Migrations.PostgreSQL --startup-project src/Host/FSH.Starter.Api
# …and the same with the MSSQL env vars + --project …Migrations.MSSQL
```

## Step 3 — review the generated SQL before applying

```bash
Expand All @@ -65,13 +93,17 @@ dotnet run --project src/Host/FSH.Starter.DbMigrator -- list-pending # to prev

## Notes

- A **new module** also needs a `{X}/` folder in the Migrations project and the runtime project referenced from it — see `add-module`.
- A **new module** also needs a `{X}/` folder in **both** Migrations projects, and the runtime project referenced from both — see `add-module`.
- `dotnet ef` against a `BaseDbContext` works because the 4-arg ctor is satisfied by the startup host's DI.
- Write EF config with the **portable helpers** (`.HasJsonColumn()`, `.HasNotDeletedFilter()`, …), never provider SQL literals — see the provider table in `.agents/rules/database.md`. A literal fails model-build on the other provider.
- Some DDL has no EF API and is hand-written into the MSSQL migration: the audit `CREATE JSON INDEX` and the chat full-text catalog. Preserve those blocks if you ever regenerate those migrations.

## Checklist

- [ ] `dotnet tool restore` done (first time)
- [ ] Built **before** `migrations add`
- [ ] `--context {X}DbContext` + `--output-dir {X}` (lands in the right folder)
- [ ] Migration added for **both** providers (PostgreSQL *and* MSSQL)
- [ ] `verify-migrations` skill green for every maintained provider (advisory drift on the others reviewed)
- [ ] Reviewed the generated SQL for data loss
- [ ] Applied via DbMigrator `apply` (or `ef database update` for one context locally)
2 changes: 1 addition & 1 deletion .agents/skills/testing-guide/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ Don't weaken these to make a change pass — fix the code.

## Integration tests

`Integration.Tests` runs over real Postgres/Redis/MinIO via Testcontainers — **Docker required**. Set the
`Integration.Tests` runs over real Postgres/Redis/MinIO via Testcontainers — **Docker required**. `FSH_TEST_DB_PROVIDER=MSSQL` runs the same suite against SQL Server 2025 instead of Postgres. Set the
Finbuckle tenant context inline, rewire `IStorageService` post-registration for MinIO, force long-polling
for SignalR. All detailed in `.agents/rules/integration-testing.md`.

Expand Down
76 changes: 76 additions & 0 deletions .agents/skills/verify-migrations/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
name: verify-migrations
description: Check that every DbContext's migrations are up to date with the model, for each database provider this project maintains. Run before committing an entity or EF-config change. Catches the "generated the PostgreSQL migration, forgot the SQL Server one" mistake. See .agents/rules/database.md.
---

# Verify Migrations

With one migrations project per provider (`FSH.Starter.Migrations.PostgreSQL`,
`FSH.Starter.Migrations.MSSQL`), forgetting the second one **fails nothing**: the build passes, the
tests pass, the API starts. The forgotten provider simply deploys against a stale schema.

`MigrationDriftTests` (in `Architecture.Tests`) compares each context's live model against that
provider's own snapshot — the in-process equivalent of `has-pending-model-changes`. No database, no
Docker, under a second.

## Step 1 — build

The guard reads the compiled model, so a stale build gives a stale answer.

```bash
dotnet build src/FSH.Starter.slnx
```

## Step 2 — run the guard

```bash
dotnet test src/Tests/Architecture.Tests --no-build \
--filter "FullyQualifiedName~MigrationDriftTests" \
--logger "console;verbosity=detailed"
```

`verbosity=detailed` matters: advisory findings for an unmaintained provider are printed rather than
asserted, and the default logger hides output from passing tests.

## Step 3 — read the result

- **Green, no output** — nothing to do.
- **Failure** — a provider this project maintains is behind. The message names the context, the
provider, and the exact `dotnet ef migrations add` command. Run it (fill in `<Name>` and
`<Folder>`), rebuild, re-run.
- **`[migration-drift]` line but green** — a provider this project does *not* maintain is behind.
Not fatal; fix it only if you still intend to deploy that engine.

To check both providers regardless of what the project maintains:

```bash
FSH_MIGRATIONS_PROVIDERS=both dotnet test src/Tests/Architecture.Tests --no-build \
--filter "FullyQualifiedName~MigrationDriftTests"
```

## Which providers are enforced

`FshMaintainedDbProviders` in `src/Directory.Build.props` (`both` | `POSTGRESQL` | `MSSQL`), with the
`FSH_MIGRATIONS_PROVIDERS` env var as a per-run override. Ships as `both` because the modules bundled
with the kit are kept in sync for both engines. **Once a project settles on one engine, narrow it** —
then the other becomes advisory and never blocks a build over migrations for an engine nobody uses.

## Notes

- **Not every difference between the two projects is a mistake.** `AddMessageChronologicalSortKey`
exists only in MSSQL, because the Guid v7 sort key is added only when
`ChatDbContext` sees SQL Server. Likewise the trigram and JSON-containment indexes are removed from
the MSSQL model on purpose. The guard already accounts for this: it compares each provider against
its own snapshot, never one snapshot against the other.
- **Hand-written DDL must survive a regeneration.** The MSSQL migrations carry SQL that EF has no API
for: `CREATE JSON INDEX` on `audit.AuditRecords`, and the chat full-text catalog. If you ever
regenerate those migrations, re-apply those blocks.
- A missing snapshot for a context in one project (a new module that only got a folder in one
migrations project) is reported by `Every_Context_Should_Have_A_Snapshot_In_Both_Providers`.

## Checklist

- [ ] Built before running the guard
- [ ] Guard green for every maintained provider
- [ ] Advisory drift reviewed, and either fixed or consciously accepted
- [ ] Generated migrations reviewed for data loss before applying (see `create-migration`)
Loading
Loading