Add upsertsWriteNullValues option so upserts don't treat nulls as absent (#2998)

[Adam-Langley]

Closes #2998.

The problem

The DO UPDATE SET clause of an upsert is built with toColumns(true) — nullToAbsent — so columns holding null are dropped from the statement entirely and a conflicting row keeps whatever value it had before:

-- upserting a row whose `title` is null
INSERT INTO "todos" ("id", "content") VALUES (?, ?)
  ON CONFLICT("id") DO UPDATE SET "id" = ?, "content" = ?
--                                ^ "title" is absent, so the old title survives

INSERT ... ON CONFLICT DO UPDATE therefore yields a different row depending on whether it inserted or updated. Passing a complete row and getting a half-updated one back is surprising, and it's the behavior reported in #2998.

This only affects insertables that distinguish null from absent by value — generated row classes and RawValuesInsertable. Companions are unaffected either way: their generated toColumns ignores nullToAbsent and writes exactly the fields that are present.

The change

A new DriftDatabaseOptions.upsertsWriteNullValues, defaulting to false, so behavior is unchanged unless opted in. When enabled:

INSERT INTO "todos" ("id", "content") VALUES (?, ?)
  ON CONFLICT("id") DO UPDATE SET "id" = ?, "title" = ?, "content" = ?,
                                  "target_date" = ?, "category" = ?, "status" = ?

The runtime change is one expression in insert.dart:

final updateSet = upsertInsertable.toColumns(
  !ctx.options.upsertsWriteNullValues,
);

It's enabled declaratively through a new upserts_write_null_values build option, which makes drift_dev emit the options override on the generated database — reusing the mechanism store_date_time_values_as_text already uses, and composing with it:

targets:
  $default:
    builders:
      drift_dev:
        options:
          upserts_write_null_values: true

I went with an opt-in flag rather than just changing the default, since the current behavior is observable and someone may well be relying on it. Happy to flip it to a default (or a breaking change slated for drift 3) if you'd prefer — the flag plumbing is the same either way.

Results

Before — the same assertions against develop unmodified

The new test file with the three upsertsWriteNullValues: true lines removed (the API doesn't exist yet); everything else byte-identical.

00:00 +0: SQL generation with the default options drops null columns from the update clause
00:00 +1: SQL generation with the default options drops null columns for an explicit DoUpdate clause
00:00 +2 -1: SQL generation DESIRED behaviour writes null columns into the update clause [E]
  No matching calls. All calls: ... MockExecutor.runInsert(
    'INSERT INTO "todos" ("id", "content") VALUES (?, ?)
     ON CONFLICT("id") DO UPDATE SET "id" = ?, "content" = ?', [3, content, 3, content])

00:00 +2 -2: SQL generation DESIRED behaviour writes null columns for an explicit DoUpdate clause [E]
  No matching calls. ... (same truncated update clause)

00:00 +6 -3: against a real database DESIRED: writes nulls on upsert [E]
  Expected: <null>
    Actual: 'initial title'

00:00 +7 -3: Some tests failed.

7 passed, 3 failed. The integration failure (Expected: <null> Actual: 'initial title') is #2998 reproduced against a real sqlite3 database.

After

00:00 +0: SQL generation with the default options drops null columns from the update clause
00:00 +1: SQL generation with the default options drops null columns for an explicit DoUpdate clause
00:00 +2: SQL generation with upsertsWriteNullValues writes null columns into the update clause
00:00 +3: SQL generation with upsertsWriteNullValues writes null columns for an explicit DoUpdate clause
00:00 +4: SQL generation with upsertsWriteNullValues does not change the INSERT part of the statement
00:00 +5: SQL generation with upsertsWriteNullValues leaves absent companion values out of the update clause
00:00 +6: SQL generation with upsertsWriteNullValues does not affect DoNothing clauses
00:00 +7: against a real database keeps stale values by default
00:00 +8: against a real database writes nulls with upsertsWriteNullValues
00:00 +9: against a real database upserting a new row is unaffected by the option
00:00 +10: All tests passed!

10 passed. The two "with the default options" tests pass in both runs — that's the backwards-compatibility guarantee: with the flag off the emitted SQL is unchanged.

Generator tests

00:00 +0: does not override options by default
00:02 +1: overrides options with the build option
00:02 +2: combines with store_date_time_values_as_text
00:02 +3: All tests passed!

Full suites

Suite	Result
drift	909 passed, 5 skipped, 0 failed
drift_dev	510 passed, 1 skipped, 0 failed
dart analyze --fatal-infos (both)	No issues found
dart format (changed files)	Clean

Scope / notes

• The INSERT half of the statement is untouched; only DO UPDATE SET changes.
• DoNothing and DoUpdate.withExcluded are unaffected — both covered by tests.
• drift3_preview's compiler has the same toColumns(true) in addDoUpdate (drift/lib/src/drift3_preview/src/query_builder/compiler.dart:1096), but there's no DriftDatabaseOptions equivalent in that runtime to read the flag from. I left it alone rather than introducing a database-options concept to the preview package unilaterally — glad to follow up if you'd like it covered, and if so, guidance on where you'd want those options to live would help.