SmartNull: stay missing through the whole chain

A missing field used to become an empty SmartString at the first method call, committing the chain to a type based on a guess. Now transforms hand the same SmartNull back, so "still missing" travels the whole chain and the caller picks the ending: ->or('n/a') for a value, ->implode() for a collection. Output is unchanged - echoes "", or() still fires - but chains no longer dead-end when string and array methods mix.

map() skips its callback on a missing key, since there's no value to pass it; a NULL value in an existing key still runs the callback. set() takes one argument as SmartString's value-producing set and keeps the write guard for two.

Author: daveedis

CHANGELOG.md

@@ -87,6 +87,12 @@ the docs - IDEs show a strikethrough with the replacement.
 
 ### Behavior changes
 
+- A missing field stays a SmartNull through the whole chain instead of
+  becoming an empty SmartString at the first method call. Same output as
+  before (echoes `""`, `or()` still fires), but chains no longer dead-end:
+  `$row->missing->trim()->implode(', ')` works where it previously threw.
+  `map()` skips its callback on a missing key; NULL values in existing
+  keys still run it.
 - `isset()`, `empty()`, and `??` treat a stored null as missing, matching
   plain PHP arrays: on a NULL column `isset($row->field)` is now false and
   `$row->field ?? 'none'` returns `'none'`. Previously they answered "does
@@ -107,8 +113,10 @@ the docs - IDEs show a strikethrough with the replacement.
   See [UPGRADING.md](UPGRADING.md).
 - All writes to a `SmartNull` throw "Cannot set values on SmartNull": property
   writes (previously created a silent dynamic property that shadowed
-  chaining) and `->set()` (previously discarded the value) now match the
-  existing `['key'] =` guard.
+  chaining) and two-argument `->set($key, $value)` (previously discarded the
+  value) now match the existing `['key'] =` guard. One-argument
+  `->set($value)` is SmartString's set: not a write, it produces that value
+  and ends the chain, like `or()`.
 - Raw-mode arrays no longer answer SmartString methods on missing keys:
   `$row->missing->or('n/a')` on a raw array throws the standard
   undefined-method Error instead of returning an HTML-encoding SmartString.

docs/ai-reference.md

@@ -151,8 +151,11 @@ depends on WHERE you read (changed in 3.0):
 SmartNull behavior: `echo` → `""`; `value()` → null; `count()` → 0;
 `foreach` iterates zero times; `toArray()` → `[]`; `json_encode()` → null;
 SmartArray methods return empty results; SmartString methods (HTML mode)
-behave as on null; guards (`or404()` etc.) FIRE (empty = missing); any
-write to it throws `RuntimeException` ("Cannot set values on SmartNull").
+behave as on null, except transforms return the same SmartNull (chain stays
+missing, accepts value or collection endings) and `map()` skips its callback
+(a NULL value in an existing key still runs it); guards (`or404()` etc.)
+FIRE (empty = missing); one-argument `set($value)` produces that value; all
+other writes throw `RuntimeException` ("Cannot set values on SmartNull").
 It carries the source's mysqli metadata and load handler.
 
 ## Iteration and Keys

docs/internal/design-decisions.md

@@ -18,6 +18,26 @@ in signatures, docblocks, the changelog, and tests.
   the value side, `foreach`/`count()`/`keys()` on the array side. An empty
   SmartArray from `first()` would fatal on `->value()`.
 
+- **SmartNull propagates through SmartString transforms (2026-08-04).** In
+  HTML mode, `__call` tries public SmartString methods first and classifies
+  by result: a still-null result means nothing was produced, so the
+  SmartNull itself returns and the chain stays open for a value or a
+  collection ending. `map()` returns the SmartNull without running its
+  callback, since a missing key has no value to pass (a NULL value in an
+  existing key still runs it). Rejected alternatives: renaming `map()` back
+  to `apply()` (dodges one collision but keeps the name-based routing that
+  broke `map`/`htmlEncode`/`set`, and the next shared name re-breaks it);
+  routing shared names to SmartString by class ("string wins" fixes the
+  three methods but breaks `first()->map()` collection chains the same
+  way); running map's callback with null, the v2 `apply()` behavior (typed
+  callbacks throw TypeError, side effects fire for rows that don't exist,
+  and a value-returning callback turns a collection-shaped SmartNull into
+  a SmartString mid-chain). The classifier is `isNull()`, not
+  `isMissing()`: `isMissing()` counts `""`, which would discard a produced
+  empty string and wrongly re-fire `ifNull()` later in the chain. The
+  `isPublic()` reflection check exists because `method_exists()` reports
+  private methods - that false positive is how `htmlEncode()` broke.
+
 - **Missing-key warnings are rows-only (2026-08-04).** Key access warns
   only on rows inside a result set, where keys are column names and a miss
   is almost always a typo; top-level, derived (indexBy()/column() maps),

src/SmartNull.php

@@ -4,6 +4,7 @@
 namespace Itools\SmartArray;
 
 use Iterator, ArrayAccess, Countable;
+use ReflectionMethod;
 use RuntimeException;
 use Itools\SmartString\SmartString;
 use JetBrains\PhpStorm\Deprecated;
@@ -220,15 +221,23 @@ public function __get(string $name): SmartNull
     /**
      * All writes throw: a SmartNull marks a missing key or empty result, so
      * there is nothing real to write to and the value would be silently lost.
-     * Same guard for property, set(), and array syntax.
+     * Same guard for property syntax, array syntax, and two-argument set().
      */
     public function __set(string $name, mixed $value): void
     {
         $this->throwCannotSet();
     }
 
-    public function set(int|string $key, mixed $value): never
+    /**
+     * One argument is SmartString's set($value): produce that value and end the
+     * chain, like or(). Two arguments is SmartArray's set($key, $value), a
+     * write, and all writes throw (see __set above).
+     */
+    public function set(mixed ...$args): mixed
     {
+        if (count($args) === 1 && $this->useSmartStrings) {
+            return $this->__call('set', $args);
+        }
         $this->throwCannotSet();
     }
 
@@ -240,24 +249,42 @@ private function throwCannotSet(): never
     /**
      * Emulate response methods for SmartArray and SmartString.
      *
-     * Since when we access a non-existent element we don't know if we were expecting a SmartArray or SmartString,
-     * we return this object that can handle both.
+     * A missing key doesn't tell us whether the caller expected a value or a
+     * collection, so this object answers for both. SmartString methods (HTML mode
+     * only) are tried first, and the result decides what comes back: a still-null
+     * result means nothing was produced, so the SmartNull itself returns and the
+     * chain stays open for either ending, ->or('n/a') for a value or ->implode()
+     * for a collection. Produced results (or() fallbacks, int() and other scalars)
+     * return as usual. map() propagates without running its callback: a missing
+     * key has no value to pass it, while a NULL value in an existing key still
+     * runs the callback.
      *
-     * Unknown methods are forwarded too, so they throw the same undefined-method
+     * Everything else delegates to an empty SmartArray/SmartArrayHtml of the same
+     * mode. Unknown methods are forwarded too, so they throw the same undefined-method
      * Error as the rest of the library ("did you mean" hint + caller's file:line).
      *
      * @param $name
      * @param mixed ...$arguments
-     * @return array|false|float|int|SmartString|string|null
+     * @return array|false|float|int|SmartNull|SmartString|string|null
      */
     public function __call($name, array $arguments): mixed
     {
         // SmartString methods only delegate in HTML mode: raw values are plain scalars
         // with no methods, so a miss answers SmartString calls the same way - with the
-        // standard undefined-method Error
-        if ($this->useSmartStrings && !method_exists(SmartArrayBase::class, $name) && method_exists(SmartString::class, $name)) {
-            return SmartString::new(null)->$name(...$arguments);
+        // standard undefined-method Error. The isPublic() check keeps private helpers
+        // out: method_exists() reports them, but they aren't part of the API
+        $isSmartStringMethod = $this->useSmartStrings
+            && method_exists(SmartString::class, $name)
+            && (new ReflectionMethod(SmartString::class, $name))->isPublic();
+
+        if ($isSmartStringMethod) {
+            if ($name === 'map') {
+                return $this;
+            }
+            $result = SmartString::new(null)->$name(...$arguments);
+            return $result instanceof SmartString && $result->isNull() ? $this : $result;
         }
+
         return $this->useSmartStrings
             ? (new SmartArrayHtml([], $this->getInternalProperties()))->$name(...$arguments)
             : (new SmartArray([], $this->getInternalProperties()))->$name(...$arguments);

tests/Unit/SmartNullTest.php

@@ -20,10 +20,14 @@
  * access, and method calls all keep working, and the chain resolves to an
  * empty/null value of the right type for the mode it was born in.
  *
- * Method calls are handled by __call, which routes three ways:
- * - names on SmartArrayBase  -> an empty SmartArray/SmartArrayHtml (mode inherited)
- * - names only on SmartString -> SmartString::new(null)
- * - names on neither          -> the library's undefined-method Error
+ * Method calls are handled by __call. In HTML mode, public SmartString methods
+ * are tried first and the result decides what comes back: value producers like
+ * or() return their fallback, terminals like int() return their scalar, and
+ * transforms like trim() propagate the same SmartNull so the chain stays open
+ * for either a value or a collection ending. map() propagates without running
+ * its callback (a missing key has no value to pass). Everything else delegates
+ * to an empty SmartArray/SmartArrayHtml of the same mode, and unknown names
+ * throw the library's undefined-method Error.
  */
 class SmartNullTest extends SmartArrayTestCase
 {
@@ -224,17 +228,93 @@ public function testDelegatedArraysCarryTheSourceMetadata(string $class): void
     //endregion
     //region Method delegation: SmartString methods
 
-    public function testSmartStringOnlyMethodsDelegateToANullSmartStringInHtmlMode(): void
+    public function testTransformsPropagateTheSameSmartNullInHtmlMode(): void
     {
+        // A missing key stays missing through a chain: there is nothing to
+        // transform, so the SmartNull itself comes back and the chain stays
+        // open for either ending, ->or() for a value or ->implode() for a collection
         $smartNull = $this->smartNullFrom(SmartArrayHtml::class);
 
-        $trimmed = $smartNull->trim();
-        $this->assertInstanceOf(SmartString::class, $trimmed);
-        $this->assertNull($trimmed->value(), 'delegation target is SmartString::new(null)');
+        $transforms = ['trim' => [], 'maxChars' => [5], 'dateFormat' => ['Y-m-d'], 'numberFormat' => [], 'add' => [5]];
+        foreach ($transforms as $method => $args) {
+            $this->assertSame($smartNull, $smartNull->$method(...$args), "->$method() propagates");
+        }
+
+        $this->assertSame('n/a', $smartNull->trim()->maxChars(5)->or('n/a')->value(), 'chain resolves at the end');
+        $this->assertSame('', $smartNull->trim()->implode(', ')->value(), 'collection ending still works after a transform');
+    }
+
+    public function testValueProducersEndTheChainInHtmlMode(): void
+    {
+        $smartNull = $this->smartNullFrom(SmartArrayHtml::class);
 
         $fallback = $smartNull->or('n/a');
         $this->assertInstanceOf(SmartString::class, $fallback);
         $this->assertSame('n/a', $fallback->value());
+        $this->assertSame('n/a', $smartNull->ifNull('n/a')->value());
+    }
+
+    public function testOrThrowThrowsInHtmlMode(): void
+    {
+        $smartNull = $this->smartNullFrom(SmartArrayHtml::class);
+
+        $this->expectException(RuntimeException::class);
+        $this->expectExceptionMessage('no user found');
+
+        $smartNull->orThrow('no user found');
+    }
+
+    public function testMapPropagatesWithoutRunningTheCallbackInHtmlMode(): void
+    {
+        // map() takes a user callback, and a missing key has no value to pass it
+        $smartNull = $this->smartNullFrom(SmartArrayHtml::class);
+        $calls     = 0;
+
+        $result = $smartNull->map(function ($value) use (&$calls) {
+            $calls++;
+            return 'computed';
+        });
+
+        $this->assertSame($smartNull, $result);
+        $this->assertSame(0, $calls, 'the callback never runs on a missing key');
+        $this->assertSame('n/a', $smartNull->map('strtoupper')->or('n/a')->value(), 'chain stays open after map');
+    }
+
+    public function testMapStillRunsOnAKeyThatExistsWithANullValue(): void
+    {
+        // The boundary map() propagation must not cross: NULL is a present value
+        // (SmartString(null), the ordinary path), only an absent key is a SmartNull
+        $row    = SmartArrayHtml::new(['bio' => null]);
+        $result = $row->bio->map(fn($value) => $value ?? 'default');
+
+        $this->assertInstanceOf(SmartString::class, $result);
+        $this->assertSame('default', $result->value());
+    }
+
+    public function testMapOnACollectionShapedSmartNullKeepsCollectionChainsWorking(): void
+    {
+        // first() on an empty result, then a per-element map: the SmartNull
+        // propagates, so collection endings and iteration still degrade gracefully
+        $smartNull = $this->smartNullFrom(SmartArrayHtml::class);
+
+        $result = $smartNull->map(fn($value) => strtoupper((string)$value))->implode(', ');
+        $this->assertInstanceOf(SmartString::class, $result);
+        $this->assertSame('', $result->value());
+    }
+
+    public function testMapDelegatesToAnEmptyArrayInRawMode(): void
+    {
+        // Raw mode has no SmartString delegation: map is SmartArray's per-element
+        // map, which returns an empty array of the same mode
+        $result = $this->smartNullFrom(SmartArray::class)->map(fn($value) => $value);
+
+        $this->assertSame(SmartArray::class, get_class($result));
+        $this->assertSame([], $result->toArray());
+    }
+
+    public function testHtmlEncodeReturnsEmptyStringInHtmlMode(): void
+    {
+        $this->assertSame('', $this->smartNullFrom(SmartArrayHtml::class)->htmlEncode());
     }
 
     public function testSmartStringTypeCastsReturnEmptyScalarsInHtmlMode(): void
@@ -373,9 +453,10 @@ public function testArrayWriteThrowsRuntimeException(string $class): void
     }
 
     #[DataProvider('modeProvider')]
-    public function testSetThrowsTheSameGuardAsArraySyntax(string $class): void
+    public function testTwoArgumentSetThrowsTheSameGuardAsArraySyntax(string $class): void
     {
-        // All writes throw the same guard: property, set(), and array syntax
+        // Two arguments is SmartArray's set($key, $value), a write, and all
+        // writes throw the same guard: property, set(), and array syntax
         $smartNull = $this->smartNullFrom($class);
 
         $this->expectException(RuntimeException::class);
@@ -384,6 +465,28 @@ public function testSetThrowsTheSameGuardAsArraySyntax(string $class): void
         $smartNull->set('key', 'value');
     }
 
+    public function testOneArgumentSetProducesTheValueInHtmlMode(): void
+    {
+        // One argument is SmartString's set($value): not a write, it produces
+        // a new value and ends the chain, like or()
+        $smartNull = $this->smartNullFrom(SmartArrayHtml::class);
+
+        $result = $smartNull->set('fallback');
+        $this->assertInstanceOf(SmartString::class, $result);
+        $this->assertSame('fallback', $result->value());
+
+        $this->assertSame($smartNull, $smartNull->set(null), 'set(null) produces nothing, so the chain stays missing');
+    }
+
+    public function testOneArgumentSetThrowsInRawMode(): void
+    {
+        // Raw mode has no SmartString delegation, so the write guard answers
+        $this->expectException(RuntimeException::class);
+        $this->expectExceptionMessage('Cannot set values on SmartNull - this value came from a missing key or empty result, check ->isNotEmpty() first');
+
+        $this->smartNullFrom(SmartArray::class)->set('value');
+    }
+
     #[DataProvider('modeProvider')]
     public function testPropertyWriteThrowsTheSameGuardAsArraySyntax(string $class): void
     {