Reject nonnumeric list elements in strict text and codec conversion

README.md

@@ -328,6 +328,10 @@ result is read from a `buffer`; one stored element never changes meaning with
 the operator that consumes it. Structural buffer equality and scalar reductions
 normalize each input by this rule too, as do mixed buffer/list materialization
 and numeric byte or sample conversion. A raised read stops subsequent work.
+`str_from_bytes` and the `inflate`/`deflate` codecs (including their `zlib_*` forms) reject nonnumeric list elements with a builtin-named
+`type_mismatch` error by default; `EIGS_STRICT=0` retains numeric-zero
+substitution. `str_from_bytes` stops at numeric NUL, without inspecting later
+elements; byte codecs consume the complete list.
 
 Fixed- or optional-shape builtin argument lists reject surplus outer elements in strict mode with a catchable `type_mismatch` error naming the builtin and maximum width. The check precedes the call's mutation, I/O and tape effects. `EIGS_STRICT=0` keeps each builtin's legacy result, including existing null/error stand-ins. Scalar overloads, lists used as data and genuinely variadic arguments keep their documented meaning.
 

changes/fixed/1590-strict-byte-list-elements.md

@@ -0,0 +1 @@
+- Reject nonnumeric list elements in strict byte conversions while preserving compatibility-mode substitution and numeric NUL termination.

docs/BUILTINS.md

@@ -203,7 +203,7 @@ For serialization: reconstruct strings/floats from raw bytes (the inverse of an
 
 | Name | Signature | Description |
 |------|-----------|-------------|
-| `str_from_bytes` | `str_from_bytes of <list\|buffer>` | Build a string from raw byte values (0–255) — the list form of `chr` (`chr of n` == `str_from_bytes of [n]` for 1–255), inverting an `ord`-over-bytes loop. Strings are NUL-terminated: a `0` byte ends the string — keep NUL-bearing binary in a buffer. |
+| `str_from_bytes` | `str_from_bytes of <list\|buffer>` | Build a string from raw byte values (0–255) — the list form of `chr` (`chr of n` == `str_from_bytes of [n]` for 1–255), inverting an `ord`-over-bytes loop. Nonnumeric list elements raise `type_mismatch` by default and convert to numeric zero under `EIGS_STRICT=0`. Strings are NUL-terminated: a numeric byte that converts to `0` ends conversion, and later elements are not inspected — keep NUL-bearing binary in a buffer. |
 | `f64_to_bytes` | `f64_to_bytes of x` | List of 8 ints: the big-endian IEEE-754 encoding of double `x` (network byte order, portable across host endianness). A non-number raises by default; under `EIGS_STRICT=0` it encodes as `0.0` (#971). |
 | `f64_from_bytes` | `f64_from_bytes of <list\|buffer>` | Decode a double from the first 8 big-endian IEEE-754 bytes. Inverse of `f64_to_bytes`. A `NaN` bit pattern raises a `value` error by default; under `EIGS_STRICT=0` it collapses to `0` and sets `math_flags.invalid` (#971). |
 
@@ -227,10 +227,10 @@ Requires the `zlib` build (`make zlib`, `-DEIGENSCRIPT_EXT_ZLIB=1
 
 | Name | Signature | Description |
 |------|-----------|-------------|
-| `inflate` | `inflate of <list\|buffer>` | Raw DEFLATE decompression (windowBits −15) — the ZIP member format, so `.xlsx`/`.ods` entries are readable. Dual of `deflate`. |
-| `deflate` | `deflate of <list\|buffer>` | Raw DEFLATE compression (windowBits −15, default level). Dual of `inflate`. |
-| `zlib_inflate` | `zlib_inflate of <list\|buffer>` | Wrapped decompression with windowBits 15+32: auto-detects **zlib AND gzip** headers — this is what makes plain `.gz` files readable (`read_bytes of path` then `zlib_inflate`). Dual of `zlib_deflate`. |
-| `zlib_deflate` | `zlib_deflate of <list\|buffer>` | zlib-wrapped compression (RFC 1950 header, default level). Dual of `zlib_inflate`. |
+| `inflate` | `inflate of <list\|buffer>` | Raw DEFLATE decompression (windowBits −15) — the ZIP member format, so `.xlsx`/`.ods` entries are readable. Dual of `deflate`. Byte lists must be numeric by default; `EIGS_STRICT=0` converts nonnumbers to zero. |
+| `deflate` | `deflate of <list\|buffer>` | Raw DEFLATE compression (windowBits −15, default level). Dual of `inflate`. Byte lists must be numeric by default; `EIGS_STRICT=0` converts nonnumbers to zero. |
+| `zlib_inflate` | `zlib_inflate of <list\|buffer>` | Wrapped decompression with windowBits 15+32: auto-detects **zlib AND gzip** headers — this is what makes plain `.gz` files readable (`read_bytes of path` then `zlib_inflate`). Dual of `zlib_deflate`. Byte lists must be numeric by default; `EIGS_STRICT=0` converts nonnumbers to zero. |
+| `zlib_deflate` | `zlib_deflate of <list\|buffer>` | zlib-wrapped compression (RFC 1950 header, default level). Dual of `zlib_inflate`. Byte lists must be numeric by default; `EIGS_STRICT=0` converts nonnumbers to zero. |
 
 ### JSON
 

docs/COMPARISON.md

@@ -243,6 +243,10 @@ association contracts remain unchanged. Mixed buffer/list tensor operations and
 numeric byte/sample/device conversions also normalize reads and stop on the
 first error. Byte conversion truncates and wraps modulo 256 after normalization.
 Raw copies, typed serialization and buffer-only work areas preserve their data.
+`str_from_bytes` and the `inflate`/`deflate` codecs (including their `zlib_*` forms) reject nonnumeric list elements with a builtin-named
+`type_mismatch` error by default; `EIGS_STRICT=0` retains numeric-zero
+substitution. `str_from_bytes` stops at numeric NUL and does not inspect later
+elements, while compression codecs consume and validate the complete list.
 
 One place EigenScript is louder than NumPy: an out-of-range index in `gather`
 **raises** rather than answering a stand-in, on both containers — NumPy's

docs/SPEC.md

@@ -331,6 +331,11 @@ consequences are contracts you can rely on:
     `file_exists` of a missing path `0` — is unchanged in both modes.
   Overflow saturation is the same in both modes. (Division and modulo by zero
   raise in *both* modes — no defined value.)
+  `str_from_bytes` and the `inflate`/`deflate` codecs (including their
+  `zlib_*` forms) likewise reject a nonnumeric list element with a
+  builtin-named `type_mismatch` error by default; `EIGS_STRICT=0` retains the
+  numeric-zero substitution. `str_from_bytes` ends at a numeric byte that
+  converts to NUL, so elements after that terminator are not inspected.
 - **Integer bitwise ops act on int64, exact past 2^32.** `&` `|` `^` `~` `<<`
   `>>` and their `bit_*` builtin forms interpret operands as 64-bit integers, so
   `1 << 40` is exact where an f64 mantissa alone would not help. This is the

docs/llms.txt

@@ -212,6 +212,10 @@ sqrt(2) = 1.414213562373095
   byte/sample/device conversions normalize their buffer inputs too and stop at
   the first raised read. Numeric bytes truncate and wrap modulo 256 afterward;
   raw copies, serialization and buffer-only work areas preserve representation.
+  `str_from_bytes` and the `inflate`/`deflate` codecs (including their `zlib_*` forms) reject nonnumeric list elements with a builtin-named
+  `type_mismatch` error by default; `EIGS_STRICT=0` retains numeric-zero
+  substitution. `str_from_bytes` stops at numeric NUL without inspecting later
+  elements; byte codecs consume the complete list.
 - Hex integer literals: `0xFF`, `0X10` (digits only). Hex-float forms
   (`0x1p4`) are parse errors. No `mod` keyword — the operator is `%`.
 - f-strings interpolate: `f"step {i}: {report of x}"`. Escapes: `\n \t \r \\ \" \{ \}`. Each `{…}` converts with the builtin `str`, even if you rebind `str`.

src/builtins_buf.c

@@ -188,6 +188,22 @@ Value* builtin_buf_from_list(Value *arg) {
     return v;
 }
 
+/* Strict list-byte validation shared by text and codec conversion. Text
+ * conversion stops at its first numeric NUL, so later elements are not part
+ * of that conversion; codec conversion consumes the complete list. */
+static int strict_numeric_byte_list(Value *arg, const char *who, int nul_ends) {
+    if (!g_strict || !arg || arg->type != VAL_LIST) return 1;
+    for (int i = 0; i < arg->data.list.count; i++) {
+        Value *item = arg->data.list.items[i];
+        if (!item || item->type != VAL_NUM) {
+            rt_error(EK_TYPE, 0, "%s: expected numeric byte values", who);
+            return 0;
+        }
+        if (nul_ends && finite_num_to_byte(item->data.num) == 0) break;
+    }
+    return 1;
+}
+
 /* str_from_bytes of <list|buffer of byte ints> → string of those raw bytes.
  * Reconstructs a native string from its bytes (the inverse of an `ord` loop);
  * the list form of scalar `chr` (chr of n == str_from_bytes of [n] for
@@ -205,6 +221,7 @@ Value* builtin_str_from_bytes(Value *arg) {
     } else {
         ARG_GUARD(1, "str_from_bytes", "a list or buffer of byte values", make_str(""));
     }
+    if (!strict_numeric_byte_list(arg, "str_from_bytes", 1)) return make_null();
     char *s = xcalloc((size_t)(n > 0 ? n : 0) + 1, 1);
     int len = 0;
     for (int i = 0; i < n; i++) {
@@ -278,8 +295,9 @@ Value* builtin_f64_from_bytes(Value *arg) {
  * try/catch instead of dying on "undefined variable".
  *
  * Byte representation mirrors read_bytes/write_bytes exactly: input is
- * a list of ints 0-255 (values taken mod 256, non-numbers read as 0)
- * or a VAL_BUFFER; output is always a fresh list of ints 0-255.
+ * a list of ints 0-255 (values taken mod 256) or a VAL_BUFFER; output is
+ * always a fresh list of ints 0-255. Strict mode rejects nonnumeric list
+ * elements; compatibility mode retains their numeric-zero conversion.
  *
  * inflate/deflate are the RAW DEFLATE pair (windowBits -15) — the ZIP
  * member format, so .xlsx/.ods entries are readable. zlib_inflate/
@@ -317,6 +335,7 @@ static int zlib_bytes_arg(Value *arg, const char *who,
                  who, val_type_name(arg ? arg->type : VAL_NULL));
         return 0;
     }
+    if (!strict_numeric_byte_list(arg, who, 0)) return 0;
     unsigned char *b = xmalloc((size_t)(n > 0 ? n : 1));
     for (int i = 0; i < n; i++) {
         double dv = items ? (items[i] && items[i]->type == VAL_NUM ? items[i]->data.num : 0.0)

tests/sections/42p-strict-byte-conversion.sh

@@ -0,0 +1,14 @@
+echo "[42p] Strict byte-list conversion (#1590)"
+SB_OUTPUT=$(bash "$TESTS_DIR/test_strict_byte_conversion.sh" 2>&1); SB_RC=$?
+SB_PASS=$(printf '%s\n' "$SB_OUTPUT" | grep -c "  PASS:" || true)
+SB_FAIL=$(printf '%s\n' "$SB_OUTPUT" | grep -c "  FAIL:" || true)
+TOTAL=$((TOTAL + SB_PASS + SB_FAIL))
+PASS=$((PASS + SB_PASS))
+FAIL=$((FAIL + SB_FAIL))
+printf '%s\n' "$SB_OUTPUT"
+if ! rc_ok "$SB_RC" "$SB_OUTPUT" || { [ "$SB_PASS" -ne 12 ] && [ "$SB_PASS" -ne 23 ]; } || [ "$SB_FAIL" -ne 0 ]; then
+    FAIL=$((FAIL + 1))
+    TOTAL=$((TOTAL + 1))
+    echo "  FAIL: strict byte-list conversion summary (rc=$SB_RC)"
+fi
+echo ""

tests/test_strict_byte_conversion.sh

@@ -0,0 +1,116 @@
+#!/bin/bash
+# #1590: list-backed byte conversions reject nonnumeric elements by default,
+# retain the exact compatibility conversion with EIGS_STRICT=0, and preserve
+# str_from_bytes' terminating numeric NUL contract. Probe codec availability
+# so the zlib CI suite automatically exercises the codec rows.
+
+set -u
+TESTS_DIR="$(cd "$(dirname "$0")" && pwd)"
+EIGS="${EIGS_BIN:-$(cd "$TESTS_DIR/.." && pwd)/src/eigenscript}"
+PASS=0
+FAIL=0
+TMP=$(mktemp /tmp/eigs_strict_bytes_XXXXXX.eigs)
+trap 'rm -f "$TMP"' EXIT
+
+byte_tmo() {
+    if command -v timeout >/dev/null 2>&1; then timeout 10 "$@"
+    elif command -v gtimeout >/dev/null 2>&1; then gtimeout 10 "$@"
+    else "$@"
+    fi
+}
+
+run() {
+    local name="$1" mode="$2" expected_rc="$3" expected="$4" program="$5"
+    printf '%s\n' "$program" > "$TMP"
+    local out rc
+    if [ "$mode" = unset ]; then
+        out=$(byte_tmo env -u EIGS_STRICT "$EIGS" "$TMP" 2>&1); rc=$?
+    else
+        out=$(EIGS_STRICT="$mode" byte_tmo "$EIGS" "$TMP" 2>&1); rc=$?
+    fi
+    if [ "$rc" = "$expected_rc" ] && printf '%s' "$out" | grep -qF -- "$expected" \
+       && [[ "$out" != *"Sanitizer:"* && "$out" != *"runtime error:"* ]]; then
+        echo "  PASS: $name"
+        PASS=$((PASS + 1))
+    else
+        echo "  FAIL: $name (rc=$rc out='$out')"
+        FAIL=$((FAIL + 1))
+    fi
+}
+
+run "default rejects a mixed byte list" unset 1 \
+    "str_from_bytes: expected numeric byte values" \
+    'print of (str_from_bytes of [65, "x", 66])'
+run "explicit strict rejects after ordinary bytes" 1 1 \
+    "str_from_bytes: expected numeric byte values" \
+    'print of (str_from_bytes of [65, 66, "x"])'
+run "strict byte-list error is catchable and typed" 1 0 "type_mismatch" \
+    'try:
+    print of (str_from_bytes of [65, "x"])
+catch err:
+    print of err.kind'
+run "compatibility mode keeps the legacy truncation" 0 0 "[A]" \
+    'print of f"[{str_from_bytes of [65, "x", 66]}]"'
+run "numeric-only bytes remain accepted" unset 0 "[AB]" \
+    'print of f"[{str_from_bytes of [65, 66]}]"'
+run "empty byte list remains accepted" 1 0 "[]" \
+    'print of f"[{str_from_bytes of []}]"'
+run "genuine numeric NUL still terminates" unset 0 "[A]" \
+    'print of f"[{str_from_bytes of [65, 0, 66]}]"'
+run "elements after numeric NUL are not inspected" 1 0 "[A]" \
+    'print of f"[{str_from_bytes of [65, 0, "not converted"]}]"'
+run "strict non-list diagnostic is unchanged" 1 1 \
+    "str_from_bytes: expected a list or buffer of byte values" \
+    'print of (str_from_bytes of 42)'
+run "compatibility non-list stand-in is unchanged" 0 0 "[]" \
+    'print of f"[{str_from_bytes of 42}]"'
+
+run "numeric buffer conversion remains accepted" unset 0 "[AB]" \
+    'print of f"[{str_from_bytes of (buf_from_list of [65, 66])}]"'
+run "numeric buffer NUL remains a terminator" 1 0 "[A]" \
+    'print of f"[{str_from_bytes of (buf_from_list of [65, 0, 66])}]"'
+
+# Use a tiny valid operation: unexpected failure is a test failure, not a skip.
+printf '%s\n' 'try:
+    payload is deflate of ([])
+    print of "CODECS_ENABLED"
+catch err:
+    print of err.message' > "$TMP"
+codec_probe=$(byte_tmo env -u EIGS_STRICT "$EIGS" "$TMP" 2>&1); codec_rc=$?
+if [ "$codec_rc" != 0 ]; then
+    echo "  FAIL: codec availability probe (rc=$codec_rc out='$codec_probe')"
+    FAIL=$((FAIL + 1))
+elif [ "$codec_probe" = "CODECS_ENABLED" ]; then
+    run "raw codec round-trip accepts tiny valid bytes" unset 0 "[65, 0, 66]" \
+        'print of (inflate of (deflate of [65, 0, 66]))'
+    run "wrapped codec round-trip accepts empty input" 1 0 "[]" \
+        'print of (zlib_inflate of (zlib_deflate of []))'
+    run "default codec conversion rejects nonnumeric byte" unset 1 \
+        "deflate: expected numeric byte values" \
+        'print of (deflate of [65, "x", 66])'
+    run "explicit strict wrapped conversion rejects nonnumeric byte" 1 1 \
+        "zlib_deflate: expected numeric byte values" \
+        'print of (zlib_deflate of [65, "x", 66])'
+    run "compatibility codec conversion preserves numeric-or-zero" 0 0 "[65, 0, 66]" \
+        'print of (inflate of (deflate of [65, "x", 66]))'
+    run "raw inflate rejects a nonnumeric byte" unset 1 \
+        "inflate: expected numeric byte values" \
+        'print of (inflate of [3, "x"])'
+    run "wrapped inflate rejects a nonnumeric byte" 1 1 \
+        "zlib_inflate: expected numeric byte values" \
+        'print of (zlib_inflate of [120, 156, 3, "x", 0, 0, 0, 1])'
+    run "raw inflate compatibility retains zero conversion" 0 0 "[]" \
+        'print of (inflate of [3, "x"])'
+    run "wrapped inflate compatibility retains zero conversion" 0 0 "[]" \
+        'print of (zlib_inflate of [120, 156, 3, "x", 0, 0, 0, 1])'
+    run "raw codecs retain numeric buffer input" 1 0 "[65, 0, 66]" \
+        'print of (inflate of (buf_from_list of (deflate of (buf_from_list of [65, 0, 66]))))'
+    run "wrapped codecs retain numeric buffer input" unset 0 "[65, 0, 66]" \
+        'print of (zlib_inflate of (buf_from_list of (zlib_deflate of (buf_from_list of [65, 0, 66]))))'
+elif [[ "$codec_probe" != "deflate: compiled without zlib support"* ]]; then
+    echo "  FAIL: unexpected codec availability response '$codec_probe'"
+    FAIL=$((FAIL + 1))
+fi
+
+echo "STRICT_BYTE_CONVERSION: $PASS passed, $FAIL failed"
+[ "$FAIL" = 0 ]