Skip to content

fix(parser): canonicalize unexpected-token error shape across five sites#241

Merged
davydog187 merged 3 commits into
mainfrom
fix/parser-error-tuple-shapes
May 26, 2026
Merged

fix(parser): canonicalize unexpected-token error shape across five sites#241
davydog187 merged 3 commits into
mainfrom
fix/parser-error-tuple-shapes

Conversation

@davydog187
Copy link
Copy Markdown
Contributor

Summary

Fixes a user-reported parser error rendering bug. The Lua snippet

return { "ok" = 5 }

…previously produced:

Failed to compile Lua!

Parse Error

  (no position information)

  Parse error: {:unexpected_token, {:operator, :assign, %{line: 1, column: 14, byte_offset: 13}}, "Expected ',' or '}' in table"}

Now produces:

Failed to compile Lua!

Parse Error

  at line 1, column 14:

  Expected ',' or '}' in table

   1 │ return { "ok" = 5 }
                   ^

Suggestion:
  In a Lua table constructor, only identifier keys may use the
  `name = value` shorthand. For string or computed keys, bracket
  them: `{ ["key"] = value }` instead of `{ "key" = value }`.

Root cause

Lua.Parser.convert_error/2 pattern-matches the canonical 4-tuple
{:unexpected_token, type, pos, message} and routes it through the
beautified Lua.Parser.Error.format/2 pipeline. Five sites in
lib/lua/parser.ex instead built a malformed 3-tuple where the second
element was the entire peeked token (itself a 3-tuple):

{:error, {:unexpected_token, peek(tokens), msg}}

That falls through to the catch-all converter
(defp convert_error(other, _code)), which inspect/1-dumps the tuple
into the user-facing message with position: nil.

This is the same class of bug A36 (#222) fixed for the lone local
keyword path; five siblings were missed.

Changes

  • lib/lua/parser.ex
    • New private helper unexpected_token_error/2 that destructures
      the head token (real / EOF / empty token list) into the canonical
      error shape.
    • Five malformed sites rewritten to use the helper:
      • parse_for/1for x ? with neither = nor in
      • parse_generic_for/3 — variable list not terminated by in
      • parse_assignment_targets/2 — multi-target assignment with a bad continuation
      • parse_table_fields/2 — table field followed by unexpected token (the reported case)
      • parse_param_list_acc/2 — parameter list with a non-identifier token
    • The existing inline three-branch case in parse_local/1
      (introduced by A36) migrated to the helper for consistency.
    • suggest_for_token_error/2 extended with a tailored hint for the
      { "key" = value } mistake.
  • test/lua/parser/error_test.exs — new describe "unexpected-token sites carry position" block with 9 tests covering the five sites,
    the suggestion content, a multi-line case (line 3), and a negative
    test asserting the table-specific suggestion doesn't leak into
    unrelated errors.
  • test/lua/compiler_exception_test.exs — 1 regression test at the
    public API surface pinning the user's reported reproducer.

Verification

mix format --check-formatted   # clean
mix compile --warnings-as-errors   # clean
mix test                       # 1705 → 1715 (+10), 0 failures

Manual smoke at IEx:

Lua.eval!(~S|return { "ok" = 5 }|)
Lua.eval!("for i 1, 10 do end")
Lua.eval!("for i, j 1, 10 do end")
Lua.eval!("function f(1) end")
Lua.eval!("a, b c = 1, 2")

All render with position, source echo, caret, and a suggestion. None
contain (no position information) or :unexpected_token as raw text.

Out of scope

  • Multi-error recovery (Lua.Parser.Recovery still not wired in —
    same boundary as A36).
  • Backfilling meta on Expr.Property/Expr.Index postfix nodes.
  • Widening Lua.CompilerException's public struct fields.
  • Lexer error positions (already correct).

Plan: .agents/plans/A37-parser-error-tuple-shapes.md

Five sites in lib/lua/parser.ex emitted a 3-tuple
`{:unexpected_token, peek(tokens), msg}` instead of the canonical
4-tuple `{:unexpected_token, type, pos, msg}` the renderer
pattern-matches on. They fell through the catch-all converter,
producing user-visible output like:

    Parse Error
      (no position information)
      Parse error: {:unexpected_token, {:operator, :assign, %{...}}, ...}

Reported by a user running `return { "ok" = 5 }`.

Introduce a private helper `unexpected_token_error/2` that
destructures the head token (real / EOF / empty) into the canonical
shape, and route all five sites plus the existing inline case in
`parse_local/1` (A36) through it.

Also extend `suggest_for_token_error/2` with a tailored hint for the
common `{ "key" = value }` mistake (users reaching for JSON/map
syntax), pointing them at the bracketed form `{ ["key"] = value }`.

Test count: 1705 -> 1715 (+10), 0 failures.

Plan: A37
@davydog187 davydog187 merged commit 59b6d28 into main May 26, 2026
5 checks passed
@davydog187 davydog187 deleted the fix/parser-error-tuple-shapes branch May 26, 2026 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant