JsonObjectParser rejects valid top-level non-object JSON

[He-Pin]

Description

JsonObjectParser rejects valid JSON input when the top-level structure is not a JSON object. The 256-byte lookup table (outerChars) only tolerates {, [, ], ,, and whitespace at depth 0 outside a string. Any other byte causes a FramingException with the misleading message "Invalid JSON encountered".

Affected code

stream/src/main/scala/org/apache/pekko/stream/impl/JsonObjectParser.scala (lines 170-180):

private val outerChars: Array[Byte] =
  Array.tabulate[Byte](256) { i =>
    i.toByte match {
      case CurlyBraceStart                           => 2 // found object
      case SquareBraceStart | SquareBraceEnd | Comma => 1 // skip
      case b if isWhitespace(b.toByte)               => 1 // skip
      case _                                         => 0 // error
    }
  }

At depth 0, encountering a byte with value 0 triggers 1 / outer (line 142), raising ArithmeticException, which is converted to a FramingException.

Valid JSON inputs that are incorrectly rejected

Input	Why rejected
42 (bare number)	4 → outer=0
"hello" (bare string)	" → outer=0
true / null / false	t/n/f → outer=0
[[1], [2]] (array of arrays)	digits not in skip set
\xEF\xBB\xBF{"a":1} (UTF-8 BOM)	0xEF → outer=0

Documentation inconsistency

JsonFraming.scala line 56 states "a top-level JSON Array can also be understood" — but this only works for arrays of objects, not arrays of primitives or arrays of arrays.

Suggested improvements

1. Make the error message more specific: distinguish between "Invalid JSON" and "Top-level JSON element is not a JSON object"
2. Clarify the documentation to state that JsonFraming.objectScanner only frames JSON objects
3. Consider supporting UTF-8 BOM at the start of input (strip \xEF\xBB\xBF before parsing)

[pjfanning]

The main thing that appears to be missing here is support for JSON that starts with an array bracket [ as opposed to an object bracket {.

Apache Spark and other Big Data tooling prefers Line-Delimited JSON (also called JSON Lines, JSONL, or NDJSON) because it is more splittable than JSON. Should we also consider supporting this format?

[pjfanning]

The best workaround if you suspect that you might have a UTF8 BOM in your JSON is to run it through commons-io BOMInputStream.

[He-Pin]

Maybe this is not a problem? as it's JsonObjectParser, not JsonParser?

[pjfanning]

We have https://pekko.apache.org/docs/pekko-connectors/current/data-transformations/json.html in Pekko Connectors.
I'd be happy enough to treat this pekko-stream issue as a documentation issue and to direct people to Pekko Connectors instead.
That Pekko Connector JSON support may have its own issues though. It relies on an jsonsurfer lib that doesn't really seem active. It does have Jackson under the hood but who's to say that jsonsurfer lib works with newer versions of Jackson.