Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions src/core/jsonl/include/sourcemeta/core/jsonl.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,21 @@
/// @defgroup jsonl JSONL
/// @brief A JSON Lines (https://jsonlines.org) implementation with iterator
/// support. Every non-empty line in a JSONL stream is a complete, valid JSON
/// value, and lines are separated by newline characters (U+000A). Multi-line
/// JSON values are not supported, as per the JSONL specification. Blank and
/// whitespace-only lines are skipped rather than treated as errors. JSON Lines
/// is a convention rather than a formal specification, so tolerating stray
/// blank lines is friendlier to real-world input.
/// value of any type, and lines are separated by newline characters (U+000A),
/// optionally preceded by a carriage return (U+000D). Multi-line JSON values
/// are not supported, as per the JSONL specification.
///
/// JSON Lines and NDJSON (https://github.com/ndjson/ndjson-spec) describe the
/// same format with minor differences, and this implementation accepts a
/// superset of both:
///
/// - Blank and whitespace-only lines are skipped rather than treated as
/// errors. JSON Lines considers them invalid, while NDJSON 3.2 permits
/// ignoring them as long as the behavior is documented
/// - A newline after the last value is optional. JSON Lines makes it a
/// recommendation, while NDJSON 3.1 requires it when serializing
/// - Whitespace is tolerated anywhere around a value, including carriage
/// returns that NDJSON 3.1 only allows right before a newline
///
/// This functionality is included as follows:
///
Expand Down
4 changes: 1 addition & 3 deletions src/core/jsonl/include/sourcemeta/core/jsonl_iterator.h
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,7 @@ namespace sourcemeta::core {

/// @ingroup jsonl
/// A forward iterator to parse JSON documents out of a JSON Lines stream. Blank
/// and whitespace-only lines are skipped rather than treated as errors, as JSON
/// Lines is a convention rather than a formal specification and tolerating
/// stray blank lines is friendlier to real-world input.
/// and whitespace-only lines are skipped rather than treated as errors.
class SOURCEMETA_CORE_JSONL_EXPORT ConstJSONLIterator {
public:
/// Construct an iterator over the JSON documents in a stream.
Expand Down
2 changes: 1 addition & 1 deletion src/core/jsonl/iterator.cc
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ auto ConstJSONLIterator::parse_next() -> JSON {
row.pop_back();
}

// Skip whitespace-only lines
// NDJSON 3.2 states that "The parser MAY silently ignore empty lines"
bool has_content{false};
for (const auto character : row) {
if (character != internal::TOKEN_JSONL_WHITESPACE_SPACE<JSON::Char> &&
Expand Down
Loading