Repository navigation
Add OTLP logs support - #439
Merged
Merged
Conversation
Benchmark ResultBenchmark diff with base branchBenchmark result |
tigrannajaryan
force-pushed
the
tigran/logs
branch
from
October 7, 2026 17:23
4484669 to
858b32c
Compare
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Metadata preservation, Java cloning and restart behavior, and benchmark input isolation need correction.
5 open findings
What changed in this PR
Adds OTLP logs to STEF alongside metrics and traces, with Go conversion and generated Go/Java serialization support.
Changes:
- Adds the logs schema, record fields, and severity enum.
- Implements sorted and unsorted conversion and improves nested attribute handling.
- Adds conversion tests, serialization tests, and multipart size comparisons.
| File | Description |
|---|---|
| java/src/main/java/com/example/otelstef/WriterState.java | Registers log encoders and dictionaries. |
| java/src/main/java/com/example/otelstef/SeverityNumber.java | Defines OTLP severity constants. |
| java/src/main/java/com/example/otelstef/ReaderState.java | Registers log decoders and dictionaries. |
| java/src/main/java/com/example/otelstef/LogsWriter.java | Writes log streams. |
| java/src/main/java/com/example/otelstef/LogsReader.java | Reads log streams. |
| java/src/main/java/com/example/otelstef/LogsEncoder.java | Encodes the logs root. |
| java/src/main/java/com/example/otelstef/LogsDecoder.java | Decodes the logs root. |
| java/src/main/java/com/example/otelstef/Logs.java | Defines the Java logs root. |
| java/src/main/java/com/example/otelstef/LogRecordEncoder.java | Encodes log-record fields. |
| java/src/main/java/com/example/otelstef/LogRecordDecoder.java | Decodes log-record fields. |
| java/src/main/java/com/example/otelstef/LogRecord.java | Defines Java log records. |
| java/src/main/java/com/example/otelstef/CommonFieldCounts.java | Tracks log schema field counts. |
| go/pdata/logs/otlp2stef.go | Exposes the conversion interface. |
| go/pdata/logs/otlp2stef_unsorted.go | Implements conversion and sorted grouping. |
| go/pdata/logs/otlp2stef_unsorted_test.go | Tests conversion and ordering. |
| go/pdata/internal/otlptools/otlpval2tef.go | Exposes value conversion and fixes map indexing. |
| go/pdata/internal/otlptools/compare.go | Extends stable attribute and group comparison. |
| go/otel/otelstef/writerstate.go | Registers log encoding state. |
| go/otel/otelstef/severitynumber.go | Defines Go severity constants. |
| go/otel/otelstef/readerstate.go | Registers log decoding state. |
| go/otel/otelstef/logswriter.go | Writes Go log streams. |
| go/otel/otelstef/logswriter_test.go | Adds round-trip and fuzz tests. |
| go/otel/otelstef/logsreader.go | Reads Go log streams. |
| go/otel/otelstef/logs.go | Defines the Go logs root and codecs. |
| go/otel/otelstef/logrecord.go | Defines Go log records and codecs. |
| go/otel/otelstef/common.go | Updates embedded schema and allocation support. |
| go/otel/otel.stef | Adds logs schema and severity enum. |
| docs/benchmarks.html | Publishes logs comparisons and refreshed results. |
| benchmarks/size_test.go | Adds multipart logs size measurements. |
Files not reviewed (9)
- go/otel/otelstef/common.go: Generated file
- go/otel/otelstef/logrecord.go: Generated file
- go/otel/otelstef/logs.go: Generated file
- go/otel/otelstef/logsreader.go: Generated file
- go/otel/otelstef/logswriter.go: Generated file
- go/otel/otelstef/logswriter_test.go: Generated file
- go/otel/otelstef/readerstate.go: Generated file
- go/otel/otelstef/severitynumber.go: Generated file
- go/otel/otelstef/writerstate.go: Generated file
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
tigrannajaryan
force-pushed
the
tigran/logs
branch
5 times, most recently
from
October 7, 2026 23:16
1188af5 to
5bfdc14
Compare
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved cloning, hashing, and grouping errors affect modification tracking and preservation of log metadata.
5 open findings
5 resolved since last review
Files not reviewed (9)
- go/otel/otelstef/common.go: Generated file
- go/otel/otelstef/logrecord.go: Generated file
- go/otel/otelstef/logs.go: Generated file
- go/otel/otelstef/logsreader.go: Generated file
- go/otel/otelstef/logswriter.go: Generated file
- go/otel/otelstef/logswriter_test.go: Generated file
- go/otel/otelstef/readerstate.go: Generated file
- go/otel/otelstef/severitynumber.go: Generated file
- go/otel/otelstef/writerstate.go: Generated file
🧠 Review effort: Balanced
tigrannajaryan
force-pushed
the
tigran/logs
branch
from
October 8, 2026 13:22
9a1d4bb to
c3089e1
Compare
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved restart encoding, clone tracking, hashing, and log ordering defects affect correctness.
4 open findings
5 resolved since last review
Files not reviewed (9)
- go/otel/otelstef/common.go: Generated file
- go/otel/otelstef/logrecord.go: Generated file
- go/otel/otelstef/logs.go: Generated file
- go/otel/otelstef/logsreader.go: Generated file
- go/otel/otelstef/logswriter.go: Generated file
- go/otel/otelstef/logswriter_test.go: Generated file
- go/otel/otelstef/readerstate.go: Generated file
- go/otel/otelstef/severitynumber.go: Generated file
- go/otel/otelstef/writerstate.go: Generated file
🧠 Review effort: Balanced
Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.
tigrannajaryan
force-pushed
the
tigran/logs
branch
from
October 8, 2026 21:04
bb8bc3d to
186ec20
Compare
Extend the OpenTelemetry STEF schema with a Logs root, LogRecord fields, and the complete OTLP severity number enum. Regenerate the Go and Java record, encoder, decoder, reader, writer, dictionary, and state implementations for the new signal. Add a pdata OTLP-to-STEF logs converter with unsorted and sorted modes. Preserve resource, scope, body, attributes, timestamps, severity, trace context, flags, and event names; group equivalent resources and scopes in sorted mode and order records deterministically. Make shared attribute comparison stable across map iteration and support double, byte, map, and nested values. Expose AnyValue conversion for log bodies and fix nested map indexing so all entries are retained. Cover full log-record conversion and sorted grouping, and add the astronomy multipart log fixture plus a size benchmark comparing OTLP with sorted and unsorted STEF under uncompressed and Zstd modes. Tests: CGO_ENABLED=0 go test ./... (go/pdata) Tests: CGO_ENABLED=0 go test -run '^TestLogsMultipart$' -count=1 (benchmarks) Tests: CGO_ENABLED=0 ./gradlew clean test --tests com.example.otelstef.LogsWriterTest (java)
tigrannajaryan
force-pushed
the
tigran/logs
branch
from
October 9, 2026 00:24
186ec20 to
776215d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extend the OpenTelemetry STEF schema with a Logs root, LogRecord fields, and the complete OTLP severity number enum. Regenerate the Go and Java record, encoder, decoder, reader, writer, dictionary, and state implementations for the new signal.
Add a pdata OTLP-to-STEF logs converter with unsorted and sorted modes. Preserve resource, scope, body, attributes, timestamps, severity, trace context, flags, and event names; group equivalent resources and scopes in sorted mode and order records deterministically.
Make shared attribute comparison stable across map iteration and support double, byte, map, and nested values. Expose AnyValue conversion for log bodies and fix nested map indexing so all entries are retained.
Cover full log-record conversion and sorted grouping, and add the astronomy multipart log fixture plus a size benchmark comparing OTLP with sorted and unsorted STEF under uncompressed and Zstd modes.
Tests: CGO_ENABLED=0 go test ./... (go/pdata)
Tests: CGO_ENABLED=0 go test -run '^TestLogsMultipart$' -count=1 (benchmarks)
Tests: CGO_ENABLED=0 ./gradlew clean test --tests com.example.otelstef.LogsWriterTest (java)