Repository navigation
Observer 2.0 core API: nested-array fix, paths with indices, shape metadata, 0 B warm calls - #6
Merged
Merged
Conversation
An array rule returned depth + 1 and the matcher added the depth again, so an Obj under a property's Array looked for its names one or more levels too deep: BlockList left those values in clear and AllowList masked the whole item. An Obj directly under a root Array worked only because the root depth is 0.
The default-policy branch wrote a null directly, so a relative MaskStr, MaskInt, MaskBool, MaskRawValue or ReadStr never saw a null while the same absolute rule did. A null now reaches the relative rules; MaskAny and MaskTag keep it null as documented.
WriteStringValue(ReadOnlySpan<char>), WritePropertyName(ReadOnlySpan<char>), WriteBase64StringValue and WriteNumberValue(double) let a char-based redactor or a hash strategy write without an intermediate string. MaxValueBytes cuts them like any string; a non-finite double is written as a string so a strategy can never fail the call.
MaskTag gains an optional object Key and MaskKind.Custom, so an integration maps its own classification (a compliance taxonomy, a redactor name) to masking without casting MaskKind values. A strategy that does not know the key falls back to the kind; the built-in strategy writes "***" for Custom.
An array item level now stores its position: ToString renders items[2].sku (special names as ['a.b']), and TryGetArrayIndex, IsArrayItem and TryGetPropertyNameUtf8 give zero-allocation access for error paths and strategies. Shape observers now push array items too, so both walkers report the same paths. Name matching is unchanged: an array item is still one level that names never match.
Utf8MaskStrategy.Mask(in Utf8MaskContext, JsonWriter) is now the entry point: the context carries the value, its JSON type, the tag, the options, the property name and the whole path without allocating, so a strategy can apply a value:name discriminator. It forwards to the existing overload by default, which in turn masks like the built-in strategy, so neither has to be overridden. A number split across input segments is copied to a pooled buffer instead of an array.
Members is now a list of JsonShapeProperty (still deconstructs to name and shape) with the JsonPropertyInfo, CLR member, property and declaring type, IsRequired, IsNullable and custom attributes; nodes built from metadata carry their JsonTypeInfo and CLR type. Both nodes and properties have thread-safe annotations for integrations, FromTypeInfo takes an annotate callback, and FindMember looks a property up by its UTF-8 name like the observer. .NET 8 source-generated metadata has no attributes or reference-type nullability; .NET 9 uses JsonPropertyInfo.IsSetNullable and JsonTypeInfo.ElementType.
JsonObserverOptions.PropertyNameCaseInsensitive (default true, as before) switches rule names, PropMatches tests and shape lookups to ordinal matching per call; PropertyPath exposes the mode to custom rules. JsonShapeOptions.PropertyNameCaseInsensitive overrides it for one shape observer, and JsonShapeOptions.FromSerializerOptions takes it from the serializer's options. In exact mode a shape keeps names that differ only in case apart. Name matchers can describe themselves, which Explain will use.
Mask and Read take an in ReadOnlySequence<byte>, so a payload in several buffers, for example from a PipeReader, is masked without being copied into one. A single segment takes the span path; a byte order mark split across segments is still skipped. Values split across segments are copied to pooled buffers instead of arrays. Every masking golden case is replayed in 1-, 3- and 7-byte segments and must match the span output byte for byte; planting a segment-boundary bug in property names fails 61 of those cases.
The bounded writer (with its Utf8JsonWriter), the ignore-nulls writer and the string API's output buffer are kept per thread and reset per call; their arrays still come from the pool and are returned after every call, and a nested call on the same thread gets writers of its own. A writer is reused only while the escaping, indentation and depth settings match. The allocation test now pins 0 bytes per warm call for the span, sequence, ignore-nulls and read paths (240 to 288 bytes before).
JsonObserver.Explain(path, valueKind, options) returns a JsonPathExplanation: the outcome (Unchanged, Masked, Read, Custom, Invalid), the deciding rule, its action and one step per level. It walks the observer's rules with the same matcher the masking pass uses, so absolute, nested, relative and default-policy decisions, case sensitivity and descended unknown containers agree with the output; shape observers explain against their JsonShape. Builders now record a description beside every rule. A golden-payload test checks that every leaf Explain calls Masked is exactly a leaf the mask changed.
CHANGELOG lists the sequence input, zero-allocation bytes API, Explain, classified tags, strategy context, JsonWriter span overloads, path indices, case sensitivity and shape metadata, plus the nested Array/Obj and relative-null fixes. The README gains compiled examples for a custom strategy and for Explain, the new option row and the 0 B allocation figure.
The core API merge fixes rules of an Obj inside a property's Array, so the skill no longer documents it as a known issue. Warm UTF-8 calls with constant or tag rules now allocate 0 B; the skills and docs said ~240 B. The allocation test takes the minimum over five measured rounds, so a one-off tier-up allocation under a loaded test host no longer fails it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges the 2.0 core API work into main.
Obj(...)inside a property'sArray(...)now match (they leaked values in clear underBlockList).Relative(null)no longer throws.PropertyPathcarries array indices (items[2].sku);Utf8MaskContextexposesPropertyName,Path,IsArrayItem.JsonShapemember metadata, case-sensitivity options,ReadOnlySequence<byte>overloads,Explain(path).Full suite: 1578 passed locally (net8/9/10).
🤖 Generated with Claude Code