Skip to content

fix feature property order to follow the provider schema#538

Open
cportele wants to merge 3 commits into
masterfrom
fix-feature-property-order
Open

fix feature property order to follow the provider schema#538
cportele wants to merge 3 commits into
masterfrom
fix-feature-property-order

Conversation

@cportele

Copy link
Copy Markdown
Contributor

NOTE: This is a change in the core of the feature pipeline and needs broad testing before the release of ldproxy 4.8.

Properties backed by a joined table (objects, object arrays, value arrays, feature references) were emitted after the main-table columns even when declared before them, because the per-feature event buffer placed each property where its tokens first arrive - the provider's per-table order, not the schema order. For XML/GML this produced output that did not match the application schema's element order.

FeatureEventBuffer now re-sorts each buffered feature at flush into the declared schema order: object (and feature-root) children are ordered by their schema position; array elements keep their data order, and the children inside each element are ordered like any other object. The pass runs after the slice transformers, so transform behaviour is unchanged.

Object and array element children are consequently emitted in schema order too (previously source/arrival order); the affected token fixtures are updated. Adds FeatureEventBufferOrderSpec and re-enables the previously disabled "joined value array between main columns" mapping case.

Properties backed by a joined table (objects, object arrays, value arrays, feature references) were emitted after the main-table columns even when declared before them, because the per-feature event buffer placed each property where its tokens first arrive - the provider's per-table order, not the schema order. For XML/GML this produced output that did not match the application schema's element order.

FeatureEventBuffer now re-sorts each buffered feature at flush into the declared schema order: object (and feature-root) children are ordered by their schema position; array elements keep their data order, and the children inside each element are ordered like any other object. The pass runs after the slice transformers, so transform behaviour is unchanged.

Object and array element children are consequently emitted in schema order too (previously source/arrival order); the affected token fixtures are updated. Adds FeatureEventBufferOrderSpec and re-enables the previously disabled "joined value array between main columns" mapping case.
@cportele cportele requested a review from azahnen as a code owner June 18, 2026 18:09
@azahnen

azahnen commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

@cportele
I am hesitating to approve this since it seems to me we would be adding a second sorting logic on top of the (dysfunctional) existing sorting logic. I think we should either fix the existing logic or remove/tighten it and then introduce the new one.

I tried to remember and understand how the existing logic is supposed to work. The basic idea is that sorting happens as part of the buffer inserts using cursors:

  • FeatureEventBuffer has cursors current and currentEnclosing
  • SchemaMapping contains desired property positions (computed in SchemaMappingBase)
  • onObjectStart, onValue etc. in FeatureTokenTransformerMappings call FeatureEventBuffer.next with property positions
  • this moves the cursors to the desired position
  • the following call to FeatureEventBuffer.append inserts the tokens at cursor position
  • FeatureEventBuffer holds tokens in desired order at any time

I did not dig any deeper yet to find the underlying issue. Questions:

  1. Since the new logic uses mapping.getPositionsForTargetPath(path) as well, I assume that part is working as expected? In my memory that was the most tricky and unstable part.
  2. Do you see any inherent flaw in the design of the existing logic, or are we just looking for an implementation error?
  3. I do not see any dependency on insert order regarding joined tables etc in the design idea. If there is such an unwanted dependency, it would have to come from getPositionsForTargetPath, right?
  4. I am not opposed to using the new logic if it works better and is easier to understand. Do you see any performance hits?
  5. That being said, we cannot remove the cursor logic since it is used by slice transformers as well, right? And most likely it is also still needed on inserts for nesting?

@cportele

Copy link
Copy Markdown
Contributor Author

@azahnen - I agree that the right approach is not to put new logic on top of the current, incomplete logic. I just had a hard time to understand the current logic (and its limitations). I will take a deeper look at your questions and come back with answers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants