Skip to content

feat: support free microflow annotations#319

Open
hjotha wants to merge 2 commits intomendixlabs:mainfrom
hjotha:submit/microflow-free-annotation
Open

feat: support free microflow annotations#319
hjotha wants to merge 2 commits intomendixlabs:mainfrom
hjotha:submit/microflow-free-annotation

Conversation

@hjotha
Copy link
Copy Markdown
Contributor

@hjotha hjotha commented Apr 26, 2026

Part of #332.

Closes #318.

Summary

Adds first-class support for free microflow annotations so standalone notes can roundtrip without being attached to the following activity.

Semantics

An @annotation 'text' is treated as free when it appears before activity-binding metadata such as @position, @caption, @color, @excluded, or @anchor, and a later annotation binds the activity. Activity-bound annotations remain supported as before.

Root cause

The MDL AST had only one annotation text field, so free notes and activity-bound notes collapsed into the same property during parsing and graph building.

Fix

  • Add ActivityAnnotations.FreeAnnotation.
  • Classify free annotations in the visitor using annotation order.
  • Thread free annotations through the microflow builder.
  • Document the syntax and open design questions in a draft proposal.

Tests

Added parser and builder coverage for free-versus-attached annotation order plus a doctype fixture.

Documentation

  • docs/11-proposals/PROPOSAL_microflow_free_annotation.md
  • docs/11-proposals/README.md
  • docs/01-project/MDL_QUICK_REFERENCE.md
  • .claude/skills/mendix/write-microflows.md
  • mdl-examples/doctype-tests/free_annotation.test.mdl

Validation

  • make build
  • make test
  • make lint-go
  • make test-integration
  • ./bin/mxcli check mdl-examples/doctype-tests/free_annotation.test.mdl

Agentic Code Testing

  • Skills and language reference updated for the new annotation distinction.
  • Example fixture passes mxcli check.

Test plan

  • Free annotation order parses into FreeAnnotation.
  • Activity-bound annotations still attach normally.
  • Example fixture checks cleanly.

Symptom: standalone Mendix annotation notes that describe emits immediately before an activity were parsed as activity annotations and became attached to that activity on exec.

Root cause: the MDL annotation model only had one annotation text field, so order-sensitive free notes and activity-bound notes were collapsed into the same AST property.

Fix: add a FreeAnnotation field, classify annotations before activity metadata as free-floating when later metadata binds the activity, and preserve the free note when building the microflow graph.

Tests: added parser and builder coverage for free-versus-attached annotation order, documented the feature with a draft proposal, quick-reference entry, skill guidance, and a doctype fixture checked with mxcli.
@github-actions
Copy link
Copy Markdown

AI Code Review

We are reviewing a pull request that adds support for free microflow annotations.

Let's go through the checklist:

Overlap & duplication

  • The PR introduces a new feature: free microflow annotations. We don't see any existing proposal for this in the provided list (the proposals list in the context is empty, but note the PR adds a new proposal file).
  • The PR adds a new doctype test file: mdl-examples/doctype-tests/free_annotation.test.mdl and updates the visitor and executor tests.
  • We don't see duplication of existing implementations.

MDL syntax design

The PR does not add new MDL syntax (keywords or statement types). It only changes the interpretation of existing @annotation based on context (order relative to activity-binding metadata).

  • The existing @annotation keyword is used, so no new verb is introduced.
  • The syntax remains: @annotation 'text' (so it uses the standard property format for annotations).
  • The change is purely semantic (based on order) and does not alter the MDL grammar. Therefore, we don't have to check for new grammar rules, but note that the visitor logic is updated to classify the annotation as free or attached.

However, note that the PR updates the quick reference to add a line for "Free annotation". This is documentation only.

Full-stack consistency

Since this is a change in the interpretation of an existing annotation (not a new MDL statement), we don't expect:

  • New grammar rules (the grammar for @annotation already exists) -> so no change to MDLParser.g4 or MDLLexer.g4.
  • No new AST node type? Actually, the AST node type ActivityAnnotations is extended with a new field FreeAnnotation. This is in mdl/ast/ast_microflow.go.
  • The visitor is updated in mdl/visitor/visitor_microflow_statements.go to set either AnnotationText or FreeAnnotation based on context.
  • The executor is updated in multiple files to handle the free annotation:
    • mdl/executor/cmd_microflows_builder_annotations.go: to carry forward the free annotation in pendingAnnotations.
    • mdl/executor/cmd_microflows_builder_graph.go: to attach free annotations when encountered and to clear the free annotation field after attaching (to avoid double attachment).
    • Also, there are tests for the builder and visitor.

We note that the feature does not require:

  • LSP wiring (because it doesn't add new formatting, diagnostics, or navigation targets that are new to the language server).
  • DESCRIBE roundtrip: The feature is about preserving free annotations when roundtripping (MDL -> AST -> BSON -> MDL). The PR includes a doctype test and the executor changes to ensure that free annotations are not attached to the next activity. We should check if the DESCRIBE command (which is part of the executor) would output the free annotation in the correct place. However, note that the PR does not explicitly mention changes to the DESCRIBE command. But the feature is about roundtripping, so we assume that the DESCRIBE command (which uses the same AST and executor in reverse) would work. However, the PR does not show changes to the DESCRIBE command. But note that the feature is implemented in the microflow builder (which is used when executing MDL to modify the model). The DESCRIBE command (which reads the model and produces MDL) would need to know how to output free annotations.

Looking at the changes, we see that the executor changes are in the builder (for creating/updating microflows). The DESCRIBE command is likely in a different part of the codebase (probably in the api/ or mdl/executor/ for reading). However, the PR does not show changes to the DESCRIBE command. But note that the PR adds a doctype test and runs mxcli check on it, which only checks syntax. We would need to see if the DESCRIBE command is updated to output free annotations correctly.

However, the PR description says: "Adds first-class support for free microflow annotations so standalone notes can roundtrip without being attached to the following activity." and the validation includes ./bin/mxcli check mdl-examples/doctype-tests/free_annotation.test.mdl (which is just a syntax check).

But note: the PR also adds tests in the visitor and executor that check the AST and the built flow graph. We don't see a test that does a full roundtrip (MDL -> model -> MDL) for free annotations. However, the PR does mention that the feature is for roundtrip.

Given the checklist, we note that the feature is an existing MDL feature (annotation) with a change in semantics. Therefore, we don't expect new grammar or AST node types (though we do have an AST field addition). The visitor and executor are updated. We don't see LSP or DESCRIBE changes, but note that the DESCRIBE command might be unaffected because it already outputs annotations as @annotation before the activity. The change is that now we distinguish in the AST whether the annotation is free or attached, and when building the model we attach free annotations as standalone and attached ones as activity annotations. When reading the model back to MDL, we would have to output free annotations as @annotation before the activity-binding metadata (if any) and attached ones after.

However, the PR does not show changes to the reader (which would be in the visitor when reading from the model). But note that the visitor is used for both parsing MDL and for reading from the model? Actually, the visitor in mdl/visitor/ is for parsing MDL into AST. The reading from the model to MDL is done by the api/ package or by the executor/ when doing DESCRIBE?

Looking at the structure, the executor/ package is for executing MDL statements (to change the model). The DESCRIBE command is likely implemented in the executor/ as well (as a statement that reads the model and outputs MDL). We don't see changes to the DESCRIBE statement handler in the diff.

But note: the PR adds a test in visitor_test.go that checks the AST built from MDL. It does not test the roundtrip via DESCRIBE. However, the PR does say that the feature is for roundtrip and includes a doctype test that is checked with mxcli check (syntax only).

Given the checklist, we must check for full-stack consistency for MDL features. Since this is a change in the semantics of an existing feature, we should verify that the DESCRIBE command (which is part of the full stack) works correctly. However, the PR does not show changes to the DESCRIBE command.

But note: the feature is about preserving free annotations when going from MDL to model and back to MDL. The DESCRIBE command is the step that goes from model to MDL. We must ensure that when the model has a free annotation (not attached to any activity), the DESCRIBE command outputs it as an @annotation before the next activity-binding metadata (or at the end if there is no following activity).

We don't see changes in the DESCRIBE command in the diff, but note that the DESCRIBE command might be using the same AST structure when reading from the model. The AST node ActivityAnnotations now has two fields: AnnotationText (for attached) and FreeAnnotation (for free). When reading from the model, we would have to set one of these fields appropriately.

However, the PR does not show changes to the model-to-AST visitor (which would be in the sdk/mpr/ or mdl/backend/mpr/ for reading). But note that the visitor in mdl/visitor/ is only for parsing MDL. The reading from the model to AST is done by a different visitor (likely in the sdk/mpr/ package when deserializing BSON).

Therefore, we must check that the BSON reader (in sdk/mpr/parser.go or similar) correctly sets the FreeAnnotation or AnnotationText fields when reading from the model. The PR does not show changes to the BSON reader.

This is a potential gap: the PR only changes the writer (MDL -> model) but not the reader (model -> MDL). However, note that the feature is about roundtripping: we want to preserve free annotations when we do MDL -> model -> MDL. The PR changes the writer so that free annotations are not attached to the next activity. But when reading the model back, we must be able to distinguish a free annotation (standing alone) from an attached one (attached to the next activity).

Without changes to the reader, when we read a free annotation from the model, it would be parsed as an attached annotation (because the old logic treated every annotation as attached to the next activity). Then when we write it back as MDL, we would put it as an attached annotation (after any activity-binding metadata) which would change the meaning.

Therefore, we must also update the reader to set the FreeAnnotation field when we see an annotation that is not attached to an activity (i.e., when it is not followed by activity-binding metadata for the same activity).

However, note that the PR does not show any changes to the reader. This is a critical issue for roundtripping.

But wait: the PR description says: "The MDL AST had only one annotation text field, so free notes and activity-bound notes collapsed into the same property during parsing and graph building." and the fix includes:

  • Add ActivityAnnotations.FreeAnnotation.
  • Classify free annotations in the visitor using annotation order.
  • Thread free annotations through the microflow builder.

It does not mention the reader. However, note that the visitor mentioned is the MDL visitor (for parsing MDL). The reader (for BSON) is a different piece.

Given the above, we suspect that the reader might not be updated. However, let's look at the diff again: we don't see any changes to sdk/mpr/ or mdl/backend/mpr/ in the provided diff.

Therefore, we must flag that the reader (BSON to AST) is not updated to distinguish free and attached annotations, which breaks the roundtrip.

But note: the PR adds a test in cmd_microflows_builder_annotations_test.go that builds a flow graph from AST and checks that free annotations are not attached. This tests the writer (AST -> model). We don't see a test for the reader (model -> AST).

However, the PR does say that the feature is for roundtrip and includes a doctype test. The doctype test is only checked for syntax (with mxcli check). We would need to see if there is an integration test that does a full roundtrip.

Given the checklist, we must check for full-stack consistency. Since the feature requires changes in both the writer and the reader, and we only see the writer, we have a problem.

Test coverage

  • The PR adds new test files:
    • mdl-examples/doctype-tests/free_annotation.test.mdl (doctype test)
    • Updates to mdl/executor/cmd_microflows_builder_annotations_test.go (two new tests)
    • Updates to mdl/visitor/visitor_test.go (two new tests)
  • We don't see any integration test that does a full roundtrip (MDL -> model -> MDL) for free annotations, but the doctype test is a step in that direction.

Security & robustness

  • No obvious issues.

Scope & atomicity

  • The PR is scoped to one feature: free microflow annotations.

Code quality

  • The changes appear consistent: we see the same pattern of checking for activity-binding metadata and setting the free annotation field in multiple places.

Given the above, we have a Critical Issue regarding the roundtrip: the reader (BSON to AST) is not updated to distinguish free and attached annotations. Without this, when we read a model that contains a free annotation, it will be misinterpreted as an attached annotation, and then when written back as MDL it will be placed incorrectly (as attached) altering the diagram.

However, note that the PR might be assuming that the reader already works because the BSON structure for free annotations and attached annotations is different? Let's think:

In the Mendix model, a free annotation is a separate object (standalone) while an attached annotation is a property of an activity. Therefore, when reading the model, we should be able to tell by the BSON structure whether an annotation is free (a standalone object) or attached (a field in an activity object).

If that is the case, then the reader might already be setting the correct field? But note the PR says: "The MDL AST had only one annotation text field, so free notes and activity-bound notes collapsed into the same property during parsing and graph building." This suggests that the AST did not have a way to distinguish, so the reader must have been putting both kinds of annotations into the same field (AnnotationText).

Therefore, we must update the reader to put free annotations in the FreeAnnotation field and attached ones in the AnnotationText field.

Since we don't see changes to the reader in the diff, we must flag this as a critical issue.

But wait: the PR might be only about the writer? And the reader is already correct? Let me re-read the PR description:

"The MDL AST had only one annotation text field, so free notes and activity-bound notes collapsed into the same property during parsing and graph building."

This implies that the problem was in the AST (which had only one field) and that the parser (MDL to AST) and the graph builder (AST to model) were collapsing them. The fix is to split the AST field into two and then use the order in the MDL to decide which one to set when parsing MDL.

When reading from the model, we would have to look at the BSON to decide: if the annotation is a standalone object, then it's free; if it's a property of an activity, then it's attached.

The PR does not show changes to the reader, but note that the reader might already be correctly setting the annotation text to the AnnotationText field (because that's the only field that existed). Now we have two fields, so we must update the reader to set the appropriate field.

Without seeing the reader code, we cannot be sure. However, given that the PR does not show any changes to the reader, and the problem was that the AST had only one field, we suspect the reader is not updated.

Therefore, we raise a critical issue.

But note: the PR adds a test in the visitor that builds the AST from MDL and checks the AnnotationText vs FreeAnnotation fields. This tests the MDL->AST direction. We don't see a test for the model->AST direction.

Given the checklist, we must check for full-stack consistency. The feature requires changes in both directions. We only see changes in the MDL->AST and AST->model directions. We are missing the model->AST direction.

Recommendation

We must request changes to update the reader (BSON to AST) to correctly set the FreeAnnotation and AnnotationText fields based on whether the annotation is a standalone object or a property of an activity.

However, note that the PR might be incomplete in the diff we are seeing? But we are only given the diff in the PR.

Alternatively, we might be missing that the reader is in a different part of the code that is not shown in the diff. But the diff provided does not include any changes to sdk/mpr/ or mdl/backend/mpr/.

Given the information we have, we must flag this as a critical issue.

But let's double-check the diff: we see changes in:

  • .claude/skills/mendix/write-microflows.md
  • docs/01-project/MDL_QUICK_REFERENCE.md
  • docs/11-proposals/PROPOSAL_microflow_free_annotation.md
  • docs/11-proposals/README.md
  • mdl-examples/doctype-tests/free_annotation.test.mdl
  • mdl/ast/ast_microflow.go
  • mdl/executor/cmd_microflows_builder_annotations.go
  • mdl/executor/cmd_microflows_builder_annotations_test.go
  • mdl/executor/cmd_microflows_builder_graph.go
  • mdl/visitor/visitor_microflow_statements.go
  • mdl/visitor/visitor_test.go

No changes to the reader.

Therefore, we have:

Critical Issues

  • The reader (BSON to AST) is not updated to distinguish free and attached annotations, breaking the roundtrip for free annotations when reading from an existing model.

Moderate Issues

  • None noted.

Minor Issues

  • The PR updates the quick reference to document free annotation, but note that the quick reference already had a line for Annotation. We are adding a second line. This is acceptable.

What Looks Good

  • The changes to the visitor, executor, and AST are consistent and well-tested for the MDL->AST and AST->model directions.
  • The doctype test and unit tests are added.
  • The documentation and skills are updated.

Recommendation

We must request changes to update the BSON reader (in sdk/mpr/parser_microflow.go or similar) to set the FreeAnnotation field when reading a standalone annotation object and the AnnotationText field when reading an annotation property of an activity.

However, note that the PR might be relying on the fact that the reader already works because the BSON structure is different and the existing code already puts the annotation text in the correct place? But the PR description says the AST had only one field, implying that the reader was putting both kinds into that one field. Now we have two fields, so we must split them.

Without the reader change, when we read a model that has a free annotation, the FreeAnnotation field will be empty and the AnnotationText field will be set (because the old reader put it there). Then when we write the model back as MDL, we will see AnnotationText set and treat it as an attached annotation (because the writer logic: if AnnotationText is set, we treat it as attached unless we see later activity metadata? But note: the writer logic now checks for FreeAnnotation first? Actually, in the visitor when building the AST from MDL, we set FreeAnnotation only if there is no prior activity metadata and there is a later activity annotation. When writing from AST to model, we attach the FreeAnnotation as a standalone and the AnnotationText as attached


Automated review via OpenRouter (Nemotron Super 120B) — workflow source

The PR review questioned whether free microflow annotations were preserved when
reading an existing model back through DESCRIBE.

That path does not use ActivityAnnotations: parsed Microflows$Annotation objects
and AnnotationFlow links are already split by collectFreeAnnotations and
buildAnnotationsByTarget. Extract the free-annotation prefixing into a small
helper and add a regression test that starts from a model object collection with
one standalone note and one attached note.

Tests:
- go test ./mdl/executor -run 'TestPrependFreeAnnotationLines_ModelAnnotationsStayFree|TestFreeAnnotationBeforePositionStaysUnattached'
- make build
- make test
- make lint-go
- make test-integration
@hjotha
Copy link
Copy Markdown
Contributor Author

hjotha commented Apr 26, 2026

Addressed the review concern about the model-to-DESCRIBE side of free annotations in follow-up commit 98729a4.

Clarification: this path does not use ActivityAnnotations; existing MPR/BSON parsing produces Microflows$Annotation objects plus AnnotationFlow links. DESCRIBE separates standalone notes from attached notes via collectFreeAnnotations and buildAnnotationsByTarget.

I extracted the free-annotation prefixing into prependFreeAnnotationLines and added TestPrependFreeAnnotationLines_ModelAnnotationsStayFree, which starts from a model object collection containing both a standalone annotation and an attached annotation. The test verifies that only the standalone note is emitted as a free @annotation before activity metadata, while the attached note remains attached.

Validation rerun:

  • go test ./mdl/executor -run 'TestPrependFreeAnnotationLines_ModelAnnotationsStayFree|TestFreeAnnotationBeforePositionStaysUnattached'
  • make build
  • make test
  • make lint-go
  • make test-integration

@github-actions
Copy link
Copy Markdown

AI Code Review

Critical Issues

  • None found.

Moderate Issues

  • None found.

Minor Issues

  • AST Limitation: The ActivityAnnotations AST node only stores a single free annotation string (FreeAnnotation field), meaning multiple consecutive free annotations before activity-binding metadata would overwrite each other (only the last retained). This mirrors the existing limitation for attached annotations (AnnotationText field) but should be acknowledged as a known constraint. The proposal and tests only cover single annotations, so this is acceptable for scope but worth noting for future enhancement.
  • Test Redundancy: The visitor test TestAnnotationBeforePositionIsFreeFloating and TestAnnotationAfterPositionStaysAttached duplicate logic already covered in cmd_microflows_builder_annotations_test.go and visitor-level tests. While not harmful, consolidating annotation classification tests in one location would improve maintainability.

What Looks Good

  • Full Pipeline Compliance: Changes correctly flow through grammar (no new tokens needed, existing annotation rule reused), AST (FreeAnnotation field added), visitor (order-sensitive classification logic), executor (free annotation handling in flow graph building and DESCRIBE output), and roundtrip validation (doctest passes mxcli check).
  • Semantic Clarity: The solution correctly distinguishes free vs. attached annotations based on annotation order relative to activity-binding metadata (@position, @caption, etc.), matching the proposal semantics.
  • Documentation: All relevant docs updated (skills, quick reference, proposal, doctest) with clear examples and open questions noted.
  • Test Coverage: Comprehensive unit tests for visitor classification, builder annotation handling, and DESCRIBE output; doctest validates end-to-end behavior.
  • No Syntax Violations: Uses existing @annotation keyword appropriately; no new verbs, qualified names, or property format changes. Colon/AS guidelines not applicable as no new property lists/mappings introduced.
  • Backward Compatibility: Activity-bound annotations unchanged; free annotations only affect cases where annotation precedes binding metadata.

Recommendation

Approve the PR. The implementation correctly solves the roundtrip issue for free microflow annotations with minimal, focused changes. Minor issues noted do not block merge but should be tracked for future refinement (e.g., supporting multiple free annotations via slice in AST). All checklist items satisfied.


Automated review via OpenRouter (Nemotron Super 120B) — workflow source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MDL cannot distinguish free microflow notes from activity-bound annotations

2 participants