diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl index 24e2612f77..f05ae9bbf7 100644 --- a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl +++ b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl @@ -98,3 +98,10 @@ {"area": "cmd/mxcli", "date": "2026-09-03", "symptom": "`mxcli brain check` reports an entry as MISFILED even though the entry is correct and its anchor points at a real document — the anchor's target is simply of a document type the catalog's `objects` view does not index", "cause": "Misfiling was decided by comparing the shard against the modules of *resolved* anchors. An entry whose only anchor came back NotIndexable had an empty resolved-module list, so the comparison found no match and reported it misfiled — reintroducing, through the misfiling axis, exactly the false staleness that the NotIndexable state exists on the anchor axis to prevent", "file": "`cmd/mxcli/brain/entry.go` (`MisfiledIn`)", "insight": "When a check has two axes, an 'unknown' outcome on one of them must not be read as a negative on the other. The fix is to make misfiling *undecidable* rather than false when nothing resolved: with no resolved anchor there is no evidence about where the entry belongs, and an anchor that truly names nothing is already a failure on its own axis. Caught in development by a table test whose control stubbed the guard to `if false` — a control that deletes the block instead fails to compile on unused variables, which is not a control", "refs": ["ako/mxcli#385", "PROPOSAL_project_brain.md A1"]} {"area": "cmd/mxcli", "date": "2026-09-03", "symptom": "`mxcli brain check` exits 1 on a requirement that is simply not built yet — the entry is correct and current, and the check reports its anchor as NOT FOUND", "cause": "Requirements were recorded as ordinary brain entries, but an entry's anchor was assumed to point BACKWARD at something that exists. A decision's unresolved anchor means the decision is stale; a requirement's unresolved anchor means the work is not done. Same syntax, opposite meaning, and the store had no way to tell them apart", "file": "`cmd/mxcli/brain/entry.go` (`Kind`), `cmd/mxcli/brain/check.go` (`checkSlice`)", "insight": "Before adding a record type to an existing store, ask what a FAILED validation means for it — not just what it looks like. Requirements and decisions share the anchor syntax exactly, which is what made them look like the same thing; they differ only in the direction the anchor points, and that difference is the whole lifecycle. Measured before designing: one unbuilt requirement filed as a decision took `brain check` to exit 1, which settled it in one command. The inversion then pays for itself — a requirement is 'built' when its anchors resolve, so `brain plan` reports progress derived from the model (measured 0/1 -> 1/0 after creating the microflow, with the plan file untouched) instead of a status column that goes stale silently", "refs": ["ako/mxcli#385"]} {"area": "cmd/mxcli", "cause": "BuildResult parsed only status/restartRequired/message and left everything else in Raw, so a failed build was reported by dumping the whole response body; nothing looked at the per-problem severity/errorCode/locations mxbuild actually returns. The consistency error that stopped the build therefore arrived unmarked among the warnings, and no test was named.", "ce": ["CE0109", "CE0117"], "date": "2026-09-03", "file": "cmd/mxcli/docker/mxserve.go (BuildProblem/BuildLocation/Errors/ErrorSummary/BuildFailedError), cmd/mxcli/testrunner/build_attribution.go (new), wired at cmd/mxcli/testrunner/runner_endpoint.go", "insight": "**The serve /build response already carries everything needed to attribute a build failure, and nothing was reading it.** Measured on 11.13: `problems` is an OBJECT whose inner `problems` list holds each consistency message with severity, errorCode and locations[] {module, document, element} \u2014 the document being `Microflow 'Test_test_3'` WITHOUT its module. So an error in a generated test microflow maps back exactly. The ratio is the point: a failing blank app returns 18 problems of which 1 is the error, and printing the body meant 11,580 bytes in which nothing marked the line that mattered; filtering severity==Error renders it as one line naming the test AND the decision. **Do not guess a response shape \u2014 POST to the serve API and look.** `mxbuild --serve --host=127.0.0.1 --port=N` plus a curl to /build is the whole harness, and no fixture in the repo had ever recorded a FAILING build.\n\n**The abandoned half is the more useful lesson.** Catching these earlier \u2014 refusing an unbound variable in an IF condition at injection time \u2014 was built, passed 465 check-mdl scripts and the whole unit suite, and was WRONG. `mxcli test` execs its microflows, and the microflow validator's scope model tracks variables where they are ASSIGNED; reusing it to check READS refuses valid work. Two independent holes, both found only by `make test-integration`: `$latestHttpResponse` is a Mendix system variable that no MDL statement declares, and a loop iterator is registered only when the list's type is known (`if listType, ok := fb.varTypes[...]`). Both refuse a microflow `mx check` accepts at 0 errors. **Generalisable: a variable model built for checking writes is not a variable model for checking reads** \u2014 the write side only has to know the names being bound, the read side has to know every name that can legally be in scope, including ones the platform supplies. Before reusing any scope model in the opposite direction, enumerate what populates it and assume the list is incomplete.\n\n**Process: `make check-mdl` is NOT the over-reach guard for an exec-path change.** It runs `mxcli check` with no project, so it exercises syntax only; a new refusal on the exec path sails through all 465 scripts. `make test-integration` (what CI runs, and runnable locally with mxbuild cached) execs every doctype script against a real project and runs `mx check` on the result \u2014 that is the guard, and skipping it cost a red CI. Also note the exec/check validator split (#833): a validator fix wired only into ValidateMicroflowBody looks correct under `check -p` and does nothing under `exec`.", "refs": ["ako/mxcli-sudoku FINDINGS #46 follow-up"], "symptom": "`mxcli test --local`: an @expect that is syntactically valid but only fails inside mxbuild takes down the ENTIRE run \u2014 `Error: local runtime: build failed: The project cannot be deployed, because it contains errors.` No test results at all, valid tests in the same file never run, and the cause arrives as ~200 lines of mxbuild JSON in which the real error sits among dozens of unrelated Atlas warnings."} +{"area": "cmd/mxcli", "date": "2026-09-11", "symptom": "`mxcli marketplace update --to -p App.mpr` exits 0, reports \"N units copied, N element identities preserved, N role grant(s) restored\", and leaves the project at **CE0066** \"Entity access is out of date. Please update security by clicking the 'Update security' button in the domain model editor\" at \"Domain model of module ''\". Its \"Next, repair what a headless update leaves behind\" block names CE0463 and CE6087 only, so the error reads as needing Studio Pro — reported as \"no headless fix available\", breaking unattended upgrade pipelines (mendixlabs/mxcli#1085, against 0.21.0 on Mendix 11.12.4, UserCommons 2.1.3 -> 2.2.0). `marketplace install` has the same gap.", "cause": "A transplant is the ONE mxcli write path that does not reconcile as it writes. `TransplantModule` copies the incoming module's units in verbatim through the raw-unit writer, and `RestoreRoleGrants` (cmd/mxcli/marketplace/grants.go) runs its statements with `ex.Execute(stmt)` in a loop rather than `ExecuteProgram`, so `finalizeProgramExecution` — the only thing that calls `ReconcileMemberAccesses` outside the entity/grant/association handlers — never runs. An access rule that does not cover every member of its entity therefore arrives exactly as the package shipped it. Fixed by `marketplace.ReconcileModuleSecurity` (new, cmd/mxcli/marketplace/security.go) called at the end of `PerformUpdate` and `PerformInstall`, non-fatally (the module is already replaced; aborting there would leave the project mid-update), with the count reported by `reportSecurityReconcile` and the failure path naming `update security `.", "file": "cmd/mxcli/marketplace/security.go (new, ReconcileModuleSecurity), cmd/mxcli/marketplace/update.go (PerformUpdate, PerformInstall, UpdateResult.RulesReconciled/SecurityErr), cmd/mxcli/cmd_marketplace_update.go (reportSecurityReconcile), cmd/mxcli/cmd_marketplace_install.go; tests cmd/mxcli/marketplace/security_test.go; example mdl-examples/bug-tests/marketplace-1085-security-after-transplant.mdl", "insight": "**The control is what localises this one, and it is cheap: transplant the module UNCHANGED.** On the vendored 11.6.6 app (Administration 4.3.2, mxbuild 11.14.0) an unchanged update checks at 0 errors, and removing ONE member entry from the incoming version's rules gives exactly the reported CE0066 — so the machinery does not create the stale rule, it transmits one, and the fix belongs at the point of arrival rather than anywhere in the copy. Three things worth carrying. (1) The reporter's premise was half wrong in a way that matters more than the bug: a headless repair already existed (`UPDATE SECURITY`, #1047), but it appears in NO user-facing surface — not docs-site, not MDL_QUICK_REFERENCE.md, not `mxcli syntax`, not the update's own output — only in a skill and the CHANGELOG. An undiscoverable capability and a missing one are the same bug report. (2) Their binary could not have used it anyway: v0.21.0 (2026-09-06) has the System-skip fix but NOT the inherited-attribute reconcile that landed 2026-09-07 (`collectAttrs` is absent), so where the missing member is inherited from a same-module generalization it would have printed \"All entity access rules are up to date\" over a broken model. Check the reported version against the fix DATE, not against the fix existing. (3) Reconciling unconditionally would be wrong, and the second test is the one that says so: a module whose rules are already complete must come through untouched, because rewriting a marketplace module's domain model makes `marketplace diff` read it as a local edit — and a local edit is what makes the NEXT update refuse. The repair cannot be demonstrated by an .mdl script either: every MDL write path reconciles as it writes, so the stale state has to be built by editing stored BSON in the test.", "ce": ["CE0066"], "refs": ["mendixlabs/mxcli#1085", "mendixlabs/mxcli#1047"]} +{"area": "cmd/mxcli", "date": "2026-09-10", "symptom": "`mxcli run --local --watch` cannot start a Mendix 11.14 app: `starting web client bundler: no rollup.config.mjs in \u2026/deployment/web (run a serve Deploy build first)` \u2014 while plain `run --local` on the SAME project works. A serve Deploy build has just run", "cause": "The identical defect as ako/mxcli-ledger #146, in the SECOND copy of the gate. 11.14 bundles the browser client itself and emits no rollup config; #146 moved the gate onto the gap the step closes, but only in `BuildWebClient`. `StartWebClientWatch` carries its own independent `os.Stat(rollup.config.mjs)` check that the fix never touched \u2014 so the non-watch path started working and `--watch` kept dying on the same absent file, one call earlier. #146's own commit message claimed \"both call sites are fatal\"; the two it meant were both BuildWebClient's", "file": "`cmd/mxcli/docker/webclient_watch.go` (`StartWebClientWatch`), called fatally at `runlocal.go:659`", "insight": "**When a gate is duplicated, `grep` for the predicate before declaring the fix done** \u2014 the sibling here was one `os.Stat` of the same filename in the same package, and the commit that fixed one asserted it had fixed both. Fix shape: return a **nil watcher**, not an error, when `WebClientBundled(deployDir)` \u2014 there is no bundler to keep hot because mxbuild's serve build writes web/dist. That is only half the fix: the watch loop calls `Generation`, `WaitForRebuild` and `Log`/`Stop` with no nil branch, so the methods must be nil-receiver-safe or the startup refusal is merely traded for a SIGSEGV on the first file change (the control test panics against the unfixed build, which is what proves it). Keep a control asserting the 11.13 path still LAUNCHES the bundler, or the fix degrades to \"never watch anything\"", "refs": ["#146"]} +{"area": "cmd/mxcli", "date": "2026-09-10", "symptom": "`mxcli run --local --watch` on a Mendix 11.14 app: the FIRST build succeeds and the app boots, then every rebuild fails inside mxbuild \u2014 `Could not find a part of the path '\u2026/deployment/web/pages/.js'` when the change touched a page, or `Compilation of the app bundle failed` / `ERR_MODULE_NOT_FOUND \u2026 web/rollup.config.mjs` when it did not. Reads as a corrupt `deployment/`", "cause": "**mxbuild 11.14's own defect, not mxcli's.** The first build in an `mxbuild --serve` process does not leave the deployment in a state its own incremental build can continue from: neither the bundler config (`web/rollup.config.mjs`) nor the per-document client dirs (`web/pages/`, `web/layouts/`) survive it, and the next build needs both. Which one it dies on is just how far it gets \u2014 model changed \u2192 page export, model untouched \u2192 the bundler. Note `--serve`, `--host` and `--port` are in `mxbuild --help` but NOT on docs.mendix.com/refguide/mxbuild/, so this is an undocumented interface with no compatibility promise", "file": "`cmd/mxcli/docker/webclient_legacy_paths.go` (`legacyClientBuildHint`), called from the build-failure branch of `watchAndApply` in `runlocal.go`", "insight": "**Drive the vendor's API directly before blaming or absolving your own wrapper.** Three controls placed this in mxbuild in about ten minutes: (1) POST the SAME `/build` request twice over curl with the model untouched \u2014 Success then Failure, no mxcli in the picture; (2) run one-shot `mxbuild --target=deploy` TWICE into the same deployment dir \u2014 BUILD SUCCEEDED both times, so the 11.14 deployment shape is not the trigger, the serve process's own state is; (3) flip `EnableRspackBundler` in the model and repeat \u2014 identical failure naming `rspack.config.mjs`, which kills 'switch bundlers' as a workaround AND disproves the tempting 'it falls back to the legacy Rollup pipeline' story. Without (3) the diagnosis was plausible, quotable and wrong. mxcli cannot fix it, so the deliverable is a hint that fires on the failure's own SHAPE (message names the artifact AND the artifact is genuinely absent) rather than on a version number \u2014 so a fixed mxbuild goes quiet by itself, and it says what does NOT work (`rm -rf deployment/`, switching bundlers), because both are what a user tries first || FOLLOW-UP: the tempting workaround does NOT work, and it is worth knowing why before anyone tries again. The deleted config is recoverable \u2014 it sits on disk ~1.5s mid-build and carries nothing model-specific (input is just index.js; page discovery is delegated to a rollup plugin at build time), so it can be harvested the way `mxcli fix widgets` harvests mxbuild's output. Restoring it makes builds 2 and 3 succeed **with the model unchanged**, and fail the moment a page actually changes (missing web/pages/*.js); adding the missing dirs too moves it to a pluggable-widget export failure on two widgets that ship with a blank app, with web/dist/index.js never moving off its cold-build timestamp. So there are TWO regressions and only the visible one is recoverable. **Test a workaround against the case the feature exists for, not the case that is easy to set up** \u2014 an unchanged-model rebuild is exactly what a warm loop never does, and a green result there would have shipped a fix that helps nobody", "refs": ["#146"]} +{"area": "cmd/mxcli", "date": "2026-09-11", "symptom": "The record-narrated-demo skill MANDATES a mobile pass ('Recorded at both a desktop viewport and a real mobile device profile') and its own `take.js` cannot perform one: the 'mobile' take films the DESKTOP app in a narrow window. Separately, the caption plate made any control it covered unclickable \u2014 Playwright retried 30s against 'div.text from div#demo-narration subtree intercepts pointer events' and failed the take", "cause": "`openTake` called `browser.newContext({viewport, recordVideo})` and nothing else, but `userAgent`/`isMobile`/`hasTouch`/`deviceScaleFactor` are CONTEXT options \u2014 and **Mendix picks its navigation profile from the user agent, not the viewport**, so shrinking the viewport never routes to the phone profile. The plate had no `pointer-events: none` (only `#demo-spot` did) while being a full-width bar pinned to the bottom \u2014 exactly where Mendix puts a page footer's buttons", "file": "`.claude/skills/mendix/record-narrated-demo/take.js` (`openTake`), `narrate.js` (`OVERLAY_CSS`); embed mirror under `cmd/mxcli/skills/` is regenerated by `make sync-skills`", "insight": "**When a skill mandates a verification pass, check that the tool it ships can actually perform it** \u2014 this one could not, and nothing said so: the take succeeded, produced a plausible film, and simply filmed the wrong thing. A verification step that cannot fail is worse than no step, because it is counted as done. The fix is a `contextOptions` passthrough spread FIRST so `viewport`/`recordVideo` still win (a device preset carrying its own viewport would letterbox every take). For the plate, `pointer-events: none` makes a covered control CLICKABLE and `body { padding-bottom }` makes it VISIBLE \u2014 the film needs both, since a click landing under an opaque caption is the same dead beat by another route. Reported by ako/ChipCoV1, which shipped an app whose phone walk was impossible while seven browser tests stayed green", "refs": ["ako/ChipCoV1 FINDINGS.md"]} +{"area": "cmd/mxcli", "date": "2026-09-11", "symptom": "Two opposite `brain plan` failures with one root cause. A requirement anchored at a bare MODULE (`@Maintenance`) reports BUILT the moment the module exists, with none of its work done. A requirement anchored at a MODULE ROLE (`@Maintenance.Coordinator`) reports PLANNED forever, even once the roles exist \u2014 `describe` refuses it ('no describable document named \u2026') and `mxcli refs` finds nothing", "cause": "`catalogResolver.Resolve` in `cmd/mxcli/cmd_brain.go`. A module has a row in the catalog's `objects` view, so it resolves immediately \u2014 fine for a decision (anchors point backward) and useless for a requirement (anchors point forward, so resolution IS the progress signal). A module role is in NEITHER the objects view NOR `FindDocumentUnit`, because it is not a document \u2014 so both lookups miss and it falls through to NotFound, which for a requirement means 'not built yet' permanently", "file": "`cmd/mxcli/cmd_brain.go` (`catalogResolver.Resolve`, `moduleRoleExists` via `GetModuleSecurity`), `cmd/mxcli/brain/entry.go` (`requirementAnchorsArePlannable`)", "insight": "**A resolver's vocabulary has to cover what people actually anchor at, and the two failure directions need opposite fixes.** The module-role gap is a LOOKUP gap \u2014 fixed by consulting `GetModuleSecurity`, case-insensitively because Mendix treats role names that way and an anchor is hand-written. The module-anchor gap is SEMANTIC and cannot be fixed by lookup: the anchor resolves correctly and is still useless, so it is refused at capture time with the alternative named (an author told only 'no' deletes the anchor, which loses the signal entirely rather than fixing it). Keep the control in BOTH directions: a module anchor stays legal on a decision and on a question, or the fix degrades to 'refuse every module anchor' and the negative test still passes. Verified end to end: `@MyFirstModule.User` \u2192 1 anchor 1 resolved; `@MyFirstModule.NoSuchRole` \u2192 NOT FOUND, exit 1, so the resolver did not simply become permissive. Reported by ako/ChipCoV1", "refs": ["ako/ChipCoV1 FINDINGS.md"]} +{"area": "cmd/mxcli", "date": "2026-09-11", "symptom": "`.claude/bootstrap-mxcli.sh` (the SessionStart hook) re-downloads ~85 MB of mxcli on EVERY fresh session in an environment where mxcli is already installed on PATH", "cause": "The script gated only on `[ ! -x ./mxcli ]` and never consulted PATH, while the bootstrap skill tells you to `rm -f /mxcli` after moving the project to the repo root \u2014 so in a session image with mxcli pre-installed the guard could never be satisfied by the binary already present, forever", "file": "`cmd/mxcli/init_hook.go` (`bootstrapScriptTemplate`)", "insight": "**A guard that asks 'is the artifact HERE' rather than 'is it AVAILABLE' pays full cost for something already on the machine.** Hardlink a PATH copy in first (what `mxcli new` itself does), falling back to symlink across filesystems then to a copy, and only download when none of those work \u2014 every path leaves ./mxcli working, which the rest of the script and the generated project CLAUDE.md both assume. Keep BOTH controls when testing: nothing on PATH must still download (not silently no-op), and an existing ./mxcli must be left untouched. Reported by ako/ChipCoV1", "refs": ["ako/ChipCoV1 FINDINGS.md"]} +{"area": "cmd/mxcli", "date": "2026-09-11", "symptom": "`mxcli theme create --from ` seeds the palette and nothing else: the scaffolded brand theme still described itself in `theme list` as 'Cool slate, one teal signal colour' with Signal's six swatches, and vendored ~500 KB of IBM Plex woff2 the seeded --mxt-font never names, plus a SIL OFL licence for fonts it does not use. Separately, the primary button was never the brand colour: Atlas derives --btn-primary-bg from --brand-primary-600 = color-mix(brand, contrast 20%), so a brand blue #10069F rendered rgb(21,13,140)", "cause": "`manifest()` copied the base theme's Summary and Colorway verbatim and the walk copied every file unconditionally. The Atlas map pinned `--btn-primary-color` to `--mxt-brand-ink` \u2014 an ink each theme picks to sit on `--mxt-brand` (console pairs near-black #04211d with bright teal #2dd4bf) \u2014 while leaving the background to Atlas's derivative, so the pairing the theme designed for was never the pairing that rendered. The map's own comment already called it 'a brand-filled button'", "file": "`cmd/mxcli/theme/create_seeded.go`, `create.go` (`manifest`, the scaffold walk), `assets/*/files/theme/web/_mxcli-atlas-map.scss`", "insight": "**Inheriting a statement ABOUT the base theme into a theme whose palette is no longer the base's is a confident lie; derive it or drop it.** Two traps in the font half. (1) The decision must be made BEFORE the walk: it is taken by reading the partial and applied to files elsewhere in the tree, so doing it inline depended on WalkDir's lexical order putting `_mxcli-.scss` before `mxcli-fonts/` \u2014 true only because of the leading underscore. (2) It touches two halves \u2014 the @font-face rules and the woff2 files \u2014 and getting either alone wrong is silent: a surviving rule for a deleted file 404s, a surviving file nothing loads is the dead weight being removed. Unit tests on each half cannot catch a mismatch; the guard is an integration assertion that a scaffolded theme ships exactly the fonts it loads (control: stubbing the file half fails it with 'X is shipped but no @font-face loads it'). Reported by ako/ChipCoV1", "refs": ["ako/ChipCoV1 FINDINGS.md"]} diff --git a/.claude/skills/fix-issue/findings/mdl-backend.jsonl b/.claude/skills/fix-issue/findings/mdl-backend.jsonl index dec00b9d1e..fee1d01825 100644 --- a/.claude/skills/fix-issue/findings/mdl-backend.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-backend.jsonl @@ -87,3 +87,5 @@ {"area": "mdl/backend", "date": "2026-09-08", "symptom": "`alter page P { set PageSize = 10 on }` errors `pluggable property \"PageSize\" not found` on a grid that `create page \u2026 (PageSize: 20)` had just written and the app really pages at. `mxcli check --references` passes the script, so it fails only at exec, after earlier statements have landed; DESCRIBE PAGE prints the same capitalised `PageSize:`, so describe \u2192 edit \u2192 exec produced a script mxcli refused to run", "cause": "A pluggable property key is lowerCamel in the widget template (`pageSize`). CREATE resolves the author's spelling case-INsensitively (widget engine `lookupProperty`, and `WidgetV3.GetStringProp` before it); ALTER went through `setPluggableWidgetPropertyMut`, which compared the template key byte-for-byte, so only the exact `pageSize` worked. Direct sequel to Findings #1 (2026-07-27), which fixed the same class for FIRST-CLASS props and deliberately left the pluggable fallback case-sensitive with the comment 'template keys must match the template exactly'", "file": "`mdl/backend/pagemutator/mutator.go` (`setPluggableWidgetPropertyMut`)", "insight": "`strings.EqualFold` against the widget's own PropertyTypes. **The disproven belief is the reusable part**: keys are STORED case-sensitively, which is not a reason to MATCH them that way \u2014 the resolver searches one object type's PropertyTypes, and across every shipped template and definition that scope holds no two keys differing only in case (96 scopes, 1208 keys, 0 collisions). Measure the ambiguity before assuming it; here there was none, and the assumption cost a whole verb. **Cheapest localiser**: run the failing statement with the template's own casing \u2014 `set pageSize` succeeded where `set PageSize` failed, in ONE measurement, on the same widget in the same project. Both engines share `pagemutator`, so an engine split says nothing here (verified: modelsdk and legacy both fixed by the one change). Tests `TestSetPluggableProperty_MatchesTemplateKeyRegardlessOfCase` (+ typo-still-errors control, + `TestPluggablePropertyKeysAreUniqueIgnoringCase` pinning the no-collision argument); repro `mdl-examples/bug-tests/alter-page-pluggable-property-casing.mdl`; verified 0 errors on `mx check` 11.13.0. Two reporter claims did NOT hold: `describe page` DOES emit PageSize, but only when it differs from the widget default 20 (deliberate, so describe round-trips) \u2014 at the default it is omitted, which reads as 'describe cannot show it'. Still open and separate: `check --references` does not resolve pluggable property names at all, so a genuine typo (`PagSize`) still checks clean and fails at exec", "refs": ["mendixlabs/mxcli#1069"]} {"area": "mdl/backend", "date": "2026-09-09", "symptom": "A control that should have proven a guard was load-bearing PASSED with the guard removed — the test could not distinguish a correct writer from one that reset the property on every write", "cause": "The fixture stored `false` for a boolean property, and false is also the zero value. A writer that ignored the spec entirely wrote false; the correct writer preserved false. Identical output, so the assertion held either way", "file": "`mdl/backend/modelsdk/navigation_throw_sync_test.go`", "insight": "For a BOOLEAN property, a preservation test must exercise BOTH stored values — the non-zero one is the only case that can fail. This is the second time in one feature: CompatibilityMode needed a synthetic `true` because all seven reference configs carry false. The generalisation: when every real document agrees on a value, the fixture drawn from real documents cannot test preservation, and a synthetic counter-case is not optional. The tell is a control that fails to fail — if stubbing the guard leaves the suite green, the test is measuring nothing, and that is worse than no test because it reads as coverage", "refs": ["ako/mxcli#413", "ThrowPartialSyncError"]} {"area": "mdl/backend", "date": "2026-09-09", "symptom": "`call web service` (legacy SOAP) on the DEFAULT engine: `mxcli exec 06b-soap-examples.mdl` reported success on all three microflows and `mx check` (11.13.0) then failed the project with `[CE0008] \"No action defined.\" at Action activity 'Activity'` plus two `[CE0109] \"Undefined variable 'Root'.\"`. Only reachable by rerunning with MXCLI_ENGINE=legacy, which is why legacy was still the documented fallback", "cause": "The codec engine READ the action (`actionFromGen` \u2192 `*microflows.WebServiceCallAction`, with a raw fallback) but `microflowActionToGen` had no case for it, so it hit `default: return nil` and the enclosing ActionActivity serialized with no action at all. The #850 shape: a missing WRITE case is not an unsupported feature, it is a silent drop that exec reports as success. The CE0109s are knock-on \u2014 the dropped action never bound $Root", "file": "`mdl/backend/modelsdk/microflow_webservice_write.go` (new), `microflow_write.go` (switch case)", "insight": "**Mirror the legacy serializer, and prove it by diffing the two engines' documents \u2014 do not re-derive the shape from the metamodel.** There is no Studio Pro-authored SOAP document in this repo, so legacy's output is the only reference that exists and is also what users' projects already contain. Method: exec the same script on each engine, `mxcli bson dump` both, normalise the random $IDs, diff. Three discrepancies fell out that no amount of reading would have shown, all in how the codec emits NULLS and MARKERS: (1) a Part property with no child encodes to nil and the encoder then SKIPS the key (`if val != nil`), so an unset part is an ABSENT key, never a null one \u2014 carry the null as a primitive `bson.Null{}` value instead, which marshals in place; (2) `codec.TypeDefaults.NullFields` does emit the key but APPENDS it after every property, so it cannot reproduce alphabetical key order; (3) both NullFields and list markers are registered per `$Type` and several types are SHARED between writers \u2014 `Microflows$HttpConfiguration` needs HttpHeaderEntries marker 3 for SOAP and 2 for REST, and legacy writes CustomLocationTemplate as null for SOAP but omits it for REST, so a global registration would have silently changed the REST path. Write those explicitly per call site. (The package already carries one such collision: `Microflows$HttpHeaderEntry` is registered 2 in microflow_write.go and 3 in odata_write.go, decided by file order.) Verified: the two engines' CallWebServiceAction documents are now identical key-for-key and value-for-value; `mx check` goes 3 structural errors \u2192 0, leaving only the script's 4 deliberate CE1613 dangling refs, which is exactly legacy's output. Revert control: removing the switch case fails all six unit tests with the CE0008 message. The doctype engineScriptSkip for modelsdk/06b was removed and the script now runs on BOTH engines", "refs": []} +{"area": "mdl-backend", "date": "2026-09-10", "symptom": "`mxcli diff-local` on an MPR v2 project fails with `Error: mprcontents directory not found` while mprcontents/ exists and is populated; `MXCLI_ENGINE=legacy` works", "cause": "The modelsdk engine (the default) never overrode `Backend.ContentsDir()`, so it fell through to the generated `unimplemented` stub and returned \"\". diff-local reads \"\" as 'not a v2 project'.", "file": "mdl/backend/modelsdk/backend.go", "insight": "gen_unimplemented.go's promise that an unoverridden method 'fails loudly rather than silently dropping data' is CONDITIONAL on the method having an error to fail through: the generated body is `errUnimplemented` only when a result is `error`, a panic when there are no results at all, and a silent `var r0 T; return r0` otherwise. ContentsDir is in the third bucket and its zero value is a MEANINGFUL in-band answer (\"\" == MPR v1), so the missing implementation was indistinguishable from a v1 project rather than looking like a bug. The detectable signature was the contradiction between two questions the same command asks: Version() (implemented) says 2, ContentsDir() says v1. Guard added in mdl/backend/modelsdk/unimplemented_silent_test.go \u2014 reflect over FullBackend for error-less methods, go/parser the package for methods actually declared on *Backend, since reflection cannot tell a promoted method from an override (Go synthesises a wrapper named (*Backend).X for both). It immediately found a second one, InvalidateCache (a latent panic, no caller today).", "refs": ["mendixlabs/mxcli#1080"]} +{"area": "mdl/backend", "date": "2026-09-10", "symptom": "SOAP `call web service` writes a document mxbuild accepts and Studio Pro would not have written. A SEND MAPPING is silently DROPPED by both engines \u2014 `send mapping Mod.Export` parses, `mxcli check` passes, `exec` reports success, and nothing in the stored action references the mapping. Operation ARGUMENTS are dropped the same way", "cause": "sdk/mpr.serializeWebServiceCallAction was written without a Studio Pro reference and hardcodes five things it cannot know, and the codec engine's new writer reproduced it deliberately for parity. Measured against three Studio Pro-authored calls in ako/TestApp (Mendix 11.14.0, Clients.GetOrders / GetCustomerOrders / SaveOrder): ServiceName is the WSDL SERVICE name (\"OrdersWS\") not the local part of the imported service's qualified name (\"OrderSoapClient\"); ImportMappingCall.ContentType is \"Xml\" for a SOAP import mapping, not \"Json\"; Range.SingleObject follows cardinality (false for a list) rather than being always true; VariableType is the real result type (DataTypes$ObjectType with an Entity, DataTypes$BooleanType) rather than always DataTypes$VoidType; and a send mapping is Microflows$MappingRequestHandling {ContentType, MappingId, MappingVariableName}. Arguments live in RequestBodyHandling.ParameterMappings as Microflows$WebServiceOperationSimpleParameterMapping entries keyed by an escaped ParameterPath (\"http%3A//www.example.com/:GetOrder|OrderId\")", "file": "`sdk/mpr/writer_microflow_actions.go` (serializeWebServiceCallAction), `mdl/backend/modelsdk/microflow_webservice_write.go`", "insight": "**A guessed type name in a comment becomes a permanent refusal.** Legacy refused send mappings citing `Mendix$AdvancedRequestHandling`, said it 'requires a Studio Pro-generated example to determine the correct type storage name', and that refusal then shipped for as long as nobody went looking. The real type is `Microflows$MappingRequestHandling` \u2014 which THIS CODEBASE ALREADY WRITES for REST result/request handling \u2014 and the guessed name occurs in none of the three reference documents. The lesson is not about SOAP: when a writer refuses because a storage name is unknown, check whether a sibling feature already writes it before treating the refusal as a standing constraint. **Second, and the reason this was found at all: 'no reference exists' is a claim about where you looked.** The parity work asserted that no Studio Pro-authored SOAP document existed to pin against and used that to justify mirroring legacy; one existed in a separate repo the whole time (ako/TestApp, which carries both a consumed client and a published service). Byte-parity with what ships is a legitimate goal for a change scoped to stopping a silent drop \u2014 it is NOT evidence the shape is right, and conflating the two is how six defects got a passing test. Where a reference project exists, name it in the code so the next reader does not repeat the search", "refs": []} diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index c3caf4b7b3..09ee6bb30e 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -578,6 +578,13 @@ {"area": "mdl/executor", "date": "2026-09-09", "symptom": "A microflow whose activity has a CUSTOM error handler comes back from `describe microflow` with the handler AND every activity in its branch missing. Output stays valid MDL, `mxcli check` passes, nothing on stderr \u2014 so describe\u2192edit\u2192exec deletes the handler from the model. Reported on a create-variable activity with \"custom with rollback\" (v0.21, Mx 11.12.3); reproduced on HEAD/11.14.0 with a pure mxcli round trip", "cause": "TWO STACKED DEFECTS. (1) `getActionErrorHandlingType` was a hand-maintained switch covering 17 of the 38 action types that store ErrorHandlingType. `emitActivityStatement` walks the error branch only when `hasCustomErrorHandler(errType)` agrees, so each of the 21 missing types lost the ENTIRE `on error { \u2026 }` block, not just the suffix \u2014 CreateObject and ChangeObject among them. (2) legacy only: 9 parse functions never read ErrorHandlingType off the BSON, so the value was gone before the describer was asked. Fixing (1) alone left legacy still broken, which is how they hid each other", "file": "`mdl/executor/cmd_microflows_show_helpers.go` (`getActionErrorHandlingType` \u2192 new `actionErrorHandlingField`); `sdk/mpr/parser_microflow.go`, `sdk/mpr/parser_microflow_actions.go` (9 parsers); grammar+visitor+builder for 8 statements; `mdl/executor/validate_microflow_error_handling.go` (MDL076 table, new MDL077)", "insight": "**Count the gap before fixing the instance** \u2014 the reported activity was 1 of 21, measured by diffing the switch's cases against the action types declaring the field. **Replace the enumeration, do not extend it**: the list had already been patched per-instance (#863) and silently regrew, so it is now a reflection lookup on the `ErrorHandlingType` field, with `RestOperationCallAction` the ONE deliberate exclusion (Mendix rejects a custom handler there, CE6035). Actions embed model.BaseElement, which has no such field, so no promoted field is picked up by accident. **The trap: fixing the describer alone makes things WORSE.** 8 statement forms had no `onErrorClause` in the grammar \u2014 `declare` (the reporter's own), `set`, `change`, `log`, `show page`, `close page`, `show message`, `validation feedback` \u2014 so DESCRIBE began emitting `declare $name String = 'v' on error { \u2026 };`, which fails to parse (`mismatched input 'on' expecting ';'`). Trading a silent drop for a broken script is not a fix; the grammar was extended instead. **Measured on 11.14.0, and the result is not guessable**: all 8 accept a custom handler (0 errors), but for `on error continue` create-VARIABLE and change-VARIABLE are fine while change-OBJECT, log, show page, close page, show message and validation feedback are CE6035 \u2014 now in MDL076's deny-list, whose comment claiming Log/Change were \"unreachable from a script\" this change invalidated. New MDL077 refuses `on error` on the list-operation/aggregate forms of `set`, which genuinely have no ErrorHandlingType in the metamodel \u2014 one MDL keyword spanning activities that can and cannot hold the clause. **A non-terminating handler causing CE0108 is NOT a bug**: the branch merges back and a later variable is out of scope on the error path; Studio Pro reports the same. Controls: revert the lookup \u2192 all 7 describer cases fail naming the dropped branch; revert one parser \u2192 the legacy case fails; a no-clause microflow must still render NO suffix (#840 in reverse). Repro `mdl-examples/bug-tests/microflow-1078-error-handler-roundtrip.mdl`; describe\u2192exec\u2192describe byte-identical and reported \"Unchanged microflow\"", "refs": ["#1078", "#863", "#840"], "ce": ["CE6035", "CE0108"]} {"area": "mdl/executor", "date": "2026-09-10", "symptom": "CI `make test-integration` fails where `go test ./...` is green: `TestMxCheck_DoctypeScripts/02b-nanoflow-examples.mdl` (both engines) and `/03-page-examples.mdl` report **CE6035 \"Error handling type is not supported\"** on 11 activities \u2014 every un-annotated Change object / Log message / Validation feedback / Close page in a NANOFLOW. Self-inflicted while fixing #1078", "cause": "Eight builders were switched from `fb.ehType(nil)` to `explicitErrorHandling(fb, s.ErrorHandling)` on the action. `explicitErrorHandling` returns EMPTY for \"no clause\", and the writers turn empty into a literal `\"Rollback\"` via orDefault \u2014 but `fb.ehType(nil)` is CONTEXT-DEPENDENT and returns **Abort** in a nanoflow. So every un-annotated nanoflow activity silently changed from Abort to Rollback, which mxbuild rejects", "file": "`mdl/executor/cmd_microflows_builder_actions.go` + `cmd_microflows_builder_calls.go` (8 sites reverted to `fb.ehType(s.ErrorHandling)`); `mdl/executor/nanoflow_validation.go` (`getErrorHandling`)", "insight": "**The same helper is correct for one action and wrong for its neighbour, and the difference is what the code did BEFORE.** `explicitErrorHandling` is right for Retrieve/Delete (#1020-era): their writers emitted a hardcoded `\"Rollback\"` that those two actions accept in every flow flavour, so empty\u2192Rollback is a no-op. It is wrong wherever the builder already supplied a *context-dependent* default \u2014 empty discards the flow flavour. The rule: before replacing a default-supplying expression, ask what the OLD expression returned in every context, not just the one you are testing. Its own doc comment names Abort/nanoflow and I still missed it, because I was reading it as \"the safe choice\" rather than \"the choice that preserves THIS call site's prior value\". **`go test ./...` does not run this suite** \u2014 the mx-check round trips are behind `-tags integration` (`make test-integration`, ~30 min), so a green unit suite says nothing about whether mxbuild still accepts what mxcli writes; run it before pushing anything that touches a serialized default. Regression test `TestAuthorOnError_NanoflowKeepsAbortWithoutAClause` (control: the same statements in a microflow must NOT become Abort) \u2014 a unit test, so it catches this in seconds instead of 30 minutes. A second, quieter gap in the same change: `getErrorHandling` in nanoflow_validation.go gates the walk that looks for disallowed actions INSIDE a handler body, so the eight new statements had to be added there too or a Java action nested in `declare \u2026 on error { \u2026 }` goes unreported (test carries a `commit` control, since that entry point is `validateNanoflowBody`, NOT the exported `ValidateNanoflowBody`, which is a different check)", "refs": ["#1078"], "ce": ["CE6035"]} {"area": "mdl/executor", "date": "2026-09-10", "symptom": "In a NANOFLOW, `log ... on error continue|rollback|{ \u2026 }`, and a custom handler on change/show page/close page/show message/validation feedback, are written by mxcli and rejected by mxbuild as **CE6035 \"Error handling type is not supported\"**. `mxcli check` passed. Reachable for the first time via mendixlabs/mxcli#1078, which gave those statements an onErrorClause", "cause": "No rule covered it. MDL076 runs on the microflow validator, which has no flow flavour, so it cannot express \"CE6035 in a nanoflow but fine in a microflow\" \u2014 and every one of these six IS fine in a microflow with a custom handler", "file": "`mdl/executor/nanoflow_validation.go` (`checkNanoflowErrorHandling`, `nanoflowErrorHandlingUnsupported`, wired into `validateNanoflowStatements`)", "insight": "**A nanoflow accepts error handling on almost nothing**: measured on 11.14.0, only the two VARIABLE activities (create-variable, change-variable) take a clause; the other six are CE6035 whichever form is written, because a nanoflow activity's only accepted value is Abort \u2014 the no-clause default, which no MDL syntax writes. So the rule refuses the CLAUSE, not one spelling. The split is by ACTIVITY, not by client-side/server-side: `show message` is as client-side as it gets and still refuses one, while `declare` accepts it. Same permissive pair as `continue` in a microflow, which is the only pattern visible across both tables. **Two validators, two entry points, easy to wire to the wrong one**: `validateNanoflowBody` (unexported) is the disallowed-action walk that exec runs; `ValidateNanoflowBody` (exported) is the variable/semantic check that `check` runs. A test targeting the exported one passes vacuously \u2014 caught only because the test carried a `commit` control that also failed. Note the pre-existing consequence: every nanoflow restriction, including the 22 disallowed action types, is enforced at EXEC and not by `mxcli check` (even with `--references`) \u2014 a check/exec divergence worth closing on its own, but not in a bug fix, since it would newly reject scripts that pass check today", "refs": ["#1078"], "ce": ["CE6035"]} +{"area": "mdl-executor", "date": "2026-09-10", "symptom": "`mxcli diff-local` prints `Summary: 0 new, 1 modified, 0 deleted` and not a single line of diff \u2014 in all three formats \u2014 for an edited microflow on the default engine", "cause": "`Backend.ParseMicroflowFromRaw` was also unimplemented on the modelsdk engine and returned nil. `microflowBsonToMDL` guards nil by substituting a `-- parse failed --` stub, but the stub text is a CONSTANT, so both sides of the diff rendered the same string and the differ found zero hunks.", "file": "mdl/executor/cmd_diff_local.go", "insight": "A nil-guard written to degrade visibly degrades INVISIBLY when both sides of a comparison degrade identically \u2014 the fallback has to vary with its input, or it erases the difference it was meant to surface. The same shape bit the attribute renderer in the same command for an unrelated reason: attributeBsonToMDL read the type object from raw[\"Type\"], but Mendix stores it under the STORAGE name \"NewType\" (the per-property half of the storage-name split in CLAUDE.md), so every attribute rendered \"Unknown\" on BOTH engines and narrowing String(200) to String(50) diffed to nothing. Measurement that settled it: dump the unit's decoded top-level keys instead of trusting the field name \u2014 `Attributes` is a primitive.A whose FIRST element is an int32 typed-array marker, and each attribute map has NewType, no Type at all.", "refs": ["mendixlabs/mxcli#1080"]} +{"area": "mdl/executor", "date": "2026-09-10", "symptom": "A CONTAINER inside a DataGrid 2 control bar with `Action: nanoflow M.NF`, where the nanoflow declares a required entity parameter, built to **CE1571** \"No argument has been selected for parameter 'X'\" at Container 'containerUnlinkMat' — while `mxcli check --references` printed \"Check passed!\". Reported as mendixlabs/mxcli#1082, whose stated root cause was that MDL had no syntax for passing an argument through a container action", "cause": "The CE1571 argument check (`validateDataSourceArguments`, now `validateFlowArguments`) walked `w.GetDataSource()` only, so the identical missing-argument fault on an ACTION slot was silent. Separately the context walk let a control bar inherit its data widget's row context, so even once actions were walked the reported shape would still have passed", "file": "`mdl/executor/validate_datasource_args.go` (`argDiff`, `widgetActions`, `actionArgErrors`, `isControlBar` + the per-child walk); `mdl/executor/validate_page_button_context.go` (MDL-BUTTON01 suggestion); `cmd/mxcli/syntax/features_page.go` (page.action topic)", "insight": "**The report's root cause was wrong and the real defect was one layer up — check the premise by executing it, not by reading it.** `actionExprV3` has carried `NANOFLOW qualifiedName microflowArgsV3?` all along and `buildContainerV3` hands the action to the same `buildClientActionV3` a button uses; three pages in one script settled it (bare action → CE1571; `($P = $dgMaterials)` on the container → 0 errors; same on an actionbutton → 0 errors), and dropping the bare page took the project 1 → 0 errors as the control. What was broken is that nothing SAID so, and the author concluded from `mxcli syntax page.action` — which listed `Action: NANOFLOW Module.NF` with no argument variant two lines under `Action: MICROFLOW Module.MF(Param: $val)` — that the syntax did not exist. A silent check plus an incomplete syntax topic reads as a missing feature. **A control bar is not row-scoped, and that belongs in the WALK, not in either rule**: six containers carrying one identical fault, one `mx check` on 11.12.0, gave column → no error / control bar → CE1571, and the same pair holds for a DATA SOURCE (dataview in the control bar → CE1571, in a column → clean), so fixing it in the shared walk closed a latent false negative on the older half of the rule too. It drops only the data widget's OWN object — measured, an outer dataview's context still reaches the control bar — so the control bar is walked with its parent's incoming context, not an empty one. **Enumerate action slots by sweeping Properties for `*ast.ActionV3`, never by a key list**: `Action`/`OnClick` share one key, `OnChange` has its own, and a pluggable widget's named slot uses its own name; a key list would have fixed the reported slot and reproduced the bug on the rest. **The repro cannot be a .fail.mdl**: the rule needs a project to read the flow's signature, and `make check-mdl` runs check with no project, so a negative fixture would report a working rule as regressed (the #891/#892 trap)", "issue": "mendixlabs/mxcli#1082"} {"area": "mdl/executor", "date": "2026-09-10", "symptom": "A microflow note (Microflows$Annotation) connected to several activities came back DUPLICATED after describe -> exec: one note with N AnnotationFlows became N notes with one flow each. Measured on Mendix 11.13.0: 1 note + 2 flows -> 2 notes. mx check: 0 errors on both sides, so nothing warned.", "cause": "buildAnnotationsByTarget joined AnnotationFlows to Annotation objects and kept only the CAPTION, filing it under each flow's destination. The note's identity was discarded at that join, so the describer had no way to know two lines were the same note, and MDL had no way to say so either: ActivityAnnotations carried AnnotationText as a single STRING per activity. attachAnnotation then minted a fresh Annotation + flow per line.", "file": "mdl/executor/cmd_microflows_show_helpers.go (buildAnnotationsByTarget, annotationEmitter), mdl/executor/cmd_microflows_builder_annotations.go (attachAnnotation, defaultAnnotationGeometry), mdl/ast/ast_microflow.go (MicroflowAnnotation), mdl/visitor/visitor_microflow_statements.go (parseNoteAnnotation), mdl/grammar/domains/MDLSettings.g4 (annotationParamName)", "insight": "The relation is MANY-TO-MANY and a per-activity string lost BOTH directions — which is how a duplication report turned out to be hiding a deletion. One note to N activities duplicated (the report); N notes to one activity kept only the LAST, because the visitor assigned into the single slot. The second is strictly worse and nobody had noticed it. When a report says a round trip 'duplicates' something, check the transpose before fixing: the same missing cardinality usually destroys in the other direction. The fix that holds is giving the construct IDENTITY in the language (`id:`) rather than deduplicating on content — merging two notes because their text matches would be a second silent rewrite, so the control test is two identical-text notes that must STAY two.", "refs": ["mendixlabs/mxcli#1077"], "ce": []} {"area": "mdl/executor", "date": "2026-09-10", "symptom": "A parameterised annotation whose key is a lexer keyword parses and is SILENTLY IGNORED. `@annotation(text: 'x')` and `@annotation(position: (1, 2))` were accepted by the grammar, reached no visitor case, and vanished; `@annotation(size: (1, 2))` and `@annotation(zz: (1, 2))` worked.", "cause": "annotationParam is `annotationParamName COLON value | annotationValue`, and annotationParamName lists only IDENTIFIER plus a hand-maintained set of keywords (FROM, TO, TRUE, FALSE, TAIL). A keyword key does not FAIL the parse — it falls through to the positional alternative — so the parameter is consumed and means nothing.", "file": "mdl/grammar/domains/MDLSettings.g4 (annotationParamName), mdl/visitor/visitor_microflow_statements.go", "insight": "The failure mode of that grammar rule is silence, not a parse error, so a new annotation parameter cannot be assumed to work because a probe script parsed clean — probe that the VALUE arrives, not that the text is accepted. Words that are already MDL keywords are exactly the readable ones you reach for (`text`, `position`, `size`, `caption`), so this will keep recurring; anything added must be listed in annotationParamName. The visitor now records an unusable parameter on ActivityAnnotations.InvalidNotes and MDL079 refuses it, matching how @curve's InvalidCurves feeds MDL060.", "refs": ["mendixlabs/mxcli#1077", "mendixlabs/mxcli#884"], "ce": []} {"area": "mdl/executor", "date": "2026-09-10", "symptom": "A note written inside an `on error { … }` body arrived DETACHED — the Annotation object reached the model, its AnnotationFlow did not, so the note floated free on the canvas. DESCRIBE then never emitted it at all. Separately, any refusal raised while building a loop body was swallowed and exec reported success.", "cause": "Both sub-builders collect into their own slices and the merge back into the parent was incomplete: the error-handler merge copied objects and sequence flows but not annotationFlows or errors, and the loop merge copied annotationFlows but not errors. On the read side, collectErrorHandlerStatements is a SECOND describer (the main traversal never steps inside a handler block) and emitted no annotations.", "file": "mdl/executor/cmd_microflows_builder_flows.go (addErrorHandlerFlow merge), mdl/executor/cmd_microflows_builder_control.go (loop sub-builders), mdl/executor/cmd_microflows_show_helpers.go (collectErrorHandlerStatements)", "insight": "A sub-builder is a second copy of the builder's state and every field it collects needs an explicit line in the merge — the ones that get forgotten are the rare ones (annotationFlows) and the ones whose absence looks like success (errors). Grep the sub-builder's struct fields against the merge block rather than trusting it. The read side has the same shape: collectErrorHandlerStatements duplicates the traversal, so anything added to the main describer has to be added there too or the round trip loses it one nesting level down — fixing only the write half would have left the note attached in the model and still absent from DESCRIBE, which is differently wrong rather than fixed.", "refs": ["mendixlabs/mxcli#1077"], "ce": []} +{"area": "mdl/executor", "date": "2026-09-11", "symptom": "`ALTER WORKFLOW M.W INSERT OUTCOME 'X' ON ` printed \"Altered workflow\" and the project then could not be LOADED: `mx check` died at \"Loading the mpr file\" with System.InvalidCastException (UserTaskOutcome -> ConditionOutcome) at UnitContentsLoader.FillProperties, before validating anything; Studio Pro would not open it either (ako/mxcli#415). Five more combinations do the same: INSERT OUTCOME on a parallel split (InvalidCastException), INSERT PATH on a decision or a user task, and INSERT CONDITION on a user task or a parallel split (InvalidOperationException).", "cause": "An activity's outcome list is typed per activity kind (generated/metamodel: SingleUserTask/MultiUserTask hold UserTaskOutcome, ExclusiveSplit/CallMicroflowTask hold ConditionOutcome, ParallelSplit holds ParallelSplitOutcome), and each inserting ALTER op hard-codes ONE outcome $Type: InsertOutcome writes Workflows$UserTaskOutcome, InsertPath writes Workflows$ParallelSplitOutcome, InsertBranch writes Workflows$*ConditionOutcome. None of them read the target activity's kind — both mutator implementations (mdl/backend/wfmutator and mdl/backend/mcp) go straight from findActivityByCaption to appending the element, and the doc comments on both ('adds a new outcome to a user task', 'adds a concurrent path to a parallel split') state the intent that nothing enforced.", "file": "mdl/executor/validate_workflow_activity_kind.go (new), mdl/executor/validate_workflow_refs.go (one call in validateAlterWorkflowRefs), mdl/executor/alter_workflow_outcome_kind_test.go, mdl-examples/bug-tests/wf-415-alter-outcome-activity-kind.mdl", "insight": "The reported combination was one of six, and finding the other five cost nothing but a probe matrix: three inserting ops x the activity kinds they can be pointed at, each run against a project measured at 0 errors first. Build the matrix from `generated/metamodel` rather than from probing alone — it TYPES each activity's Outcomes slice, so it answers which combinations are legal without a single mxbuild run, and the probe then only has to confirm the consequence. The guard is written against the semantic types (a type switch over workflows.UserTask / ExclusiveSplitActivity / ...) instead of a table of Workflows$ storage-name strings, so an activity kind cannot inherit a wrong answer by being absent from a list; an unmodelled kind returns 'accepted' rather than being refused. Placement was the whole design: validateAlterWorkflowRefs is ALREADY the one function `check --references` and `exec` both call, so wiring the guard there got both passes for free and needed no backend-interface change, no mock stub, and no second copy for the MCP mutator — the alternative (a new WorkflowMutator.ActivityType method) would have had to be implemented twice and guarded at two call sites. Two measurements kept the scope honest and were both counter-intuitive. (1) INSERT BOUNDARY EVENT on a decision does NOT corrupt: it is a silent no-op (document byte-identical, 'Altered workflow' printed), because bsonnav.DSetArray does not create a key the document lacks - so it is refused as a no-silent-success defect, not as a load failure, and the changelog says which. (2) The DROP ops, left as 'not measured' in the issue, are NOT the same defect: removing an element cannot write a wrong type, and DROP OUTCOME / DROP PATH on a decision leave the project loadable at one ordinary CE6686. Resolve the target the way the mutator does (name OR caption, nested flows included, @N to disambiguate) and return nil when the reference is ambiguous or missing: a guard that refused on a failed lookup would convert a resolution problem into a type problem and reject valid scripts, and the mutator's own message is better. The fixture cannot be a .fail.mdl - the guard needs the STORED workflow to know the target's kind, and `make check-mdl` runs `check` with no project - so the bug-test file documents the six corrupting forms in comments and executes the correct form of each, verified at 0 errors on mxbuild.", "refs": ["ako/mxcli#415"], "ce": ["CE6686"], "rules": []} +{"area": "mdl/executor", "date": "2026-09-10", "symptom": "A SOAP `call web service` naming a mapping and service that REALLY EXIST produced a project Mendix could not LOAD: `mx check` stopped before validation with `Mendix.Modeler.Storage.StorageLoadException \u2026 The text 'c2d1682f-09de-4cc7-95a3-82d5ee5ef243' is not a valid ImportMappingIdentifier`. With the load fixed, the same call was still invalid: CE0386 'Operation GetOrder does not exist in consumed web service Clients.OrderSoapClient' (the operation does exist)", "cause": "Two independent defects, both about a name. (1) `resolveMappingRefForWrite` converted the receive mapping's qualified name to the mapping unit's `$ID`; ImportMappingCall.ReturnValueMapping is an ImportMappingIdentifier \u2014 a qualified name \u2014 so a UUID there is unloadable, not merely invalid. (2) ServiceName was derived as the local part of the imported service's qualified name ('OrderSoapClient'), but it is the WSDL `` ('OrdersWS'); Mendix resolves the operation WITHIN the named service, so a wrong one hides every operation", "file": "`mdl/executor/cmd_microflows_builder_calls.go`, `mdl/executor/webservice_names.go` (new)", "insight": "**A green gate can be green BECAUSE the fixture is broken.** The UUID substitution only happened when the mapping lookup SUCCEEDED. The only SOAP fixture (mdl-examples/doctype-tests/06b-soap-examples.mdl) names a service and mappings that do not exist \u2014 deliberately, to demonstrate dangling references \u2014 so exec took the fallback every time and the qualified name survived. A VALID reference was the single input that triggered an unloadable project, and no test used one. When a code path branches on 'did the lookup resolve', the fixture must cover BOTH branches; a fixture built to show error handling covers only one. **Second: fixing one name unmasks the next error, so work the chain against a real project rather than declaring victory at the first green.** Measured on ako/TestApp (11.14.0, baseline 0 errors), the same one-statement script went StorageLoadException -> CE0386 -> CE0243+CE0366+CE0178 as ReturnValueMapping, then ServiceName were fixed; each error was hidden by the one before it. **Third, two API traps found by debugging rather than reading**: `ListRawUnitsByType` matches the $Type EXACTLY despite its parameter being named typePrefix ('WebServices$ImportedService' returns 0, 'WebServices$ImportedServiceImpl' returns 1) \u2014 which is why the pre-existing resolveWebServiceReference, asking for 'WebServices$ImportedWebService', resolves nothing on any real project; and a unit unmarshalled into map[string]any nests sub-documents as maps, not bson.D, so a lookup asserting bson.D finds nothing, which is indistinguishable from 'no such service' and falls back to the wrong answer instead of failing", "refs": []} +{"area": "mdl/executor", "date": "2026-09-10", "symptom": "A SOAP `call web service` assigning its result was rejected twice over: `[CE0243] \"The mapping used to return a value of type 'Nothing', but now returns a value of type 'Clients.Order'\"` and `[CE0366] \"Cannot store in variable when there is no return value\"` \u2014 on a call whose receive mapping plainly produces an entity", "cause": "Both engines wrote the result handling's VariableType as `DataTypes$VoidType` unconditionally. Void means the call returns nothing, so it contradicts the mapping AND makes the assignment illegal. Studio Pro writes the entity the mapping produces: `DataTypes$ObjectType{Entity: \"Clients.Order\"}`, which is the Entity of the import mapping's ROOT `ImportMappings$ObjectMappingElement`", "file": "`mdl/executor/webservice_names.go` (resolveImportMappingEntity), `sdk/mpr/writer_microflow_actions.go`, `mdl/backend/modelsdk/microflow_webservice_write.go`", "insight": "**Fixing one wrong name in a SOAP call reveals the next; work the chain against a real project instead of stopping at the first green.** On ako/TestApp (11.14.0, baseline 0 errors) the SAME one-statement script went StorageLoadException (ReturnValueMapping written as a UUID) -> CE0386 (ServiceName derived instead of read) -> CE0243+CE0366 (VariableType Void) -> CE0178 (operation arguments), four rounds, each error invisible until the previous fix landed. Any of them could have been called 'the' bug. **The enabler each time was reading the referenced DOCUMENT rather than deriving from the statement**: the WSDL service name is in `Description.Services[].Name` of the imported service, the result entity in `Elements[0].Entity` of the import mapping \u2014 both structured, neither needing the embedded WSDL to be parsed. `ListRawUnitsByType` reaches them on both engines with no new backend method, but note it matches the $Type EXACTLY despite the parameter being named typePrefix, and the types are `WebServices$ImportedServiceImpl` and `ImportMappings$ImportMapping` (NOT the `Mappings$` prefix their child elements use). **Every resolver returns \"\" rather than guessing** \u2014 unresolvable, ambiguous, or wrong-shaped all fall back to what shipped, because a made-up name reproduces the same error with different text in it and is harder to recognise. Remaining and measured: CE0178 needs operation arguments, which MDL cannot express at all (callWebServiceStatement has no argument list), and Range.SingleObject differs from Studio Pro with no error yet attached \u2014 the reference mapping roots carry MaxOccurs 1 while the calls carry SingleObject false, so it is not the mapping's cardinality and would be a guess", "refs": []} +{"area": "mdl/executor", "date": "2026-09-11", "symptom": "A DESCRIBE-side resolver for SOAP references had been unreachable since it was written, and its unit test passed. `describe microflow` printed the right service and mapping names throughout", "cause": "`resolveWebServiceReference` looked the service up by ELEMENT ID among units of type `WebServices$ImportedWebService`. It could not match on two independent counts: the stored $Type is `WebServices$ImportedServiceImpl` (ImportedWebService is the SDK name, and nothing is stored under it), and the value compared was never an id \u2014 ImportedService is a BY_NAME_REFERENCE, so it already held `Clients.OrderSoapClient`. Same for the two mapping resolvers. Every call fell through to a fallback that returned the stored string, which is the correct answer", "file": "`mdl/executor/cmd_microflows_format_action.go` (formatWebServiceCallAction; the five resolvers removed)", "insight": "**A resolver whose fallback is the correct answer is indistinguishable in its output from one that works \u2014 so only the input side can prove it runs.** Nothing in the DESCRIBE text could ever have been wrong, which is why this survived: the test that covered it, `TestFormatAction_WebServiceCallResolvesKnownReferences`, built a world where ServiceID WAS a unit id and the unit type WAS `ImportedWebService`, neither of which occurs in any project \u2014 the green test asserted the fiction, not the code. The replacement inverts it: the mock backend calls `t.Fatal` if it is consulted at all, so the test fails unless no lookup happens. **The wider fact, measured, is that the structured DESCRIBE branch is unreachable for real SOAP calls anyway**: all three of ako/TestApp's carry 15 keys and `webServiceActionRequiresRawBSON` supports 9, and mxcli's own writer emits the same 15, so every SOAP call on either engine describes as `call web service raw ''`. A branch nothing reaches cannot be validated by any amount of passing tests over it. **Check a BY_NAME_REFERENCE before writing a resolver**: `modelsdk/gen/*/refs.go` states the kind (`codec.RefByName` here), and an existing test two packages away was already passing `Mod.Service` as the value", "refs": []} +{"area": "mdl/executor", "date": "2026-09-11", "symptom": "An XPath constraint naming an association BARE \u2014 `[Ticket_Reporter = $currentUser]` \u2014 passes `mxcli check --references`, `exec` reports success, and the build then fails with `Error(s) in XPath constraint` (CE0161) naming the retrieve activity. Because `exec` applies statements one at a time and cannot roll back, a script carrying this in the middle leaves the model half-updated", "cause": "An association in an XPath constraint must be QUALIFIED (`Module.Assoc`); an attribute is bare. Nothing checked it. The fix (MDL-XPATH01, `mdl/executor/validate_xpath_association.go`) flags a bare name that is NOT an attribute of the constrained entity AND IS a known association, so it can name the spelling to use. Measured on a blank Mendix 11.14.0 app with qualification as the only variable: unqualified \u2192 check passed / build exit 3; qualified \u2192 check passed / BUILD SUCCEEDED", "file": "`mdl/executor/validate_xpath_association.go`, wired from `validateFlowBodyReferences` in `validate.go`; script-declared associations collected by `scriptContext.recordAssociation`", "insight": "**`scriptContext` has TWO parallel collectors over the same statement types \u2014 `collectDefinitions` (whole program) and `collectSingle` (incremental) \u2014 and a case added to one only is silently half-collected.** That is exactly how this rule first shipped: it fired against STORED associations and stayed silent on script-declared ones, which is the majority shape (one script creating the entity, the association and the microflow together) and the one that reaches a build half-written. Unit tests on the matcher were all green, because the rule was correct and simply never given the data \u2014 the miss only showed up running the real `.mdl` through `make check-mdl`. Both collectors now call one `recordAssociation`/`recordEntityAttrs` helper, and `TestBothCollectorsRecordAssociationsAndAttrs` asserts they agree (control: reverting the `collectDefinitions` case fails it with `associations[X] = \"\"`). Second lesson: **a rule that needs a project cannot be tested by a `.fail.mdl`** \u2014 `make check-mdl` runs `mxcli check` with NO project, so such a file passes silently and asserts nothing. The negative case belongs in Go; keep the `-ok.mdl` as the positive control so the rule cannot degrade to rejecting every constraint", "refs": ["ako/ChipCoV1 FINDINGS.md"]} diff --git a/.claude/skills/mendix/cheatsheet-errors/SKILL.md b/.claude/skills/mendix/cheatsheet-errors/SKILL.md index 0bed537717..7a97e60e26 100644 --- a/.claude/skills/mendix/cheatsheet-errors/SKILL.md +++ b/.claude/skills/mendix/cheatsheet-errors/SKILL.md @@ -265,7 +265,8 @@ Run with `-p` for the fullest coverage. | CE0104 | Action activity is unreachable | Code after RETURN | | CE0105 | Must end with end event | Missing RETURN | | CE0117 | Error in expression | Unqualified association path | -| CE1571 | No argument selected for parameter | `$currentObject` in a control-bar button (not row-scoped) — `check` flags MDL-BUTTON01 | +| CE1571 | No argument selected for parameter | A microflow/nanoflow call with a parameter nothing fills — as a `datasource:` **or** an `action:`. Give it an argument (`action: nanoflow M.NF($P = $value)`), or nest the widget in a data container of the parameter's type. `check -p` reports both | +| CE1571 | …in a control bar | A control bar is **not** row-scoped, so the grid's row does not fill it: pass the grid's selection (`$dgOrders`, with `Selection:` set) or move the widget into a column. `$currentObject` there is MDL-BUTTON01 | | CE1834 | The 'Page' property is required | Workflow user task without a `page` — `check` flags MDL-WF01 | | CE1876 | Single outcome must not contain activities | Single-outcome user task with a nested activity flow — `check` flags MDL-WF02 | | CE6686 | Outcomes do not match the configured expression/microflow | An enumeration decision or call-microflow activity missing the empty outcome (`'' -> { }`) or an enumeration value — `check` flags the missing empty one as MDL-WF06 | diff --git a/.claude/skills/mendix/create-page/reference/widgets.md b/.claude/skills/mendix/create-page/reference/widgets.md index 4a462da8c3..7774da193b 100644 --- a/.claude/skills/mendix/create-page/reference/widgets.md +++ b/.claude/skills/mendix/create-page/reference/widgets.md @@ -112,6 +112,11 @@ describe icon collection Atlas_Core.Atlas_Filled -- every icon + its reference - `action: microflow Module.MicroflowName(Param: $value)` - Call microflow with parameters - `action: nanoflow Module.NanoflowName` - Call nanoflow (client-side) - `action: nanoflow Module.NanoflowName(Param: $value)` - Call nanoflow with parameters +- `action: nanoflow Module.NanoflowName($Param = $value)` - Also accepted (microflow-style) +- **Every parameter needs an argument, or an enclosing data container of its + type.** A flow called with a parameter nothing fills is **CE1571**; `mxcli + check -p` reports it. This is the same on every widget that takes an action, + a clickable `container` included. - `action: show_page Module.PageName` - Navigate to page - `action: show_page Module.PageName(Param: $value)` - Navigate with parameters - `action: show_page Module.PageName($Param = $value)` - Also accepted (microflow-style) @@ -847,6 +852,42 @@ controlbar controlBar1 { } ``` +**A control bar is not row-scoped.** It sits above the rows, so the grid's +current object is not in scope there — an action calling a flow with a parameter +gets **CE1571** unless you give it an argument, and `$currentObject` is unbound +(`mxcli check` reports MDL-BUTTON01). The remedy is the grid's **selection**, +addressed by the widget's own name and available once `selection:` is set: + +```sql +datagrid dgMaterials ( + datasource: database from Module.Material, + selection: single +) { + column colName (attribute: Name, caption: 'Name') + + -- Row-scoped: the grid's row supplies the parameter, no argument needed. + column colRow (caption: 'Row') { + container cRowUnlink (action: nanoflow Module.ACT_UnLink, class: 'command') { + actionbutton btnRowUnlink (caption: 'Unlink') + } + } + + controlbar controlBar1 { + -- Not row-scoped: pass the selection explicitly. + container cUnlink ( + class: 'command', + action: nanoflow Module.ACT_UnLink ($Material = $dgMaterials) + ) { + actionbutton btnUnlink (caption: 'Unlink') + } + } +} +``` + +A **container** takes an argument list exactly like an `actionbutton` does — the +two share one action grammar. Reaching for a button because a container "cannot +pass parameters" changes the rendering for no reason (mendixlabs/mxcli#1082). + ### Charts (Charts.mpk — ColumnChart / BarChart / AreaChart / PieChart) Charts are pluggable widgets whose data lives in one or more `series` object-list diff --git a/.claude/skills/mendix/download-marketplace-content/SKILL.md b/.claude/skills/mendix/download-marketplace-content/SKILL.md index 5c7432314c..fbee61d838 100644 --- a/.claude/skills/mendix/download-marketplace-content/SKILL.md +++ b/.claude/skills/mendix/download-marketplace-content/SKILL.md @@ -217,6 +217,20 @@ an mxcli defect**: design properties an older Atlas spelled differently. Project-level: the location in the check output is empty, so the message alone does not say which module caused it. +A third, **CE0066** "Entity access is out of date", is *not* left to you. The install and +the update copy the incoming module's units in verbatim, so a package whose access rules do +not cover every member of their entities used to land that error with nothing said about it +([mendixlabs/mxcli#1085](https://github.com/mendixlabs/mxcli/issues/1085)). Both now +reconcile the module's rules for themselves and report the count; nothing is written when +the rules already match. When the line does appear, the module's domain model now differs +from the package, so `marketplace diff` reads it as a local edit — the same consequence +Studio Pro's **Update security** button has. Repair it by hand, on a project updated by an +older mxcli, with: + +```bash +mxcli -p app.mpr -c "update security UserCommons" +``` + Measured end to end on a vanilla 11.12.1 app carrying the agent-editor stack: `mx check` reported **203 errors** (202 × CE0463 + 1 × CE6087) and **0** after the two commands, with the project still MPR v2 — 1,868 `.mxunit` files, a 249,856-byte index, before and after. diff --git a/.claude/skills/mendix/manage-security/SKILL.md b/.claude/skills/mendix/manage-security/SKILL.md index 92908590a8..5f06eadaa5 100644 --- a/.claude/skills/mendix/manage-security/SKILL.md +++ b/.claude/skills/mendix/manage-security/SKILL.md @@ -304,6 +304,18 @@ Three things worth knowing, each of which was a defect until A run that skipped something says so. `All entity access rules are up to date` means every module was looked at. +The commonest source of a stale rule is a module that arrived from outside +Studio Pro. `mxcli marketplace install` and `mxcli marketplace update` copy the +incoming units in verbatim, so a package whose rules do not cover their entities' +members used to land CE0066 in the project with nothing said about it +([mendixlabs/mxcli#1085](https://github.com/mendixlabs/mxcli/issues/1085)); both +now run this reconcile for the module they copy in and report the count. Run it +by hand for a module installed some other way, or by an older mxcli: + +```bash +mxcli -p app.mpr -c "update security UserCommons" +``` + A member name that matches nothing is now an error rather than a silent skip: ``` diff --git a/.claude/skills/mendix/project-brain/SKILL.md b/.claude/skills/mendix/project-brain/SKILL.md index d626e6d81b..eddad58933 100644 --- a/.claude/skills/mendix/project-brain/SKILL.md +++ b/.claude/skills/mendix/project-brain/SKILL.md @@ -131,6 +131,23 @@ more decisions: So anchor a requirement at what you are *going to* build. `@Sales.ACT_Order_Approve` before that microflow exists is correct, not a mistake. +**Never anchor a requirement at a bare module.** `@Sales` resolves the instant +the module exists — long before any of the work inside it — so the requirement +reports **built** with nothing done. `mxcli brain capture --requirement` refuses +it and names the alternative, because the failure is silent and flattering: +the plan shows progress that has not happened and nothing else disagrees. +Anchor at a document the slice actually creates. (Measured on a real project: +two requirements anchored at a module both read as built after slice 01.) + +A module **role** is a fine anchor and resolves like any document — a security +requirement anchored at the roles it creates is measured correctly. Anchoring at +the documents whose access rules the roles govern works too, and says something +slightly different; either is legitimate. + +A **theme** has no model element to anchor at (it is files under `theme/`, which +is the point of `mxcli theme`). Anchor the branding requirement at the branded +**layout** the slice adds — the model-side half of the same work. + ### Progress is derived, never written ```bash diff --git a/.claude/skills/mendix/record-narrated-demo/SKILL.md b/.claude/skills/mendix/record-narrated-demo/SKILL.md index e781bf83a2..7c56a13d38 100644 --- a/.claude/skills/mendix/record-narrated-demo/SKILL.md +++ b/.claude/skills/mendix/record-narrated-demo/SKILL.md @@ -107,6 +107,38 @@ DataGrid2 screens were unreadable: eight columns compressed to eight single characters, headers degraded to bare sort arrows. The app *functioned* on a phone and was not *usable* on one, and only the mobile recording showed the difference. +**A narrow viewport is not a mobile profile.** Mendix picks its navigation +profile from the **user agent**, not from the window size, so a take that only +shrinks `viewport` films the *desktop* app in a narrow window — the phone profile +is never routed to, and the pass cannot show the thing it exists to find while +looking entirely plausible. Pass the device through `contextOptions`: + +```js +const take = await openTake(browser, { + size: { width: 430, height: 932 }, + contextOptions: { + userAgent: 'Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) ' + + 'AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1', + isMobile: true, + hasTouch: true, + deviceScaleFactor: 3, + }, +}); +``` + +`viewport` and `recordVideo` are set by `openTake` itself and win over anything +in `contextOptions`, because both are load-bearing for the cut — a device preset +carrying its own `viewport` would silently letterbox every take. + +Also worth knowing before you read a mobile take as a layout bug: **a Mendix page +carries its own layout, and the layout names the navigation profile.** The Phone +profile controls the home page and the menu; it does not re-skin the pages a user +reaches afterwards. A phone user routed to a page built on a desktop layout gets +the desktop frame whatever profile routed them there — measured at 430×932, a +232 px rail on a 430 px screen with the row's action laid out 42 px past the +right edge. That is a real defect and the take is right to fail on it, but the +fix is per-page layouts, not a theme tweak. + ### `recordVideo` needs a Node script, not `playwright-cli` `mxcli verify`'s browser checks run bash scripts against a persistent diff --git a/.claude/skills/mendix/record-narrated-demo/narrate.js b/.claude/skills/mendix/record-narrated-demo/narrate.js index 22b45a1cf7..f04cfdf842 100644 --- a/.claude/skills/mendix/record-narrated-demo/narrate.js +++ b/.claude/skills/mendix/record-narrated-demo/narrate.js @@ -15,8 +15,18 @@ // const OVERLAY_CSS = ` + /* pointer-events: none for the same reason #demo-spot has it, and the reason + is easy to miss here: the plate is a full-width bar pinned to the BOTTOM of + the viewport, which is exactly where Mendix puts a page footer's buttons. + Without this, any control the plate covers becomes unclickable and + Playwright retries for 30s against "div.text from div#demo-narration + subtree intercepts pointer events" before failing the take — a failure that + names the overlay but not the reason. Reported by ako/ChipCoV1, where it + killed the take on an Approve button. The caption is read, never clicked, + so it gives up pointer events for free. */ #demo-narration { position: fixed; left: 0; right: 0; bottom: 0; z-index: 2147483647; + pointer-events: none; display: flex; align-items: center; gap: 14px; padding: 16px 22px; background: rgba(17, 24, 39, .94); @@ -49,6 +59,15 @@ const OVERLAY_CSS = ` opacity: 0; } #demo-spot.on { opacity: 1; animation: demo-pulse 1.6s ease-in-out infinite; } + + /* Reserve the plate's own height at the foot of the page. pointer-events + above makes a covered control CLICKABLE; this makes it VISIBLE, and the + film needs both — a click that lands under an opaque caption is a beat the + viewer cannot see happen, which is the same dead beat by another route. + Applied once at install, before the take starts, so nothing shifts mid-shot. + --demo-plate-height is overridable for an unusually long caption. */ + :root { --demo-plate-height: 96px; } + body { padding-bottom: var(--demo-plate-height) !important; } @keyframes demo-pulse { 0%, 100% { box-shadow: 0 0 0 4px rgba(37,99,235,.22); } 50% { box-shadow: 0 0 0 9px rgba(37,99,235,.10); } diff --git a/.claude/skills/mendix/record-narrated-demo/take.js b/.claude/skills/mendix/record-narrated-demo/take.js index 589b9b118c..a21f0edd2a 100644 --- a/.claude/skills/mendix/record-narrated-demo/take.js +++ b/.claude/skills/mendix/record-narrated-demo/take.js @@ -50,6 +50,24 @@ const DEFAULTS = { // failure mode is to keep going and hand you a plausible-looking film with a // dead beat in it; fail loudly instead. strict: true, + // Anything else Playwright's newContext() takes — merged in below. The mobile + // pass this skill MANDATES needs `userAgent`, and needs it here rather than in + // `viewport`, because MENDIX PICKS ITS NAVIGATION PROFILE FROM THE USER AGENT + // AND NOT FROM THE VIEWPORT. Without this passthrough a "mobile" take films + // the DESKTOP app in a narrow window: the phone profile is never routed to, so + // the recording cannot show the thing the pass exists to find, and it looks + // plausible while doing it. + // + // contextOptions: { + // userAgent: 'Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) …', + // isMobile: true, + // hasTouch: true, + // deviceScaleFactor: 3, + // } + // + // Reported by ako/ChipCoV1, which shipped an app whose phone walk was + // impossible while seven browser tests stayed green. + contextOptions: {}, }; async function openTake(browser, opts = {}) { @@ -63,7 +81,13 @@ async function openTake(browser, opts = {}) { // gap between this instant and the first mark is the offset the cut must // subtract. Capture it before newContext so it is never an underestimate. const videoT0 = Date.now(); + // contextOptions is spread FIRST so viewport and recordVideo still win: those + // two are load-bearing for the cut (see the size/viewport note above), and a + // device preset that carried its own viewport would silently letterbox every + // take. Everything else a device needs — userAgent, isMobile, hasTouch, + // deviceScaleFactor — passes straight through. const context = await browser.newContext({ + ...o.contextOptions, viewport: o.viewport, recordVideo: { dir: o.videoDir, size: o.size }, }); diff --git a/.claude/skills/mendix/write-workflows/SKILL.md b/.claude/skills/mendix/write-workflows/SKILL.md index d78693aa12..a173381e5a 100644 --- a/.claude/skills/mendix/write-workflows/SKILL.md +++ b/.claude/skills/mendix/write-workflows/SKILL.md @@ -171,6 +171,24 @@ Consecutive `set`s may chain in one statement: See `mdl-examples/doctype-tests/24-workflow-examples.mdl` for the full ALTER surface (insert path, drop path, insert condition, boundary events). +**The INSERT op has to match the activity kind.** An activity's outcome list is +typed, and each op writes exactly one outcome type into it: + +| Op | Writes | Only on | +|----|--------|---------| +| `insert outcome '' on X { }` | `UserTaskOutcome` | a user task | +| `insert condition '' on X { }` | `…ConditionOutcome` | a decision, a call microflow | +| `insert path on X { }` | `ParallelSplitOutcome` | a parallel split | +| `insert boundary event on X timer '' { }` | a boundary event | user task, call microflow, call workflow, wait for notification | + +Aim one at the wrong kind and the outcome lands in a list that cannot hold it, +which is **not** a build error: the project stops **loading**, so Studio Pro will +not open it and `mx check` dies before it validates anything (ako/mxcli#415). +mxcli refuses all of these now — at `check --references` and at `exec`, which +call the same function — and the refusal names the op that fits the target. The +`drop` ops are unaffected: removing a branch cannot write a wrong type, and it +leaves an ordinary build error (`CE6686`) rather than an unloadable project. + ## DESCRIBE round-trip `DESCRIBE WORKFLOW Module.Name` emits **executable, re-runnable** MDL — user diff --git a/.claude/skills/mendix/xpath-constraints/SKILL.md b/.claude/skills/mendix/xpath-constraints/SKILL.md index ce80c51196..5fe1b288ad 100644 --- a/.claude/skills/mendix/xpath-constraints/SKILL.md +++ b/.claude/skills/mendix/xpath-constraints/SKILL.md @@ -158,6 +158,15 @@ where [not(Module.Order_Customer/Module.Customer)] **Rule**: Always use the fully qualified association name (`Module.AssociationName`). +> **A bare association name is now caught before the build (MDL-XPATH01).** +> `[Order_Customer = $currentUser]` used to pass `mxcli check --references`, get +> written by `exec`, and only fail at the build with *"Error(s) in XPath +> constraint"* (**CE0161**) — which is the expensive shape, because `exec` cannot +> roll back and stops with the model half-updated. `check` now names the +> association and the qualified spelling to use instead. It fires only when the +> bare name is not an attribute of the constrained entity **and** is a known +> association, so attributes stay bare and XPath functions are never touched. + > **`= empty` does not work on associations (CE0161 / MDL047).** `= empty` tests > *attribute* nullability only. To test whether an object *has no* associated > object, use negated existence: `[not(Module.Order_Customer/Module.Customer)]` — diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c0845ce93..de88d3536b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,60 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed +- **The SessionStart hook re-downloaded 85 MB on every session when mxcli was already on `PATH`** (reported in ako/ChipCoV1's FINDINGS.md) — `bootstrap-mxcli.sh` gated only on `[ ! -x ./mxcli ]` and never consulted `PATH`, while the bootstrap skill has you delete the hardlink `mxcli new` leaves in the project. In a session image with mxcli pre-installed the guard could therefore never be satisfied by the copy already on the machine, forever. It now hardlinks a `PATH` copy into place (falling back to a symlink across filesystems, then a copy) and downloads only when none of those work. With nothing on `PATH` it still downloads, and an existing `./mxcli` is still left untouched. + +- **`theme create --from ` seeded the palette and nothing else** — a scaffolded brand theme still described itself in `mxcli theme list` as *"Cool slate, one teal signal colour"* and showed Signal's six swatches, so the one command whose job is to tell themes apart showed the wrong one. The colorway is now derived from the seeded brand/info/success/warning/danger/ink-muted (falling back to the base's for entries a design did not declare, since a partial seed is normal), and the summary names the design file rather than describing the base's colours. + + It also vendored ~500 KB of IBM Plex `.woff2` that the seeded `--mxt-font` never names, with a SIL OFL licence for fonts the theme does not use. A family the seeded fonts no longer name now has its `@font-face` rules **and** its files dropped together — per family, so a theme that changes its body font and keeps IBM Plex Mono for code ships only the mono weights. A design that says nothing about fonts keeps them all. + +- **The primary button was never the brand colour.** Atlas derives `--btn-primary-bg` from `--brand-primary-600` = `color-mix(in srgb, , 20%)`, so a brand blue `#10069F` rendered as `rgb(21,13,140)`. mxcli pinned `--btn-primary-color` to `--mxt-brand-ink` — an ink each theme picks to sit on `--mxt-brand`, console pairing near-black `#04211d` with bright teal `#2dd4bf` — while leaving the background to Atlas's derivative, so the pairing the theme designed for was never the pairing that rendered. The map's own comment already called it *"a brand-filled button"*; now it is one, in all three themes, with hover and active still tracking Atlas's relationship to the base. + +- **`brain plan` mismeasured two kinds of requirement anchor, in opposite directions** (reported in ako/ChipCoV1's FINDINGS.md) — a requirement anchored at a bare **module** reported *built* the instant the module existed, with none of its work done; one anchored at a **module role** reported *planned* forever, even once the roles existed. Both went through `catalogResolver.Resolve`, and they need opposite fixes. + + The module-role half is a **lookup** gap: a role is in neither the catalog's `objects` view nor `FindDocumentUnit` (it is not a document), so both lookups missed and it fell through to `NotFound` — which, for a requirement whose anchors point forward, means "not built yet" permanently. The resolver now consults `GetModuleSecurity`, case-insensitively, because Mendix treats role names that way and an anchor is written by hand. Verified in both directions on a real project: `@MyFirstModule.User` → *1 anchor, 1 resolved*; `@MyFirstModule.NoSuchRole` → **NOT FOUND**, exit 1, so the resolver has not simply become permissive. + + The module half is **semantic** and no lookup can fix it: the anchor resolves perfectly and is still useless, because resolution *is* the progress signal and a module resolves before any of the work inside it. `NewRequirement` now refuses it and names the alternative — an author told only "no" deletes the anchor, which loses the measurement rather than fixing it. A module anchor stays legal on a **decision** and on a **question**, whose anchors point backward; that control is what stops the fix degrading into "refuse every module anchor". + +- **An unqualified association in an XPath constraint passed every check and failed the build** (MDL-XPATH01, reported in ako/ChipCoV1's FINDINGS.md) — `[Ticket_Reporter = $currentUser]` passed `mxcli check --references`, `exec` reported success, and mxbuild then failed with *"Error(s) in XPath constraint"* naming the retrieve activity. An association in a constraint must be written qualified; an attribute is written bare, and nothing checked the difference. + + Measured on a blank Mendix 11.14.0 app with the qualification as the only variable between two runs of the same script: `[Ticket_Reporter = …]` → check passed, build exit 3; `[MyFirstModule.Ticket_Reporter = …]` → check passed, **BUILD SUCCEEDED**. This is the expensive shape rather than a cosmetic one, because `exec` applies statements one at a time and cannot roll back — a script carrying it in the middle writes everything before it and stops at the build, with nothing having reported a problem. + + The rule is deliberately narrow: it fires only on a bare name that is **not** an attribute of the constrained entity **and is** a known association, so it names the spelling to use rather than merely suspecting one. That narrowness is also why it needs no XPath keyword list — `and`, `or`, `not` and `contains` are not association names. Names inside string literals are skipped, with Mendix's doubled-quote escaping tracked so a literal cannot swallow the rest of the constraint. + + It reads the script's own declarations as well as the project's, which is most of the value: the common shape is one script creating the entity, the association and the microflow that constrains on it. The first cut collected only on `scriptContext.collectSingle` and not on its twin `collectDefinitions`, so it fired against stored associations and stayed silent on exactly those scripts — green unit tests throughout, because the matcher was right and simply never given the data. Both collectors now share one helper and `TestBothCollectorsRecordAssociationsAndAttrs` asserts they agree. + +- **`run --local --watch` could not start a Mendix 11.14 app** — the same defect as ako/mxcli-ledger #146, in the copy of the gate that fix did not touch. `--watch` died at `starting web client bundler: no rollup.config.mjs in …/deployment/web (run a serve Deploy build first)`, and the advice was not actionable because a serve Deploy build had just run: 11.14 bundles the browser client itself and emits no rollup config, because there is nothing left to configure. The reporting project routed around it by using plain `run --local --screenshot` with a restart per change. +- **`run --local --watch` could not start a Mendix 11.14 app** — the same defect as ako/mxcli-ledger #146, in the copy of the gate that fix did not touch. `--watch` died at `starting web client bundler: no rollup.config.mjs in …/deployment/web (run a serve Deploy build first)`, and the advice was not actionable because a serve Deploy build had just run: 11.14 bundles the browser client itself and emits no rollup config, because there is nothing left to configure. The reporting project routed around it by using plain `run --local --screenshot` with a restart per change. + + #146 measured the inversion on a blank app with the version as the only variable — 11.13 `rollup.config.mjs` PRESENT / `dist/index.js` ABSENT, 11.14 exactly the reverse — and moved the gate onto the gap the step closes. But that fix landed on `BuildWebClient` alone, and `StartWebClientWatch` carries its own independent copy of the same `os.Stat(rollup.config.mjs)` check. So `run --local` started working on 11.14 while `run --local --watch` kept failing, one call earlier, on the same absent file. The commit message asserted *"both call sites are fatal"*; the two call sites it meant were both `BuildWebClient`'s. + + On 11.14 there is no bundler to keep hot and nothing for it to do, so `StartWebClientWatch` now returns a **nil watcher** rather than an error, and every method on `*WebClientWatcher` is nil-safe — `Generation`, `WaitForRebuild`, `Log`, `Stop`. That matters as much as the gate: the watch loop calls three of them per change with no branch, so a fix that only stopped the startup refusal would have traded it for a segfault on the first file edit (`TestNilWebClientWatcherIsSafeToUse` fails with a SIGSEGV against the unfixed build). The 11.13 path is unchanged and covered by a control that requires the bundler launch to still be attempted, and "neither a config nor a bundle" is still an error. + + Fixing the gate then exposed a **defect in mxbuild 11.14 itself**, which the gate had been hiding: the first build in an `mxbuild --serve` process does not leave the deployment in a state its own incremental build can continue from — neither the bundler config nor `web/pages/`/`web/layouts/` survive it — so the first build succeeds and every later one fails. Measured over mxbuild's own HTTP API with no mxcli involved: the same `/build` request POSTed twice, model untouched between them, goes **Success then Failure** (`ERR_MODULE_NOT_FOUND … web/rollup.config.mjs`, imported from mxbuild's own `rollup-runner.mjs`). A one-shot `mxbuild --target=deploy` run twice into the same deployment directory succeeds both times, so it is the serve process rather than the 11.14 deployment shape. + + It is not the app's bundler choice: with *App Settings > Runtime > App bundler* flipped to Rspack the failure is identical and names `rspack.config.mjs`. That control matters — without it, "the incremental build falls back to the legacy Rollup pipeline" is a plausible, quotable and wrong diagnosis. mxcli cannot fix this from outside the process, so `legacyClientBuildHint` recognises it by its own **shape** (the failure names an artifact AND the artifact is genuinely absent), never by Mendix version, and says what does *not* work — deleting `deployment/`, switching bundlers — because both are what a user tries first. Docs say plainly that `--watch` is not usable on 11.14 yet, with the restart-per-change command that is. Worth noting `--serve` is in `mxbuild --help` but not on docs.mendix.com, so it carries no compatibility promise. + On 11.14 there is no bundler to keep hot and nothing for it to do, so `StartWebClientWatch` now returns a **nil watcher** rather than an error, and every method on `*WebClientWatcher` is nil-safe — `Generation`, `WaitForRebuild`, `Log`, `Stop`. That matters as much as the gate: the watch loop calls three of them per change with no branch, so a fix that only stopped the startup refusal would have traded it for a segfault on the first file edit (`TestNilWebClientWatcherIsSafeToUse` fails with a SIGSEGV against the unfixed build). The 11.13 path is unchanged and covered by a control that requires the bundler launch to still be attempted, and "neither a config nor a bundle" is still an error — with the message that names both halves, so a user on 11.14 does not read the old text and look for a build that already ran. +- **An `ALTER WORKFLOW` insert aimed at the wrong activity kind wrote a model Mendix cannot load** (ako/mxcli#415) — `alter workflow M.W insert outcome 'X' on decision9 { };` printed `Altered workflow`, and the project then failed to **load**: `mx check` died at *"Loading the mpr file"* with `System.InvalidCastException: Unable to cast object of type 'UserTaskOutcome' to type 'ConditionOutcome'`, before validating anything, so Studio Pro would not open it either. The MDL-WF04 class — the blast radius is the whole project, not one document — and every mxcli-side command reported success, which is what let it survive any number of later scripts. + + An activity's outcome list is **typed**, and each inserting op writes exactly one outcome type into it: `INSERT OUTCOME` a `UserTaskOutcome`, `INSERT PATH` a `ParallelSplitOutcome`, `INSERT CONDITION` a `…ConditionOutcome`. `generated/metamodel` types the receiving list per activity, and none of the three ops checked what it was pointed at. + + The report named one combination; there are **six**, and all of them were measured on mxbuild 11.10.0 against a project sitting at 0 errors. `INSERT OUTCOME` on a decision or a parallel split is `InvalidCastException`; `INSERT PATH` on a decision or a user task, and `INSERT CONDITION` on a user task or a parallel split, are `InvalidOperationException`. All six are refused now, by the one function `check --references` and `exec` already share, and the refusal names the op that *does* fit the target rather than only saying no. + + `INSERT BOUNDARY EVENT` on a decision or parallel split is refused for a different, measured reason: it did not corrupt anything — it printed `Altered workflow` and left the document byte-identical. A statement that reports success and changes nothing is its own defect, and only four activity kinds declare a `BoundaryEvents` list. + + The `DROP` ops are deliberately untouched, which the issue had left open as unmeasured: removing an element cannot write a wrong type, and `DROP OUTCOME` / `DROP PATH` on a decision each leave the project **loadable** at one ordinary `CE6686` — the correct result of removing a branch from an enumeration decision. +- **`marketplace update` and `marketplace install` left the model at CE0066, with nothing said about it** (mendixlabs/mxcli#1085) — a headless module update exited 0, reported *"81 units copied, 45 element identities preserved, 6 role grant(s) restored"*, and `mx check` then gave **CE0066** *"Entity access is out of date. Please update security by clicking the 'Update security' button in the domain model editor"* at the module's domain model. The command's own *"Next, repair what a headless update leaves behind"* block named CE0463 and CE6087 only, so the error read as needing Studio Pro — it was reported as *"no headless fix available"*, which is what breaks an unattended upgrade pipeline. + + A transplant is the one mxcli write path that does **not** reconcile as it writes. `TransplantModule` copies the incoming module's units in verbatim, and `RestoreRoleGrants` runs its statements one at a time rather than as a program, so the executor's finalize step — the only caller of `ReconcileMemberAccesses` outside the entity, grant and association handlers — never runs. An access rule that does not cover every member of its entity therefore arrived exactly as the package shipped it. + + `PerformUpdate` and `PerformInstall` now reconcile the module they copied in, and report the count. A failure there is reported rather than returned — the module is already in place, and aborting would leave the project mid-update — and names `update security ` as the repair. + + A module whose rules are already complete is **not** rewritten, which is the other half of the contract rather than an optimisation: reconciling unconditionally would make every install differ from its package, `marketplace diff` would read that as a local edit, and a local edit is what makes the next update refuse. + + Measured on a Mendix 11.6.6 app with Administration 4.3.2, mxbuild 11.14.0 — the control first, because it is what localises the defect: transplanting the module **unchanged** checks at 0 errors, so the machinery does not create the stale rule, it transmits one. Removing a single member entry from the incoming version's rules then gives exactly the reported CE0066, and 0 errors with the fix. + + The reporter's second point — that no headless repair existed — was already untrue, and that is the more useful half of this: `UPDATE SECURITY` has done this since mendixlabs/mxcli#1047, but appeared in no user-facing surface at all (not the docs site, not `MDL_QUICK_REFERENCE.md`, not `mxcli syntax`, not the update's output), only in a skill and this file. It is now documented in all of them. Note also that v0.21.0 could not have run it successfully in every case: it carries the System-skip fix but not the inherited-attribute reconcile that landed the day after the release, so where the missing member is one a specialization inherits, that build reports *"All entity access rules are up to date"* over a broken model. + - **`mxcli check -p` emptied the catalog's source index** (mendixlabs/mxcli#1081) — running a check against a project shrank `.mxcli/catalog.db` and left `CATALOG.SOURCE` at zero rows. Measured on `testdata/expr-checker`: a source-mode cache of 2,191,360 bytes with 106 source rows became 1,253,376 bytes with 0. Not just source — `REFS` 271→0, `PERMISSIONS` 162→0, `STRINGS` 1496→0, `XPATH_EXPRESSIONS` 5→0, so `show references`/`callers`/`impact`, `search` and `show languages` answered "requires refresh catalog full source" instead of answering. The catalog cache records the mode it was built in, and the modes nest: source ⊃ full ⊃ fast. `check`'s expression-type tier asks for a **fast** catalog — attribute types, enumeration cases, microflow return types — and while the cache is valid a source cache satisfies that, so nothing was lost. The loss began the moment the cache went stale, which in practice means *any* project save, `mxcli exec` or branch switch between two commands: the fast rebuild was then written over the cache unconditionally. A build now **never narrows the cache**, guarded at the single save site rather than at `check`, because `show structure`, `describe`, `show catalog tables` and plain `refresh catalog` all reach it through the same seam and reproduce the bug identically. @@ -27,6 +81,18 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). `ALTER WORKFLOW` is covered too, and was the second half of the same gap: an `INSERT AFTER … decision` (or `REPLACE ACTIVITY`, or an activity inside an inserted outcome, path, branch or boundary event) reaches the identical CE6686 — measured, inserting one into a project sitting at 0 errors takes it to 1 — and no workflow rule had ever looked at an ALTER-introduced activity. MDL-WF06 now runs over what the statement adds. The other workflow rules deliberately stay CREATE-only: MDL-WF01/WF02 describe a state a later `SET ACTIVITY` in the same script can repair, and MDL-WF05 resolves jump targets against activities an ALTER statement cannot see. Verified in both directions on a blank 11.10.0 app: `mdl-examples/bug-tests/wf-enum-decision-empty-outcome.fail.mdl` — three flagged constructs, a decision, a call microflow and an `ALTER`-inserted decision — executed with `--no-check` gives `mx check` a CE6686 per construct, and the identical script with the empty outcomes (`wf-enum-decision-empty-outcome.mdl`, the positive control `make check-mdl` requires to pass) gives 0. +- **`mxcli check` did not check a widget ACTION's arguments, so CE1571 was left to mxbuild** (mendixlabs/mxcli#1082) — a `container` in a DataGrid 2 control bar with `Action: nanoflow M.ACT_UnLink`, against a nanoflow declaring a required entity parameter, passed `mxcli check --references` with *"Check passed!"* and then failed the build with **CE1571** *"No argument has been selected for parameter 'LogisticWhitelist'"* at that container. The rule that exists for exactly this (`validateDataSourceArguments`) walked `w.GetDataSource()` only; the identical fault on an action slot was silent. The control is the same microflow and the same missing argument written twice in one run — as a data source it was reported with the fix spelled out, as an action `check` passed and mxbuild raised CE1571. + + The report's stated root cause — *"the CONTAINER widget's `Action:` clause does not support explicit parameter binding"* — does not hold, and measurement is what showed it: `actionExprV3` has always carried `NANOFLOW qualifiedName microflowArgsV3?`, and `buildContainerV3` hands the action to the same `buildClientActionV3` an `actionbutton` uses. Three pages, one script, one `mx check` on 11.12.0: the bare action is CE1571, `Action: nanoflow M.ACT_UnLink ($LogisticWhitelist = $dgMaterials)` on the **container** is 0 errors and round-trips through `describe page`, and the reporter's actionbutton workaround is 0 errors too — dropping the bare page alone takes the project 1 → 0. So the feature was there; what was missing is anything telling the author so, which is why the two documentation halves are part of the fix rather than a footnote. + + The rule now sweeps a widget's `Properties` for every `*ast.ActionV3` rather than a list of known keys — `Action:`/`OnClick:` share one key, `OnChange:` has its own, and a pluggable widget's named slot (`createFileAction:`) uses its own — so no slot is covered by omission, which is the shape of the bug being fixed. Both halves of the older rule apply: a parameter nothing fills, and an argument naming no parameter. + + **A control bar is not row-scoped, and that turned out to belong in the shared context walk.** Six containers carrying one identical fault, one `mx check`: nested in a dataview of the parameter's type → no error; of a different type → CE1571; page parameter of the exact type → CE1571; inside a **column** → no error; no context at all → CE1571; inside the **control bar** → CE1571. The walk let a control bar inherit its grid's row context, so the reported shape would have stayed silent even with actions covered. Fixing it in the walk closed a latent false negative on the data-source half too — measured on the same project, a dataview whose microflow source needs an argument is CE1571 in the control bar and clean in a column. It drops only the data widget's **own** object: an outer context still reaches the control bar (`dataview > datagrid > controlbar > action`, no argument → no error), so the control bar is walked with its parent's incoming context rather than an empty one. + + `mxcli check -p` now reports the reporter's original script, naming the remedy it went looking for: *"pass the selection of `dgMaterials` (`$dgMaterials`, once the widget has `Selection:` set), or move the widget into a column, which is row-scoped"*. Against the measured project the rule's verdicts match mxbuild's five errors widget for widget. Both guards have a control: stubbing the action walk fails 7 of the new tests, and restoring the control bar's inherited context fails the two control-bar tests with `got []`. + +- **`mxcli syntax page.action` taught the form that fails** (mendixlabs/mxcli#1082) — it listed `Action: NANOFLOW Module.NF` with no argument variant, two lines under `Action: MICROFLOW Module.MF(Param: $val)` and `Action: SHOW_PAGE Module.Page(Param: $currentObject)`, which is how an author concludes that a nanoflow action takes no arguments. The topic now carries `Action: NANOFLOW Module.NF(Param: $val)`, says that a flow action is a **call** needing an argument per parameter (CE1571) whatever widget it is on — a clickable `CONTAINER` as much as an `ACTIONBUTTON` — and its example is a control-bar container passing the grid's selection. `MDL-BUTTON01`'s suggestion advised *"move the button into a grid column, or pass a page parameter"* and never mentioned the selection, the one remedy that keeps the widget where it is; it now names it, carrying the data widget's own name through the walk because that name **is** the variable. + - **`CREATE WORKFLOW` did not auto-wire a `call microflow` activity nested in a decision's enum branch** (ako/mxcli#417) — `mxcli check` passed, `exec` reported success, and mxbuild then gave **CE6685** *"The parameters of the selected microflow have changed"* and **CE6686** *"The current outcomes of the call microflow activity do not match the configured microflow"*, once each per nested activity. `describe workflow` showed them written with no `with (...)` and no `outcomes` clause, while the control — the identical statement in the workflow's MAIN flow — was auto-wired to `with (Ctx = '$WorkflowContext') outcomes DEFAULT -> { }` and checked at 0 errors. That contrast is what made the gap invisible: the feature demonstrably worked, one nesting level up. `autoBindActivitiesInFlow` enumerated the flows to recurse into with a per-outcome type switch that handled `BooleanConditionOutcome` and `VoidConditionOutcome` only. An `EnumerationValueConditionOutcome`'s flow was never entered, and no boundary-event body was entered at all — so a decision's `''` branch was wired and its `'Module.Enum.Value'` branches were not. `deduplicateActivityNamesInFlow` walks the same tree and had the boundary-event half of the same gap (CE0495, undetected name collisions inside a boundary event). diff --git a/cmd/mxcli/brain/entry.go b/cmd/mxcli/brain/entry.go index feff2de6e0..47a484c05a 100644 --- a/cmd/mxcli/brain/entry.go +++ b/cmd/mxcli/brain/entry.go @@ -126,6 +126,9 @@ func NewRequirement(text string, anchors []string, slice string, now time.Time) if !sliceName.MatchString(slice) { return Entry{}, fmt.Errorf("slice %q: use letters, digits, '-' and '_' (a leading number orders it: 01-accounts)", slice) } + if err := requirementAnchorsArePlannable(anchors); err != nil { + return Entry{}, err + } e.Kind, e.Slice = KindRequirement, slice // The id folds in the slice, so the same sentence can legitimately appear // as a requirement of two slices without the second being refused as a @@ -134,6 +137,40 @@ func NewRequirement(text string, anchors []string, slice string, now time.Time) return e, nil } +// requirementAnchorsArePlannable refuses a requirement anchored at a bare +// MODULE. +// +// A requirement's anchors point FORWARD: not resolving means "not built yet", +// which is what lets `brain plan` derive progress from the model instead of from +// a status column somebody has to remember to update. A module anchor breaks +// that, because a module resolves the instant it exists — long before any of the +// work inside it. Measured on ako/ChipCoV1: two requirements anchored at +// `@Maintenance` (the brand theme, and the tablet/phone profiles) both reported +// BUILT after slice 01 with none of their work done. +// +// It is refused rather than warned about because the failure is silent and +// flattering: the plan reports progress that has not happened, and nothing else +// in the system disagrees. A decision anchored at a module is still perfectly +// fine — its anchor points BACKWARD, and "this module exists" is exactly the +// fact a cross-cutting decision about it wants to assert. +func requirementAnchorsArePlannable(anchors []string) error { + parsed, err := ParseAnchors(anchors) + if err != nil { + return err + } + for i, a := range parsed { + if a.Element != "" { + continue + } + return fmt.Errorf( + "requirement anchor %s names a module, which resolves as soon as the module exists — "+ + "so the requirement would report as built before any of its work is done. "+ + "Anchor at a document the slice actually creates (@%s.SomePage, @%s.ACT_Something) instead", + anchors[i], a.Module, a.Module) + } + return nil +} + // sliceName is deliberately permissive about ordering: a slice is sorted by // name, so a numeric prefix is how a roadmap gets its order, and that is the // user's choice rather than a field mxcli maintains. diff --git a/cmd/mxcli/brain/requirement_anchor_test.go b/cmd/mxcli/brain/requirement_anchor_test.go new file mode 100644 index 0000000000..25d9410847 --- /dev/null +++ b/cmd/mxcli/brain/requirement_anchor_test.go @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: Apache-2.0 + +package brain + +import ( + "strings" + "testing" + "time" +) + +// A requirement's anchors point FORWARD — not resolving means "not built yet", +// which is what makes `brain plan` a progress report derived from the model +// rather than a status column somebody has to remember to update. A module +// anchor breaks that: a module resolves the instant it exists, so the +// requirement reports BUILT before any of its work is done. +// +// Measured on ako/ChipCoV1: two requirements anchored at @Maintenance (the +// theme, and the device profiles) both read as built after slice 01 with none +// of their work started. +func TestRequirementRefusesAModuleAnchor(t *testing.T) { + _, err := NewRequirement("Technicians can close a work order", + []string{"@Maintenance"}, "05-workflow", time.Now()) + if err == nil { + t.Fatal("a requirement anchored at a bare module must be refused") + } + // The message has to say what to do instead, or the author just deletes the + // anchor — which loses the progress signal entirely rather than fixing it. + for _, want := range []string{"@Maintenance", "resolves as soon as the module exists", "Anchor at a document"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("message missing %q: %v", want, err) + } + } +} + +// The control, in both directions: a document anchor is accepted on a +// requirement, and a module anchor stays legal on a DECISION and on a QUESTION, +// whose anchors point backward — "this module exists" is exactly what a +// cross-cutting decision about it asserts. Without this the fix could be +// "refuse every module anchor" and the test above would still pass. +func TestModuleAnchorStaysLegalOnADecisionAndAQuestion(t *testing.T) { + now := time.Now() + if _, err := NewRequirement("Coordinator can assign work", + []string{"@Maintenance.ACT_AssignWork"}, "05-workflow", now); err != nil { + t.Errorf("a document anchor must be accepted on a requirement: %v", err) + } + if _, err := NewEntry("Maintenance owns all scheduling", []string{"@Maintenance"}, now); err != nil { + t.Errorf("a module anchor must stay legal on a decision: %v", err) + } + if _, err := NewQuestion("Should Maintenance own scheduling?", []string{"@Maintenance"}, "", now); err != nil { + t.Errorf("a module anchor must stay legal on a question: %v", err) + } +} + +// A requirement with a mix is refused on the module one, and must name THAT +// anchor — blaming the first in the list sends the author to edit a good one. +func TestRequirementNamesTheOffendingAnchor(t *testing.T) { + _, err := NewRequirement("Roles are scoped", + []string{"@Maintenance.Home_Web", "@Sales"}, "01-accounts", time.Now()) + if err == nil { + t.Fatal("expected refusal") + } + if !strings.Contains(err.Error(), "@Sales") { + t.Errorf("must name the module anchor, got: %v", err) + } + if strings.Contains(err.Error(), "Home_Web") { + t.Errorf("must not blame the document anchor, got: %v", err) + } +} diff --git a/cmd/mxcli/cmd_brain.go b/cmd/mxcli/cmd_brain.go index 6177fce096..47af863c3d 100644 --- a/cmd/mxcli/cmd_brain.go +++ b/cmd/mxcli/cmd_brain.go @@ -660,11 +660,41 @@ func (r *catalogResolver) Resolve(a brain.Anchor) (brain.Resolution, error) { if a.Element == "" { return brain.Resolution{State: brain.NotFound}, nil // a module is always indexed } - unit, err := r.be.FindDocumentUnit(a.Module, a.Element) - if err != nil || unit == nil { - return brain.Resolution{State: brain.NotFound}, nil + if unit, err := r.be.FindDocumentUnit(a.Module, a.Element); err == nil && unit != nil { + return brain.Resolution{State: brain.NotIndexable, Module: a.Module, Kind: unit.Kind}, nil + } + // A module ROLE is neither in the objects view nor a document, so both + // lookups above miss it and the anchor reads as NotFound — a failure state. + // For a requirement, whose anchors point forward, that means "not built + // yet" FOREVER: the roles exist, the work is done, and `brain plan` still + // reports it planned (ako/ChipCoV1). Roles are a natural thing to anchor a + // security requirement at, so the resolver has to be able to see them. + if ok, err := r.moduleRoleExists(a.Module, a.Element); err == nil && ok { + return brain.Resolution{State: brain.Resolved, Module: a.Module, Kind: "module role"}, nil + } + return brain.Resolution{State: brain.NotFound}, nil +} + +// moduleRoleExists reports whether the module declares a role of that name. +// +// The comparison is case-insensitive because Mendix treats role names that way +// and an anchor is written by hand; a case-only mismatch reporting "not found" +// would be the same false staleness this whole file exists to avoid. +func (r *catalogResolver) moduleRoleExists(moduleName, roleName string) (bool, error) { + mod, err := r.be.GetModuleByName(moduleName) + if err != nil || mod == nil { + return false, nil // an unknown module is NotFound, not an error + } + sec, err := r.be.GetModuleSecurity(mod.ID) + if err != nil || sec == nil { + return false, err + } + for _, role := range sec.ModuleRoles { + if role != nil && strings.EqualFold(role.Name, roleName) { + return true, nil + } } - return brain.Resolution{State: brain.NotIndexable, Module: a.Module, Kind: unit.Kind}, nil + return false, nil } func (r *catalogResolver) query(sql string) ([][]any, error) { diff --git a/cmd/mxcli/cmd_marketplace_install.go b/cmd/mxcli/cmd_marketplace_install.go index 34e80ad990..406d2c2521 100644 --- a/cmd/mxcli/cmd_marketplace_install.go +++ b/cmd/mxcli/cmd_marketplace_install.go @@ -193,6 +193,7 @@ func installModule(ctx context.Context, client *marketplace.Client, v *marketpla fmt.Fprintf(out, " %d units copied, %d bundled file(s) installed.\n", res.UnitsCopied, len(res.FilesInstalled)) reportSkippedFiles(out, res.FilesSkipped) + reportSecurityReconcile(out, res) // A headless install leaves the model needing two repairs only Mendix's own // tools can make (CE0463, CE6087). 'mxcli fix' runs them without the v2 -> // v1 conversion the bare mx commands perform. diff --git a/cmd/mxcli/cmd_marketplace_update.go b/cmd/mxcli/cmd_marketplace_update.go index 193d934e2a..c96365cf5c 100644 --- a/cmd/mxcli/cmd_marketplace_update.go +++ b/cmd/mxcli/cmd_marketplace_update.go @@ -238,6 +238,29 @@ func reportSavedEdits(out io.Writer, dir string, written, unsaved []string) { } } +// reportSecurityReconcile says what the transplant's access rules needed. +// +// Both halves are worth printing. A count means the package shipped rules that +// did not cover every member of their entities — the model Mendix rejects with +// CE0066 — and that mxcli has just repaired it, which is a change to the module +// the operator will see in `diff-local` and should not have to explain to +// themselves. A failure means CE0066 is still there, so it names the one command +// that clears it rather than leaving the error to be discovered by `mx check` +// (mendixlabs/mxcli#1085). +func reportSecurityReconcile(out io.Writer, r *marketplace.UpdateResult) { + if r.SecurityErr != nil { + fmt.Fprintf(out, "\n Entity access rules could not be reconciled: %v\n", r.SecurityErr) + fmt.Fprintln(out, " Mendix reports this as CE0066 \"Entity access is out of date\". Repair it with:") + fmt.Fprintf(out, " mxcli -p -c \"update security %s\"\n", r.Module) + return + } + if r.RulesReconciled > 0 { + fmt.Fprintf(out, " %d entity access rule(s) reconciled — the package's rules did not cover every\n", + r.RulesReconciled) + fmt.Fprintln(out, " member of their entities, which Mendix reports as CE0066.") + } +} + func reportUpdate(out io.Writer, r *marketplace.UpdateResult) { fmt.Fprintf(out, "\n%s updated %s → %s\n", r.Module, r.FromVersion, r.ToVersion) fmt.Fprintf(out, " %d units copied, %d element identities preserved, %d role grant(s) restored.\n", @@ -261,6 +284,7 @@ func reportUpdate(out io.Writer, r *marketplace.UpdateResult) { fmt.Fprintf(out, " %s\n", g) } } + reportSecurityReconcile(out, r) // A newer module's pages reference widget definitions the project has not // resynced, so `mx check` reports CE0463 until it is told to. Measured on // Administration 4.3.2 → 4.5.0: 11 CE0463 errors, and 0 after the resync. diff --git a/cmd/mxcli/docker/runlocal.go b/cmd/mxcli/docker/runlocal.go index 73defaed7a..e63726df43 100644 --- a/cmd/mxcli/docker/runlocal.go +++ b/cmd/mxcli/docker/runlocal.go @@ -656,6 +656,9 @@ func RunLocal(opts LocalRunOptions) error { var watcher *WebClientWatcher if opts.Watch { fmt.Fprintln(w, "Starting incremental web client bundler...") + // A nil watcher is not a failure: on Mendix 11.14+ mxbuild's serve build + // writes web/dist itself, so there is no bundler to keep hot. Every + // watcher method is nil-safe, so the watch loop needs no branch. watcher, err = StartWebClientWatch(WebClientOptions{DeployDir: opts.DeployDir, MxBuildPath: mxbuildPath, Stdout: w}) if err != nil { return fmt.Errorf("starting web client bundler: %w", err) @@ -1265,6 +1268,11 @@ func watchAndApply(opts LocalRunOptions, serve *ServeServer, rt *LocalRuntime, w if raw := strings.TrimSpace(string(build.Raw)); raw != "" && raw != build.Message { fmt.Fprintf(opts.Stderr, " %s\n", raw) } + // One failure shape is not the user's model: on Mendix 11.14+ the + // first build in a serve process does not leave the deployment in a + // state its own incremental build can continue from, so every rebuild + // fails on paths inside deployment/ and reads as a corrupt deployment. + fmt.Fprint(opts.Stderr, legacyClientBuildHint(opts.DeployDir, build.Message, string(build.Raw))) continue } // If the serve build touched web/ source, wait (briefly) for the diff --git a/cmd/mxcli/docker/webclient_1114_test.go b/cmd/mxcli/docker/webclient_1114_test.go index ac58c86c42..b54abae347 100644 --- a/cmd/mxcli/docker/webclient_1114_test.go +++ b/cmd/mxcli/docker/webclient_1114_test.go @@ -20,6 +20,7 @@ import ( "path/filepath" "strings" "testing" + "time" ) // deployWith builds a deployment dir with the given web/ contents. An empty @@ -96,3 +97,83 @@ func TestBuildWebClientStillRunsRollupWhenConfigPresent(t *testing.T) { t.Errorf("must not report a missing config when one is present: %v", err) } } + +// The same gate exists a second time, in StartWebClientWatch — and #146 fixed +// only BuildWebClient. So `run --local` started working on 11.14 while +// `run --local --watch` kept dying at the same file, one call earlier: +// +// starting web client bundler: no rollup.config.mjs in …/deployment/web +// (run a serve Deploy build first) + +func TestStartWebClientWatchSkipsWhenMxbuildAlreadyBundled(t *testing.T) { + // The 11.14 shape. There is no bundler to keep hot, because mxbuild's own + // serve build writes web/dist — so this is a nil watcher, not an error. + var out bytes.Buffer + wc, err := StartWebClientWatch(WebClientOptions{ + DeployDir: deployWith(t, "", "console.log(1)"), + MxBuildPath: "/nonexistent/modeler/mxbuild", // control: must never be reached + Stdout: &out, + }) + if err != nil { + t.Fatalf("11.14 shape must not fail --watch, got: %v", err) + } + if wc != nil { + t.Fatalf("expected a nil watcher when mxbuild already bundled, got %v", wc) + } + if !strings.Contains(out.String(), "bundled by mxbuild") { + t.Errorf("expected a line explaining why no bundler runs, got %q", out.String()) + } +} + +func TestNilWebClientWatcherIsSafeToUse(t *testing.T) { + // The nil watcher reaches the watch loop, which calls these three without a + // branch. If any of them panicked, the 11.14 fix above would trade a clean + // refusal at startup for a crash on the first file change. + var wc *WebClientWatcher + if got := wc.Generation(); got != 0 { + t.Errorf("Generation() on nil = %d, want 0", got) + } + rebuilt, err := wc.WaitForRebuild(0, time.Millisecond, time.Millisecond) + if rebuilt || err != nil { + t.Errorf("WaitForRebuild on nil = (%v, %v), want (false, nil)", rebuilt, err) + } + if got := wc.Log(); got != "" { + t.Errorf("Log() on nil = %q, want empty", got) + } + if err := wc.Stop(); err != nil { + t.Errorf("Stop() on nil = %v, want nil", err) + } +} + +func TestStartWebClientWatchStillRefusesWhenNothingToServe(t *testing.T) { + // Neither a config nor a bundle: the build produced no client at all, and + // starting a watch over nothing would serve a blank page. + _, err := StartWebClientWatch(WebClientOptions{ + DeployDir: deployWith(t, "", ""), + MxBuildPath: "/nonexistent/modeler/mxbuild", + }) + if err == nil { + t.Fatal("expected an error when there is neither a bundle nor a config") + } + for _, want := range []string{"11.14", "rollup config", "deployment/"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error message missing %q: %v", want, err) + } + } +} + +func TestStartWebClientWatchStillLaunchesWhenConfigPresent(t *testing.T) { + // The 11.13 shape: a config and no bundle. The bundler must still be + // launched — proven by it failing on the bogus tooling path rather than + // skipping. Without this the fix above could be "never watch anything". + _, err := StartWebClientWatch(WebClientOptions{ + DeployDir: deployWith(t, "export default {}", ""), + MxBuildPath: "/nonexistent/modeler/mxbuild", + }) + if err == nil { + t.Fatal("expected the bundler launch to run (and fail on the tooling path)") + } + if strings.Contains(err.Error(), "no rollup.config.mjs") { + t.Errorf("must not report a missing config when one is present: %v", err) + } +} diff --git a/cmd/mxcli/docker/webclient_legacy_paths.go b/cmd/mxcli/docker/webclient_legacy_paths.go new file mode 100644 index 0000000000..fba0fb93c6 --- /dev/null +++ b/cmd/mxcli/docker/webclient_legacy_paths.go @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: Apache-2.0 + +package docker + +import ( + "fmt" + "os" + "path/filepath" + "strings" +) + +// webclient_legacy_paths.go recognises one mxbuild failure that only ever +// appears under --watch, and only on Mendix 11.14+. +// +// On 11.14 the FIRST build in an `mxbuild --serve` process succeeds and every +// SUBSEQUENT build in that process fails, because the first one does not leave +// the deployment in a state its own incremental build can continue from. Two +// artifacts are missing, and which one the build dies on depends on how far it +// gets before it needs them: +// +// the bundler's config file web/rollup.config.mjs or web/rspack.config.mjs +// the per-document client web/pages/, web/layouts/ +// +// Measured against mxbuild 11.14.0 driven directly over its HTTP API — no +// mxcli in the picture — on a blank app, POSTing the SAME /build request twice +// with the model untouched between them: +// +// build 1 Success +// build 2 Failure — ERR_MODULE_NOT_FOUND for web/rollup.config.mjs, +// imported from mxbuild's own tools/node/rollup-runner.mjs +// +// It is not the app's choice of bundler. With App Settings > Runtime > App +// bundler flipped to Rspack the shape is identical, naming the other file: +// "Failed to load Rspack configuration file … web/rspack.config.mjs". So +// switching bundlers is not a workaround. +// +// Nor is restoring the deleted config, which was worth ruling out because the +// config IS recoverable: it appears on disk for ~1.5s mid-build and carries +// nothing model-specific (no page list, no widget list, no hashes; its input is +// just index.js, with page discovery delegated to a rollup plugin at build +// time), so it could be captured and put back the way `mxcli fix widgets` +// harvests mxbuild's own output. Measured, that rescues only the case nobody +// needs: +// +// build model state result +// 1 - cold Success +// 2,3 unchanged config restored Success +// 2' CHANGED config restored Failure - missing web/pages/*.js +// 3' CHANGED config restored + dirs Failure - widget export +// +// So there are two regressions, not one. The config deletion is the visible, +// recoverable half; underneath it the per-document client export expects +// deployment state 11.14's cold build no longer produces, and supplying the +// missing directories only moves the failure to exporting two pluggable widgets +// that ship with a blank app. Across 2' and 3', web/dist/index.js never moved +// off its cold-build timestamp. Nothing outside the process fixes that half, +// which is why this file reports rather than repairs. +// +// The controls that place this in mxbuild rather than here: a one-shot +// `mxbuild --target=deploy` run TWICE into the same deployment directory +// succeeds both times (so the 11.14 deployment shape is not the trigger — a +// fresh process is happy with the directory a serve process chokes on), and +// mxcli's /build request carries exactly the four fields mxbuild advertises in +// its own error response. +// +// What mxcli can do is not let the failure read as the user's model being +// broken. The message names absolute paths inside deployment/ and nothing +// about why they are missing, so the natural response is `rm -rf deployment/` +// — which costs a cold build and changes nothing, because the next second +// build fails the same way. + +// legacyClientDirs are the per-document client output directories a pre-11.14 +// serve Deploy build creates and 11.14+ does not. +var legacyClientDirs = []string{"pages", "layouts"} + +// bundlerConfigs are the bundler config files 11.14's first serve build does +// not leave behind. One per bundler, because the app can be set to either and +// both fail the same way. +var bundlerConfigs = []string{"rollup.config.mjs", "rspack.config.mjs"} + +// legacyClientBuildHint explains an incremental build that failed on something +// this Mendix version's first build did not leave behind. It returns "" for +// every other failure. +// +// message is the serve response's message; raw is the full body, which is +// where the bundler-config failure puts its detail (the message there is only +// "Compilation of the app bundle failed"). The check is on the failure's own +// shape rather than on the Mendix version, so a future mxbuild that fixes this +// goes quiet on its own and no unrelated build failure is explained away. +func legacyClientBuildHint(deployDir, message, raw string) string { + if missing := missingClientDirs(deployDir, message); len(missing) > 0 { + return hintText("this build wants " + strings.Join(missing, " and ") + + ", which this Mendix version's first build does not create") + } + if name := missingBundlerConfig(deployDir, message+raw); name != "" { + return hintText("mxbuild's own bundler cannot find web/" + name + + ", which its own first build does not leave behind") + } + return "" +} + +// missingClientDirs returns the per-document client directories the failure +// names and the deployment genuinely lacks. +func missingClientDirs(deployDir, message string) []string { + if !strings.Contains(message, "Could not find a part of the path") { + return nil + } + var named []string + for _, dir := range legacyClientDirs { + if !strings.Contains(message, filepath.Join("web", dir)+string(filepath.Separator)) { + continue + } + if _, err := os.Stat(filepath.Join(deployDir, "web", dir)); os.IsNotExist(err) { + named = append(named, "web/"+dir+"/") + } + } + return named +} + +// missingBundlerConfig returns the bundler config file the failure names and +// the deployment genuinely lacks, or "". +func missingBundlerConfig(deployDir, body string) string { + if !strings.Contains(body, "ERR_MODULE_NOT_FOUND") { + return "" + } + for _, name := range bundlerConfigs { + if !strings.Contains(body, name) { + continue + } + if _, err := os.Stat(filepath.Join(deployDir, "web", name)); os.IsNotExist(err) { + return name + } + } + return "" +} + +func hintText(what string) string { + return fmt.Sprintf( + " This is not a problem with your model, and deployment/ is not corrupt:\n"+ + " %s.\n"+ + " On Mendix 11.14 the first build in a serve process succeeds and every later\n"+ + " one fails this way — measured with the model untouched between two identical\n"+ + " builds, and with either app bundler, so switching bundlers does not help.\n"+ + " Deleting deployment/ does not either: the next second build fails the same way.\n"+ + " Until mxbuild closes this, drop --watch and restart per change:\n"+ + " mxcli run --local --screenshot\n", + what) +} diff --git a/cmd/mxcli/docker/webclient_legacy_paths_test.go b/cmd/mxcli/docker/webclient_legacy_paths_test.go new file mode 100644 index 0000000000..c65d0f7acd --- /dev/null +++ b/cmd/mxcli/docker/webclient_legacy_paths_test.go @@ -0,0 +1,123 @@ +// SPDX-License-Identifier: Apache-2.0 + +package docker + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +// Both messages below are real, captured from mxbuild 11.14.0 driven over its +// own HTTP API with no mxcli involved. Which one a build produces depends only +// on how far it gets before it needs something the first build did not leave: +// with the model changed it dies exporting the per-document client, with the +// model untouched it gets to the bundler. +const ( + // Second serve build after an ALTER PAGE edit. + real1114MissingClientDirs = "One or more errors occurred. " + + "(Could not find a part of the path '/w/W1114/deployment/web/layouts/Atlas_Core.Atlas_TopBar.js'.) " + + "(Could not find a part of the path '/w/W1114/deployment/web/pages/MyFirstModule.Home_Web.js'.) " + + "(Could not find a part of the path '/w/W1114/deployment/web/layouts/Atlas_Core.Atlas_Default.js'.)" + + // Second serve build, byte-identical request, model untouched. App bundler + // = Rollup (the 11.14 default; EnableRspackBundler false in the model). + real1114MissingRollupConfig = `{"problems":{"errors":[{"message":"Compilation of the app bundle failed.",` + + `"details":"Error [ERR_MODULE_NOT_FOUND]: Cannot find module ` + + `'/w/W1114/deployment/web/rollup.config.mjs' imported from ` + + `/home/vscode/.mxcli/mxbuild/11.14.0/modeler/tools/node/rollup-runner.mjs"}],"problems":[]},` + + `"status":"Failure","message":"Compilation of the app bundle failed."}` + + // The same, with App Settings > Runtime > App bundler = Rspack. This is the + // control that kills "just switch bundlers" as a workaround. + real1114MissingRspackConfig = `{"problems":{"errors":[{"message":"Compilation of the app bundle failed.",` + + `"details":"Failed to load Rspack configuration file.\n{\n \"code\": \"ERR_MODULE_NOT_FOUND\",\n` + + ` \"url\": \"file:///w/W1114/deployment/web/rspack.config.mjs\"\n}"}],"problems":[]},` + + `"status":"Failure","message":"Compilation of the app bundle failed."}` + + bundleFailedMessage = "Compilation of the app bundle failed." +) + +// deploy1114 is the shape a 11.14 first build leaves: web/dist and nothing +// else — no bundler config, no per-document client directories. +func deploy1114(t *testing.T) string { + t.Helper() + dir := t.TempDir() + if err := os.MkdirAll(filepath.Join(dir, "web", "dist"), 0o755); err != nil { + t.Fatal(err) + } + return dir +} + +func TestLegacyClientBuildHintExplainsBothRealFailures(t *testing.T) { + for name, tc := range map[string]struct{ message, raw, want string }{ + "missing client dirs": {real1114MissingClientDirs, "", "web/pages/"}, + "missing rollup config": {bundleFailedMessage, real1114MissingRollupConfig, "web/rollup.config.mjs"}, + "missing rspack config": {bundleFailedMessage, real1114MissingRspackConfig, "web/rspack.config.mjs"}, + } { + hint := legacyClientBuildHint(deploy1114(t), tc.message, tc.raw) + if hint == "" { + t.Errorf("%s: must be explained", name) + continue + } + // The three things a user acts on: it is not their model, what is + // actually missing, and a command that works today. + for _, want := range []string{"not a problem with your model", tc.want, "run --local --screenshot"} { + if !strings.Contains(hint, want) { + t.Errorf("%s: hint missing %q:\n%s", name, want, hint) + } + } + // Both obvious wrong moves must be closed off — measured, neither helps. + for _, want := range []string{"switching bundlers does not help", "Deleting deployment/ does not either"} { + if !strings.Contains(hint, want) { + t.Errorf("%s: hint missing %q:\n%s", name, want, hint) + } + } + } +} + +func TestLegacyClientBuildHintStaysSilentOnOtherFailures(t *testing.T) { + dir := deploy1114(t) + // Every one of these is a real failure with a different cause. A hint on + // any of them sends the user chasing a Mendix-version story that has + // nothing to do with their build. + for name, tc := range map[string]struct{ message, raw string }{ + "model errors": {"1 error occurred. CE0142: The microflow must return a Boolean.", ""}, + "scss": {"Expected expression. _custom.scss 180:35", ""}, + "widget export": {"Deployment failed during export of pluggable widgets.", ""}, + "other path": {"Could not find a part of the path '/w/W1114/userlib/missing.jar'.", ""}, + "other module": {bundleFailedMessage, `{"details":"Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/w/node_modules/left-pad'"}`}, + "empty": {"", ""}, + } { + if hint := legacyClientBuildHint(dir, tc.message, tc.raw); hint != "" { + t.Errorf("%s must not be explained as the 11.14 serve shape:\n%s", name, hint) + } + } +} + +func TestLegacyClientBuildHintStaysSilentWhenTheArtifactsExist(t *testing.T) { + // The pre-11.14 shape: the first build made these, so a failure naming one + // is a real failure and not this. Without the stat the hint would fire on + // any 11.13 build that lost a page file or a config, a different bug. + dir := deploy1114(t) + for _, d := range legacyClientDirs { + if err := os.MkdirAll(filepath.Join(dir, "web", d), 0o755); err != nil { + t.Fatal(err) + } + } + for _, f := range bundlerConfigs { + if err := os.WriteFile(filepath.Join(dir, "web", f), []byte("export default {}"), 0o644); err != nil { + t.Fatal(err) + } + } + for name, tc := range map[string]struct{ message, raw string }{ + "client dirs": {real1114MissingClientDirs, ""}, + "rollup config": {bundleFailedMessage, real1114MissingRollupConfig}, + "rspack config": {bundleFailedMessage, real1114MissingRspackConfig}, + } { + if hint := legacyClientBuildHint(dir, tc.message, tc.raw); hint != "" { + t.Errorf("%s: must not fire when the artifact exists:\n%s", name, hint) + } + } +} diff --git a/cmd/mxcli/docker/webclient_watch.go b/cmd/mxcli/docker/webclient_watch.go index 87aa089991..daa7fdc9a1 100644 --- a/cmd/mxcli/docker/webclient_watch.go +++ b/cmd/mxcli/docker/webclient_watch.go @@ -118,6 +118,9 @@ func (wc *WebClientWatcher) snapshot() (gen int, building, exited bool, lastErr // Generation returns the number of successful bundles so far. Capture it before // triggering a source change, then pass it to WaitForBundle. func (wc *WebClientWatcher) Generation() int { + if wc == nil { + return 0 + } wc.mu.Lock() defer wc.mu.Unlock() return wc.gen @@ -125,10 +128,40 @@ func (wc *WebClientWatcher) Generation() int { // StartWebClientWatch launches the incremental bundler and blocks until the first // bundle completes (so web/dist exists before the app boots). +// +// It returns (nil, nil) on Mendix 11.14+, where mxbuild's own serve build writes +// web/dist and emits no rollup config — there is no bundler to run. Every method +// on the returned watcher is nil-safe, so callers need no branch for it. func StartWebClientWatch(opts WebClientOptions) (*WebClientWatcher, error) { + w := opts.Stdout + if w == nil { + w = io.Discard + } webDir := filepath.Join(opts.DeployDir, "web") if fi, err := os.Stat(filepath.Join(webDir, "rollup.config.mjs")); err != nil || fi.IsDir() { - return nil, fmt.Errorf("no rollup.config.mjs in %s (run a serve Deploy build first)", webDir) + // The Mendix 11.14 shape, and the second copy of the gate that made + // mxcli unable to start any 11.14 app (ako/mxcli-ledger #146). That fix + // landed on BuildWebClient only, so `run --local` started working and + // `run --local --watch` kept failing on the absence of a file whose + // purpose 11.14 had served: + // + // 11.13.0 rollup.config.mjs PRESENT dist/index.js ABSENT + // 11.14.0 rollup.config.mjs ABSENT dist/index.js PRESENT + // + // When mxbuild bundles the client itself there is no incremental + // bundler to keep hot and nothing for it to do. A nil watcher says + // exactly that, and the loop treats it as "the serve build produces + // web/dist" — with ensureClientServed still guarding the result. + if WebClientBundled(opts.DeployDir) { + fmt.Fprintln(w, " Web client bundled by mxbuild; no incremental bundler needed") + return nil, nil + } + return nil, fmt.Errorf("no rollup.config.mjs and no bundle at %s\n"+ + " Mendix 11.13 and earlier emit a rollup config for mxcli to run; 11.14+ writes\n"+ + " the bundle itself. Neither is present, so the build did not produce a client:\n"+ + " run a serve Deploy build first (or delete deployment/ if it was built by an\n"+ + " older Mendix version).", + webClientBundlePath(opts.DeployDir)) } nodeBin, runner, err := resolveNodeTooling(opts.MxBuildPath) if err != nil { @@ -234,6 +267,9 @@ func (wc *WebClientWatcher) waitForFirstBuild(timeout time.Duration) error { // Reliable because file-change detection is fast (CHOKIDAR_USEPOLLING ~1s), so a // rebuild that is going to happen starts well within a few-second settle window. func (wc *WebClientWatcher) WaitForRebuild(sinceGen int, settle, buildTimeout time.Duration) (bool, error) { + if wc == nil { + return false, nil // mxbuild bundles the client itself; nothing to wait for + } settleDeadline := time.Now().Add(settle) sawBuild := false for { @@ -270,11 +306,16 @@ func (wc *WebClientWatcher) WaitForRebuild(sinceGen int, settle, buildTimeout ti } // Log returns the captured watcher output. -func (wc *WebClientWatcher) Log() string { return wc.log.String() } +func (wc *WebClientWatcher) Log() string { + if wc == nil { + return "" + } + return wc.log.String() +} // Stop terminates the watcher process. func (wc *WebClientWatcher) Stop() error { - if wc.cmd == nil || wc.cmd.Process == nil { + if wc == nil || wc.cmd == nil || wc.cmd.Process == nil { return nil } _ = signalProcessGroup(wc.cmd.Process, syscall.SIGTERM) diff --git a/cmd/mxcli/init_hook.go b/cmd/mxcli/init_hook.go index dd1ea962df..3c75b84c02 100644 --- a/cmd/mxcli/init_hook.go +++ b/cmd/mxcli/init_hook.go @@ -49,6 +49,27 @@ set -e MPR='%s' TAG="${MXCLI_TAG:-nightly}" +if [ ! -x ./mxcli ]; then + # Prefer a copy that is already on this machine. Some environments ship mxcli + # pre-installed on PATH, and the bootstrap instructions have you delete the + # hardlink 'mxcli new' left in the project — after which this guard could + # never be satisfied by the PATH binary and re-downloaded ~85 MB on EVERY + # fresh session, forever. (ako/ChipCoV1) + # + # Hardlink first because that is what 'mxcli new' does and it costs nothing; + # fall back to a symlink across filesystems, then to a copy. Any of the three + # leaves ./mxcli working, which is what the rest of this script and the + # project's own CLAUDE.md assume. + onpath=$(command -v mxcli 2>/dev/null || true) + if [ -n "$onpath" ] && [ -x "$onpath" ]; then + echo "mxcli found on PATH (${onpath}) — linking it in rather than downloading." + ln -f "$onpath" ./mxcli 2>/dev/null || + ln -sf "$onpath" ./mxcli 2>/dev/null || + cp "$onpath" ./mxcli + chmod +x ./mxcli 2>/dev/null || true + fi +fi + if [ ! -x ./mxcli ]; then os=$(uname -s | tr 'A-Z' 'a-z') case "$(uname -m)" in diff --git a/cmd/mxcli/marketplace/security.go b/cmd/mxcli/marketplace/security.go new file mode 100644 index 0000000000..7f1923c74a --- /dev/null +++ b/cmd/mxcli/marketplace/security.go @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: Apache-2.0 + +package marketplace + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/backend" +) + +// ReconcileModuleSecurity brings one module's entity access rules back into sync +// with its domain model, and reports how many rules it had to change. +// +// This is the `UPDATE SECURITY ` statement's work, and Studio Pro's +// "Update security" button's, reached directly because a module that has just +// been transplanted is not the product of any MDL program. +// +// It is needed because a transplant is the one write path in mxcli that does not +// reconcile as it writes. Every other one does — the executor's finalize step +// after a program, `GRANT`, the association handlers — but `TransplantModule` +// copies raw units in verbatim and `RestoreRoleGrants` runs its statements one +// at a time, which never reaches the finalize step. So an access rule that does +// not cover every member of its entity arrives exactly as the package shipped +// it, and Mendix rejects the model with CE0066 "Entity access is out of date" +// (mendixlabs/mxcli#1085). The update itself wrote nothing wrong, which is why +// it had nothing to report and exited 0. +// +// A module whose rules are already complete is left alone: ReconcileMemberAccesses +// returns 0 and writes nothing. That matters more than it looks — reconciling +// unconditionally would rewrite a marketplace module's domain model on every +// install, `marketplace diff` would read the result back as a local edit, and a +// local edit is what makes the NEXT update refuse. +func ReconcileModuleSecurity(mprPath, moduleName string, newBackend func() backend.FullBackend) (int, error) { + if newBackend == nil { + return 0, fmt.Errorf("no backend factory") + } + b := newBackend() + if err := b.Connect(mprPath); err != nil { + return 0, fmt.Errorf("open %s: %w", mprPath, err) + } + defer b.Disconnect() + + mod, err := b.GetModuleByName(moduleName) + if err != nil { + return 0, fmt.Errorf("find module %s: %w", moduleName, err) + } + if mod == nil { + return 0, fmt.Errorf("module %s not found", moduleName) + } + + dm, err := b.GetDomainModel(mod.ID) + if err != nil || dm == nil { + // A module need not have a domain model — a theme or widget module has + // none, and there is nothing to reconcile. Not an error. + return 0, nil + } + + count, err := b.ReconcileMemberAccesses(dm.ID, moduleName) + if err != nil { + return 0, err + } + if err := b.Commit(); err != nil { + return count, fmt.Errorf("commit: %w", err) + } + return count, nil +} diff --git a/cmd/mxcli/marketplace/security_test.go b/cmd/mxcli/marketplace/security_test.go new file mode 100644 index 0000000000..1a57cce841 --- /dev/null +++ b/cmd/mxcli/marketplace/security_test.go @@ -0,0 +1,335 @@ +// SPDX-License-Identifier: Apache-2.0 + +package marketplace + +import ( + "archive/zip" + "os" + "path/filepath" + "sort" + "testing" + + modelsdk "github.com/mendixlabs/mxcli" + "github.com/mendixlabs/mxcli/mdl/backend" + mprbackend "github.com/mendixlabs/mxcli/mdl/backend/mpr" + "go.mongodb.org/mongo-driver/bson" +) + +// Both engines, because the reconcile they call is a separate implementation in +// each (mdl/backend/modelsdk and sdk/mpr) and a marketplace update runs under +// whichever --engine selects. A defect fixed in one of those parallel writers +// stays latent in the other until something switches engines — which is how +// mendixlabs/mxcli#1047 reached two engines at once. +var engines = []struct { + name string + backend func() backend.FullBackend +}{ + {"modelsdk", testBackend}, + {"legacy", func() backend.FullBackend { return mprbackend.New() }}, +} + +// A marketplace update copies the incoming module's units in verbatim, so an +// access rule that does not cover every member of its entity arrives in the +// project exactly as the package shipped it. Mendix rejects that model with +// CE0066 "Entity access is out of date. Please update security by clicking the +// 'Update security' button in the domain model editor" — and the update, having +// written nothing of its own, exits 0 and says nothing about it +// (mendixlabs/mxcli#1085). +// +// Measured before the fix on the vendored 11.6.6 app with Administration 4.3.2, +// mxbuild 11.14.0: one member entry removed from the incoming version's rules +// took `mx check` from 0 errors to +// +// [error] [CE0066] "Entity access is out of date…" at Domain model of module 'Administration' +// +// while the same update of an UNCHANGED module checked clean — so the machinery +// does not create the staleness, it transmits it. `update security` then +// repaired it (`Reconciled 3 access rule(s) in module Administration`, 1 -> 0 +// errors), which is what this test asserts the update now does for itself. +func TestPerformUpdate_ReconcilesAnIncompleteAccessRule(t *testing.T) { + for _, eng := range engines { + t.Run(eng.name, func(t *testing.T) { + target := copyFixture(t) + ref := copyFixture(t) + + // The control: the incoming version is genuinely missing a member + // entry. A fixture that failed to strip one would pass this test + // against unfixed code, which is the failure mode that makes a repro + // worthless. + removed := stripLastMemberAccess(t, ref, "Administration", "Account") + if got := memberRefs(t, ref, "Administration", "Account"); containsRef(got, removed) { + t.Fatalf("control: %s is still in the reference's rules %v — nothing was stripped", removed, got) + } + + res, err := PerformUpdate(target, ref, emptyMpk(t), "Administration", + "4.3.2", "4.3.3", "00000000-0000-0000-0000-000000000000", eng.backend) + if err != nil { + t.Fatalf("PerformUpdate: %v", err) + } + if res.RulesReconciled == 0 { + t.Errorf("RulesReconciled = 0; the update reported nothing about repairing the rules") + } + + got := memberRefs(t, target, "Administration", "Account") + if !containsRef(got, removed) { + t.Fatalf("the updated module's rules are still missing %s: %v\n"+ + "this is the CE0066 in mendixlabs/mxcli#1085", removed, got) + } + }) + } +} + +// The other half of the contract, and the reason this cannot simply reconcile +// unconditionally and call it done: a module whose rules already cover every +// member must come through an update untouched. Reconciling one that needs +// nothing would rewrite a marketplace module's domain model on every update, +// which `marketplace diff` reads back as a local edit — and a local edit is what +// makes the NEXT update refuse. +func TestPerformUpdate_LeavesCompleteRulesAlone(t *testing.T) { + target := copyFixture(t) + ref := copyFixture(t) + + before := memberRefs(t, ref, "Administration", "Account") + res, err := PerformUpdate(target, ref, emptyMpk(t), "Administration", + "4.3.2", "4.3.3", "00000000-0000-0000-0000-000000000000", testBackend) + if err != nil { + t.Fatalf("PerformUpdate: %v", err) + } + if res.RulesReconciled != 0 { + t.Errorf("RulesReconciled = %d on a module whose rules were already complete; want 0", + res.RulesReconciled) + } + if after := memberRefs(t, target, "Administration", "Account"); !sameRefs(before, after) { + t.Errorf("an already-complete module was rewritten by the update:\nbefore %v\nafter %v", before, after) + } +} + +// TestPerformInstall_ReconcilesAnIncompleteAccessRule — a first install copies +// the same units through the same transplant, so it lands the same CE0066. +func TestPerformInstall_ReconcilesAnIncompleteAccessRule(t *testing.T) { + target := copyFixture(t) + ref := copyFixture(t) + + removed := stripLastMemberAccess(t, ref, "Administration", "Account") + execMDL(t, target, "drop module Administration;") + + res, err := PerformInstall(target, ref, emptyMpk(t), "Administration", + "4.3.2", "00000000-0000-0000-0000-000000000000", testBackend) + if err != nil { + t.Fatalf("PerformInstall: %v", err) + } + if res.RulesReconciled == 0 { + t.Errorf("RulesReconciled = 0; the install reported nothing about repairing the rules") + } + if got := memberRefs(t, target, "Administration", "Account"); !containsRef(got, removed) { + t.Fatalf("the installed module's rules are still missing %s: %v", removed, got) + } +} + +// --- helpers ------------------------------------------------------------- + +// emptyMpk is a package with no payload. InstallPackageFiles is not what these +// tests are about, and a real .mpk would drag widget binaries into a temp dir +// for nothing. +func emptyMpk(t *testing.T) string { + t.Helper() + path := filepath.Join(t.TempDir(), "empty.mpk") + f, err := os.Create(path) + if err != nil { + t.Fatalf("create %s: %v", path, err) + } + zw := zip.NewWriter(f) + w, err := zw.Create("package.xml") + if err != nil { + t.Fatalf("write package.xml: %v", err) + } + if _, err := w.Write([]byte("")); err != nil { + t.Fatalf("write package.xml: %v", err) + } + if err := zw.Close(); err != nil { + t.Fatalf("close zip: %v", err) + } + if err := f.Close(); err != nil { + t.Fatalf("close %s: %v", path, err) + } + return path +} + +// stripLastMemberAccess removes the last MemberAccess from every populated +// access rule of one entity, and returns the reference it removed. This is what +// a package whose author never pressed "Update security" ships: rules that name +// some of the entity's members and not all of them. +// +// It edits the stored BSON rather than going through MDL on purpose — every +// mxcli write path reconciles as it writes, so no script can produce the state +// being reproduced here. +func stripLastMemberAccess(t *testing.T, mprPath, moduleName, entityName string) string { + t.Helper() + + reader, err := modelsdk.Open(mprPath) + if err != nil { + t.Fatalf("open %s: %v", mprPath, err) + } + units, err := reader.ListRawUnitsByType("DomainModels$DomainModel") + if err != nil { + reader.Close() + t.Fatalf("list domain models: %v", err) + } + + var unitID, removed string + var contents []byte + for _, u := range units { + var doc bson.D + if bson.Unmarshal(u.Contents, &doc) != nil { + continue + } + ent := findEntityDoc(doc, entityName) + if ent == nil { + continue + } + rules, _ := bsonLookup(ent, "AccessRules").(bson.A) + for _, r := range rules { + rule, ok := r.(bson.D) + if !ok { + continue + } + for i := range rule { + if rule[i].Key != "MemberAccesses" { + continue + } + arr, ok := rule[i].Value.(bson.A) + if !ok || len(arr) < 2 { + continue // empty rule, or the storage marker alone + } + if ref := memberRefOf(arr[len(arr)-1]); ref != "" { + removed = ref + } + rule[i].Value = arr[:len(arr)-1] + } + } + enc, merr := bson.Marshal(doc) + if merr != nil { + reader.Close() + t.Fatalf("re-encode domain model: %v", merr) + } + unitID, contents = string(u.ID), enc + break + } + reader.Close() + + if unitID == "" || removed == "" { + t.Fatalf("no populated access rule found on %s.%s to strip", moduleName, entityName) + } + writer, err := modelsdk.OpenForWriting(mprPath) + if err != nil { + t.Fatalf("open %s for writing: %v", mprPath, err) + } + defer writer.Close() + if err := writer.UpdateRawUnit(unitID, contents); err != nil { + t.Fatalf("write domain model: %v", err) + } + return removed +} + +// memberRefs returns every member reference named by the entity's access rules, +// sorted and deduplicated. +func memberRefs(t *testing.T, mprPath, moduleName, entityName string) []string { + t.Helper() + + reader, err := modelsdk.Open(mprPath) + if err != nil { + t.Fatalf("open %s: %v", mprPath, err) + } + defer reader.Close() + units, err := reader.ListRawUnitsByType("DomainModels$DomainModel") + if err != nil { + t.Fatalf("list domain models: %v", err) + } + + seen := map[string]bool{} + for _, u := range units { + var doc bson.D + if bson.Unmarshal(u.Contents, &doc) != nil { + continue + } + ent := findEntityDoc(doc, entityName) + if ent == nil { + continue + } + rules, _ := bsonLookup(ent, "AccessRules").(bson.A) + for _, r := range rules { + rule, ok := r.(bson.D) + if !ok { + continue + } + mas, _ := bsonLookup(rule, "MemberAccesses").(bson.A) + for _, ma := range mas { + if ref := memberRefOf(ma); ref != "" { + seen[ref] = true + } + } + } + } + out := make([]string, 0, len(seen)) + for ref := range seen { + out = append(out, ref) + } + sort.Strings(out) + return out +} + +func memberRefOf(v any) string { + d, ok := v.(bson.D) + if !ok { + return "" + } + if s, _ := bsonLookup(d, "Attribute").(string); s != "" { + return s + } + s, _ := bsonLookup(d, "Association").(string) + return s +} + +func findEntityDoc(dm bson.D, entityName string) bson.D { + ents, _ := bsonLookup(dm, "Entities").(bson.A) + for _, e := range ents { + ed, ok := e.(bson.D) + if !ok { + continue + } + if name, _ := bsonLookup(ed, "Name").(string); name == entityName { + return ed + } + } + return nil +} + +func bsonLookup(d bson.D, key string) any { + for _, kv := range d { + if kv.Key == key { + return kv.Value + } + } + return nil +} + +func containsRef(refs []string, want string) bool { + for _, r := range refs { + if r == want { + return true + } + } + return false +} + +func sameRefs(a, b []string) bool { + if len(a) != len(b) { + return false + } + for i := range a { + if a[i] != b[i] { + return false + } + } + return true +} diff --git a/cmd/mxcli/marketplace/update.go b/cmd/mxcli/marketplace/update.go index 68c735e33b..dae0d2fba7 100644 --- a/cmd/mxcli/marketplace/update.go +++ b/cmd/mxcli/marketplace/update.go @@ -113,7 +113,12 @@ type UpdateResult struct { IdentitiesLost []string GrantsRestored int GrantsDropped []string - FilesInstalled []string + // RulesReconciled counts the entity access rules the update had to bring + // back into sync with the domain model after copying the module in, and + // SecurityErr records why that could not be done when it failed. + RulesReconciled int + SecurityErr error + FilesInstalled []string // FilesSkipped records bundled files that were deliberately not installed: // a widget older than the copy the project already has, or the unpacked twin // of a widget the package also ships as a .mpk. Reported rather than silent — @@ -180,17 +185,25 @@ func PerformUpdate(mprPath, referenceMpr, targetMpk, moduleName, fromVersion, to return nil, fmt.Errorf("install the new version's bundled files: %w", err) } + // The module is in place; its access rules are whatever the package shipped. + // A failure here is reported rather than returned: the update has already + // replaced the module, so aborting now would leave the project mid-update + // over a repair the operator can run themselves. + reconciled, secErr := ReconcileModuleSecurity(mprPath, moduleName, newBackend) + return &UpdateResult{ - Module: moduleName, - FromVersion: fromVersion, - ToVersion: toVersion, - UnitsCopied: copied, - IdentitiesKept: applied, - IdentitiesLost: missing, - GrantsRestored: restored, - GrantsDropped: dropped, - FilesInstalled: files, - FilesSkipped: skippedFiles, + Module: moduleName, + FromVersion: fromVersion, + ToVersion: toVersion, + UnitsCopied: copied, + IdentitiesKept: applied, + IdentitiesLost: missing, + GrantsRestored: restored, + GrantsDropped: dropped, + RulesReconciled: reconciled, + SecurityErr: secErr, + FilesInstalled: files, + FilesSkipped: skippedFiles, }, nil } @@ -419,11 +432,16 @@ func PerformInstall(mprPath, referenceMpr, packageMpk, moduleName, version, vers if err != nil { return nil, fmt.Errorf("install the package's bundled files: %w", err) } + // Same transplant, same gap: the package's access rules arrive verbatim and + // nothing has checked them against the entities they govern. + reconciled, secErr := ReconcileModuleSecurity(mprPath, moduleName, newBackend) return &UpdateResult{ - Module: moduleName, - ToVersion: version, - UnitsCopied: copied, - FilesInstalled: files, - FilesSkipped: skippedFiles, + Module: moduleName, + ToVersion: version, + UnitsCopied: copied, + RulesReconciled: reconciled, + SecurityErr: secErr, + FilesInstalled: files, + FilesSkipped: skippedFiles, }, nil } diff --git a/cmd/mxcli/syntax/features_page.go b/cmd/mxcli/syntax/features_page.go index a588e89264..3b59eebacd 100644 --- a/cmd/mxcli/syntax/features_page.go +++ b/cmd/mxcli/syntax/features_page.go @@ -210,15 +210,15 @@ CREATE PAGE Sales.Detail (Title: 'Detail', Layout: Atlas_Core.Atlas_Default) { Register(SyntaxFeature{ Path: "page.action", - Summary: "Button actions: save, cancel, close, delete, show page, microflow", + Summary: "Widget actions: save, cancel, close, delete, show page, microflow, nanoflow", Keywords: []string{ "action", "save", "cancel", "close", "delete", "show page", "navigate", "microflow", "create object", "button style", "primary", "danger", "success", "icon", "linkbutton", "link button", }, - Syntax: "Action: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: OPEN_LINK 'https://example.com'\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $currentObject)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nA SHOW_PAGE argument must be the enclosing widget's context object --\neither $currentObject or the name of the variable the enclosing data\nwidget is bound to. Mendix infers it from that widget, so naming any\nother variable is refused (MDL-PAGEARG01); call a microflow instead.\n\nOPEN_LINK takes a static web address and stores it as a\nForms$StaticOrDynamicString. Mendix also supports a DYNAMIC address, read\nfrom an attribute at runtime; MDL cannot author that one, and DESCRIBE\nflags such a button rather than printing its address as a literal.\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\nIcon: 'Module.IconCollection.IconName' -- e.g. 'Atlas_Core.Atlas_Filled.pencil'\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", - Example: "ACTIONBUTTON btnSave (Caption: 'Save', Action: SAVE_CHANGES, ButtonStyle: Primary)\nACTIONBUTTON btnEdit (Caption: 'Edit',\n Action: SHOW_PAGE Module.EditPage(Item: $currentObject))\nLINKBUTTON btnDelete (Caption: 'Delete', Action: DELETE,\n Icon: 'Atlas_Core.Atlas_Filled.pencil')", + Syntax: "Action: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: NANOFLOW Module.NF(Param: $val)\nAction: OPEN_LINK 'https://example.com'\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $currentObject)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nA microflow or nanoflow action is a CALL: it needs an argument for every\nparameter the flow declares, or Mendix rejects the page with CE1571. The\nargument list is the same on every widget that takes an action -- a\nCONTAINER (which is clickable) as much as an ACTIONBUTTON. An enclosing\ndata container of the right type supplies it without an argument; a data\ngrid's CONTROL BAR does not, because it is not row-scoped -- pass the\ngrid's selection there (`$dgOrders`).\n\nA SHOW_PAGE argument must be the enclosing widget's context object --\neither $currentObject or the name of the variable the enclosing data\nwidget is bound to. Mendix infers it from that widget, so naming any\nother variable is refused (MDL-PAGEARG01); call a microflow instead.\n\nOPEN_LINK takes a static web address and stores it as a\nForms$StaticOrDynamicString. Mendix also supports a DYNAMIC address, read\nfrom an attribute at runtime; MDL cannot author that one, and DESCRIBE\nflags such a button rather than printing its address as a literal.\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\nIcon: 'Module.IconCollection.IconName' -- e.g. 'Atlas_Core.Atlas_Filled.pencil'\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", + Example: "ACTIONBUTTON btnSave (Caption: 'Save', Action: SAVE_CHANGES, ButtonStyle: Primary)\nACTIONBUTTON btnEdit (Caption: 'Edit',\n Action: SHOW_PAGE Module.EditPage(Item: $currentObject))\nLINKBUTTON btnDelete (Caption: 'Delete', Action: DELETE,\n Icon: 'Atlas_Core.Atlas_Filled.pencil')\n\n-- A clickable CONTAINER in a data grid's control bar, calling a nanoflow\n-- with the grid's selection as its argument.\nDATAGRID dgOrders (DataSource: DATABASE FROM Sales.Order, Selection: Single) {\n COLUMN colNr (Attribute: Number, Caption: 'Order #')\n CONTROLBAR cb {\n CONTAINER cShip (Class: 'command',\n Action: NANOFLOW Sales.ACT_Ship($Order = $dgOrders)) {\n ACTIONBUTTON btnShip (Caption: 'Ship')\n }\n }\n}", SeeAlso: []string{"page.widgets"}, }) diff --git a/cmd/mxcli/syntax/features_security.go b/cmd/mxcli/syntax/features_security.go index 74b0ea1a41..270c885789 100644 --- a/cmd/mxcli/syntax/features_security.go +++ b/cmd/mxcli/syntax/features_security.go @@ -65,6 +65,35 @@ func init() { SeeAlso: []string{"security.module-role", "security.microflow-access"}, }) + Register(SyntaxFeature{ + Path: "security.update-security", + Summary: "Repair entity access rules that no longer match their domain model (CE0066)", + Keywords: []string{ + "update security", "CE0066", "entity access out of date", + "reconcile", "member access", "update security button", + }, + Syntax: "UPDATE SECURITY;\n" + + "UPDATE SECURITY ;\n" + + "UPDATE SECURITY IN ;\n\n" + + "This is the headless equivalent of Studio Pro's 'Update security'\n" + + "button in the domain model editor. It adds the member entries an\n" + + "access rule is missing, removes entries for members that no longer\n" + + "exist, and reports how many rules it changed.\n\n" + + "You rarely need it for models mxcli writes — every write path\n" + + "reconciles as it writes. It is for a model that arrived from\n" + + "somewhere else: a module imported or updated outside Studio Pro,\n" + + "whose rules do not cover every member of their entities. Mendix\n" + + "rejects that with CE0066 'Entity access is out of date'.\n\n" + + "A project whose rules are already complete is not written to; the\n" + + "command says 'All entity access rules are up to date'. System is\n" + + "skipped — its access rules are the platform's.", + Example: "-- After a headless module install or update:\n" + + "UPDATE SECURITY UserCommons;\n\n" + + "-- Every module in the project:\n" + + "UPDATE SECURITY;", + SeeAlso: []string{"security.entity-access", "security.module-role"}, + }) + Register(SyntaxFeature{ Path: "security.microflow-access", Summary: "Grant or revoke execution rights on microflows", diff --git a/cmd/mxcli/syntax/features_workflow.go b/cmd/mxcli/syntax/features_workflow.go index 3cb3465de9..4fdc200b41 100644 --- a/cmd/mxcli/syntax/features_workflow.go +++ b/cmd/mxcli/syntax/features_workflow.go @@ -201,8 +201,16 @@ func init() { // ACTIVITY keyword. This entry previously showed the operand order // reversed, advertised a BEFORE that does not exist, and omitted // ACTIVITY, so none of it parsed. - Syntax: "ALTER WORKFLOW Module.Name SET DISPLAY '';\nALTER WORKFLOW Module.Name SET DUE DATE '';\nALTER WORKFLOW Module.Name SET OVERVIEW PAGE Module.Page;\nALTER WORKFLOW Module.Name SET ACTIVITY ;\nALTER WORKFLOW Module.Name INSERT AFTER ;\nALTER WORKFLOW Module.Name DROP ACTIVITY ;\nALTER WORKFLOW Module.Name REPLACE ACTIVITY WITH ;\nALTER WORKFLOW Module.Name INSERT OUTCOME '' ON { };\nALTER WORKFLOW Module.Name DROP OUTCOME '' ON ;", - Example: "ALTER WORKFLOW HR.LeaveApproval SET DUE DATE 'addDays([%CurrentDateTime%], 7)';\nALTER WORKFLOW HR.LeaveApproval INSERT AFTER ReviewTask\n CALL MICROFLOW HR.NotifyHR;\nALTER WORKFLOW HR.LeaveApproval DROP ACTIVITY ObsoleteStep;", + // + // The four INSERT ops that add to an activity's outcome list each write + // ONE outcome type, and the list is typed per activity kind — INSERT + // OUTCOME only on a user task, INSERT PATH only on a parallel split, + // INSERT CONDITION only on a decision or call microflow. Aiming one at + // the wrong kind used to produce a project Mendix could not LOAD + // (ako/mxcli#415); it is refused now, but the entry documented only two + // of the ops, which is how an author reached for the wrong one. + Syntax: "ALTER WORKFLOW Module.Name SET DISPLAY '';\nALTER WORKFLOW Module.Name SET DUE DATE '';\nALTER WORKFLOW Module.Name SET OVERVIEW PAGE Module.Page;\nALTER WORKFLOW Module.Name SET ACTIVITY ;\nALTER WORKFLOW Module.Name INSERT AFTER ;\nALTER WORKFLOW Module.Name DROP ACTIVITY ;\nALTER WORKFLOW Module.Name REPLACE ACTIVITY WITH ;\nALTER WORKFLOW Module.Name INSERT OUTCOME '' ON { };\nALTER WORKFLOW Module.Name DROP OUTCOME '' ON ;\nALTER WORKFLOW Module.Name INSERT CONDITION '' ON { };\nALTER WORKFLOW Module.Name INSERT PATH ON { };\nALTER WORKFLOW Module.Name INSERT BOUNDARY EVENT ON TIMER '' { };", + Example: "ALTER WORKFLOW HR.LeaveApproval SET DUE DATE 'addDays([%CurrentDateTime%], 7)';\nALTER WORKFLOW HR.LeaveApproval INSERT AFTER ReviewTask\n CALL MICROFLOW HR.NotifyHR;\nALTER WORKFLOW HR.LeaveApproval DROP ACTIVITY ObsoleteStep;\n\n-- The INSERT op has to match the activity kind: an outcome list is typed,\n-- and the wrong one is refused (it would leave a project Mendix cannot open).\nALTER WORKFLOW HR.LeaveApproval INSERT OUTCOME 'Rejected' ON ReviewTask { };\nALTER WORKFLOW HR.LeaveApproval INSERT CONDITION 'HR.Status.Urgent' ON Triage { };\nALTER WORKFLOW HR.LeaveApproval INSERT PATH ON NotifyAll { };", SeeAlso: []string{"workflow.create", "workflow.drop"}, }) } diff --git a/cmd/mxcli/theme/assets/console/files/theme/web/_mxcli-atlas-map.scss b/cmd/mxcli/theme/assets/console/files/theme/web/_mxcli-atlas-map.scss index fdc03a81a9..25daafc161 100644 --- a/cmd/mxcli/theme/assets/console/files/theme/web/_mxcli-atlas-map.scss +++ b/cmd/mxcli/theme/assets/console/files/theme/web/_mxcli-atlas-map.scss @@ -122,6 +122,25 @@ --btn-default-color: var(--mxt-ink); --btn-default-icon-color: var(--mxt-ink-muted); --btn-link-bg-hover: var(--mxt-surface-hover); + // The primary button is the BRAND colour, not a derivative of it. Atlas + // derives --btn-primary-bg from --brand-primary-600, which is + // color-mix(in srgb, , 20%) — so a brand theme's own colour + // never reaches its main button. Measured: a brand blue #10069F rendered rgb(21,13,140) + // (ako/ChipCoV1), and Signal's #0f6e6b is likewise shifted. + // + // That also settles a mismatch in this file. --btn-primary-color below is + // pinned to --mxt-brand-ink, an ink each theme picks to sit on --mxt-brand — + // console pairs near-black #04211d with bright teal #2dd4bf — so the pairing + // the theme designed for was never the pairing that rendered. The comment + // above already calls this "a brand-filled button"; now it is one. + // + // Hover and active keep Atlas's relationship to the base rather than being + // pinned flat, so a theme that sets --mxt-brand-hover gets a real hover and + // one that does not still gets Atlas's. + --btn-primary-bg: var(--mxt-brand); + --btn-primary-bg-hover: var(--mxt-brand-hover, var(--brand-primary-600)); + --btn-primary-bg-active: var(--mxt-brand-hover, var(--brand-primary-700)); + --btn-primary-border-color: var(--mxt-brand); --btn-primary-color: var(--mxt-brand-ink); --btn-success-color: var(--mxt-brand-ink); --btn-warning-color: var(--mxt-brand-ink); diff --git a/cmd/mxcli/theme/assets/ledger/files/theme/web/_mxcli-atlas-map.scss b/cmd/mxcli/theme/assets/ledger/files/theme/web/_mxcli-atlas-map.scss index fdc03a81a9..25daafc161 100644 --- a/cmd/mxcli/theme/assets/ledger/files/theme/web/_mxcli-atlas-map.scss +++ b/cmd/mxcli/theme/assets/ledger/files/theme/web/_mxcli-atlas-map.scss @@ -122,6 +122,25 @@ --btn-default-color: var(--mxt-ink); --btn-default-icon-color: var(--mxt-ink-muted); --btn-link-bg-hover: var(--mxt-surface-hover); + // The primary button is the BRAND colour, not a derivative of it. Atlas + // derives --btn-primary-bg from --brand-primary-600, which is + // color-mix(in srgb, , 20%) — so a brand theme's own colour + // never reaches its main button. Measured: a brand blue #10069F rendered rgb(21,13,140) + // (ako/ChipCoV1), and Signal's #0f6e6b is likewise shifted. + // + // That also settles a mismatch in this file. --btn-primary-color below is + // pinned to --mxt-brand-ink, an ink each theme picks to sit on --mxt-brand — + // console pairs near-black #04211d with bright teal #2dd4bf — so the pairing + // the theme designed for was never the pairing that rendered. The comment + // above already calls this "a brand-filled button"; now it is one. + // + // Hover and active keep Atlas's relationship to the base rather than being + // pinned flat, so a theme that sets --mxt-brand-hover gets a real hover and + // one that does not still gets Atlas's. + --btn-primary-bg: var(--mxt-brand); + --btn-primary-bg-hover: var(--mxt-brand-hover, var(--brand-primary-600)); + --btn-primary-bg-active: var(--mxt-brand-hover, var(--brand-primary-700)); + --btn-primary-border-color: var(--mxt-brand); --btn-primary-color: var(--mxt-brand-ink); --btn-success-color: var(--mxt-brand-ink); --btn-warning-color: var(--mxt-brand-ink); diff --git a/cmd/mxcli/theme/assets/signal/files/theme/web/_mxcli-atlas-map.scss b/cmd/mxcli/theme/assets/signal/files/theme/web/_mxcli-atlas-map.scss index fdc03a81a9..25daafc161 100644 --- a/cmd/mxcli/theme/assets/signal/files/theme/web/_mxcli-atlas-map.scss +++ b/cmd/mxcli/theme/assets/signal/files/theme/web/_mxcli-atlas-map.scss @@ -122,6 +122,25 @@ --btn-default-color: var(--mxt-ink); --btn-default-icon-color: var(--mxt-ink-muted); --btn-link-bg-hover: var(--mxt-surface-hover); + // The primary button is the BRAND colour, not a derivative of it. Atlas + // derives --btn-primary-bg from --brand-primary-600, which is + // color-mix(in srgb, , 20%) — so a brand theme's own colour + // never reaches its main button. Measured: a brand blue #10069F rendered rgb(21,13,140) + // (ako/ChipCoV1), and Signal's #0f6e6b is likewise shifted. + // + // That also settles a mismatch in this file. --btn-primary-color below is + // pinned to --mxt-brand-ink, an ink each theme picks to sit on --mxt-brand — + // console pairs near-black #04211d with bright teal #2dd4bf — so the pairing + // the theme designed for was never the pairing that rendered. The comment + // above already calls this "a brand-filled button"; now it is one. + // + // Hover and active keep Atlas's relationship to the base rather than being + // pinned flat, so a theme that sets --mxt-brand-hover gets a real hover and + // one that does not still gets Atlas's. + --btn-primary-bg: var(--mxt-brand); + --btn-primary-bg-hover: var(--mxt-brand-hover, var(--brand-primary-600)); + --btn-primary-bg-active: var(--mxt-brand-hover, var(--brand-primary-700)); + --btn-primary-border-color: var(--mxt-brand); --btn-primary-color: var(--mxt-brand-ink); --btn-success-color: var(--mxt-brand-ink); --btn-warning-color: var(--mxt-brand-ink); diff --git a/cmd/mxcli/theme/create.go b/cmd/mxcli/theme/create.go index 5a22c8e097..b379b6e72c 100644 --- a/cmd/mxcli/theme/create.go +++ b/cmd/mxcli/theme/create.go @@ -67,7 +67,7 @@ func Create(projectDir, name string, opts CreateOptions) (*CreateResult, error) res.Tokens = tokens } - rewrite := newRewriter(baseTheme, name, opts.Title) + rewrite := newRewriter(baseTheme, name, opts.Title, tokens) // Tokens are validated against the base before anything is written. A // design that names --mxt-brand-color instead of --mxt-brand would @@ -80,6 +80,17 @@ func Create(projectDir, name string, opts CreateOptions) (*CreateResult, error) } root := src.filesRoot() + + // Decide about fonts BEFORE walking, not while walking. The decision is + // made by reading the theme partial and the files it affects are elsewhere + // in the tree, so doing it inline would depend on WalkDir's lexical order + // putting `_mxcli-.scss` before `mxcli-fonts/` — true today only + // because of the leading underscore, and silently wrong the moment a + // partial is renamed. + if err := rewrite.planFonts(src, root, tokens); err != nil { + return nil, err + } + walkErr := fs.WalkDir(src.fsys, root, func(p string, d fs.DirEntry, err error) error { if err != nil || d.IsDir() { return err @@ -102,9 +113,18 @@ func Create(projectDir, name string, opts CreateOptions) (*CreateResult, error) return err } } + if len(rewrite.droppedFonts) > 0 { + text = dropFontFaces(text, rewrite.droppedFonts) + } out = []byte(text) } + // A vendored font file nothing loads any more is dead weight shipped + // with a SIL OFL licence for fonts the theme does not use. + if rewrite.dropsFontFile(rel) { + return nil + } + target := filepath.Join(dest, "files", filepath.FromSlash(rewrite.path(rel))) // Reported relative to the scaffold root, not to the project. Bare, the // two most important entries read as theme/web/custom-variables.scss @@ -222,13 +242,29 @@ type CreateResult struct { type rewriter struct { baseName, baseTitle string newName, newTitle string + // tokens is the seeded palette, when one was given. The manifest derives + // the colorway and summary from it rather than inheriting the base's, + // which describe a palette that is no longer there. + tokens *Tokens + // droppedFonts are the vendored families the seeded fonts no longer name. + // Collected while rewriting the partial and consumed when filtering files, + // so the files and the @font-face rules that load them are dropped + // together — either alone leaves a theme that 404s or ships dead weight. + droppedFonts []string + // keptAnyFont records whether any @font-face survived, which decides + // whether the mxcli-fonts/ directory and its licence are still shipped. + keptAnyFont bool } -func newRewriter(base *Theme, newName, newTitle string) *rewriter { +func newRewriter(base *Theme, newName, newTitle string, tokens *Tokens) *rewriter { if newTitle == "" { newTitle = defaultTitle(newName) } - return &rewriter{baseName: base.Name, baseTitle: base.Title, newName: newName, newTitle: newTitle} + return &rewriter{ + baseName: base.Name, baseTitle: base.Title, + newName: newName, newTitle: newTitle, + tokens: tokens, keptAnyFont: true, + } } // defaultTitle turns "acme-dark" into "Acme Dark". @@ -282,6 +318,14 @@ func (r *rewriter) manifest(base *Theme, opts CreateOptions) ([]byte, error) { t.Title = r.newTitle t.Local = false t.Version = "1" + // A seeded palette makes the base's summary and swatches statements about + // the wrong theme: `mxcli theme list` showed a brand theme as "Cool slate, + // one teal signal colour" with Signal's six swatches. Derive what can be + // derived; --summary still wins. + if r.tokens != nil { + t.Colorway = deriveColorway(base, r.tokens) + t.Summary = seededSummary(r.tokens) + } if opts.Summary != "" { t.Summary = opts.Summary } @@ -291,6 +335,17 @@ func (r *rewriter) manifest(base *Theme, opts CreateOptions) ([]byte, error) { base.Title, r.newName, r.newName) t.Files = append([]FileSpec(nil), base.Files...) + if !r.keptAnyFont { + // The manifest is what `theme show` reads, so an entry for a directory + // the scaffold no longer writes describes files that are not there. + kept := t.Files[:0] + for _, f := range t.Files { + if !isVendoredFontPath(f.Path) { + kept = append(kept, f) + } + } + t.Files = kept + } for i := range t.Files { t.Files[i].Path = r.path(t.Files[i].Path) // The purpose prose names the licence file, so it has to follow the diff --git a/cmd/mxcli/theme/create_seeded.go b/cmd/mxcli/theme/create_seeded.go new file mode 100644 index 0000000000..a32ef7bac7 --- /dev/null +++ b/cmd/mxcli/theme/create_seeded.go @@ -0,0 +1,210 @@ +// SPDX-License-Identifier: Apache-2.0 + +package theme + +import ( + "fmt" + "io/fs" + "path" + "regexp" + "strings" +) + +// A theme scaffolded with `--from ` gets the design's palette and +// used to keep everything else from the base theme verbatim. Two consequences, +// both reported by ako/ChipCoV1: +// +// - A brand-seeded theme still described itself in `mxcli theme list` as +// "Cool slate, one teal signal colour", and still showed Signal's six +// swatches — so the one command whose job is to tell themes apart showed the +// wrong one. Corrected by hand there; it should not need correcting. +// - It vendored ~500 KB of IBM Plex woff2 that the seeded `--mxt-font` never +// names, plus the @font-face rules loading them and a SIL OFL licence for +// fonts the theme does not use. +// +// Both are the same mistake: inheriting a statement ABOUT the base theme into a +// theme whose palette is no longer the base's. A value that describes the design +// is derived; a value that cannot be derived is dropped rather than inherited, +// because an inherited one is a confident lie and an absent one is merely +// missing. + +// colorwayTokens are the palette entries a colorway shows, in swatch order. +// This is the mapping the built-in themes already use — Signal's colorway is +// exactly its brand/info/success/warning/danger/ink-muted — so deriving one +// reproduces the hand-written value rather than inventing a new convention. +var colorwayTokens = []string{ + "--mxt-brand", + "--mxt-info", + "--mxt-success", + "--mxt-warning", + "--mxt-danger", + "--mxt-ink-muted", +} + +// deriveColorway builds the swatch list from the seeded palette, falling back +// to the base theme's value for any entry the design did not declare — a +// partial seed is normal (a brand usually pins its brand colour and leaves the +// semantic ones alone), and a hole in the swatch row would read as a defect. +func deriveColorway(base *Theme, tokens *Tokens) []string { + if tokens == nil { + return base.Colorway + } + out := make([]string, 0, len(colorwayTokens)) + for i, name := range colorwayTokens { + if v, ok := tokens.Base[name]; ok && strings.TrimSpace(v) != "" { + out = append(out, strings.TrimSpace(v)) + continue + } + if i < len(base.Colorway) { + out = append(out, base.Colorway[i]) + } + } + if len(out) == 0 { + return base.Colorway + } + return out +} + +// seededSummary replaces the base theme's summary, which describes the base's +// colours and is wrong the moment a palette is seeded over it. +// +// It states what is true and checkable — where the palette came from and how +// many declarations it carried — rather than trying to describe colours in +// prose. `--summary` overrides it for a theme that wants a real description. +func seededSummary(tokens *Tokens) string { + if tokens == nil { + return "" + } + n := tokens.Count() + decl := "declarations" + if n == 1 { + decl = "declaration" + } + return fmt.Sprintf("Project theme; palette seeded from %s (%d %s).", tokens.Source, n, decl) +} + +// fontFaceFamilyRe finds the family each @font-face block declares. The built-in +// themes group faces by family in one `@each $weight` block per family, so the +// family name is what identifies a block to keep or drop. +var fontFaceFamilyRe = regexp.MustCompile(`(?s)@each\s+\$weight[^{]*\{\s*@font-face\s*\{[^}]*font-family:\s*"([^"]+)"`) + +// vendoredFamilies lists the font families a theme partial loads with +// @font-face, in the order they appear. +func vendoredFamilies(scss string) []string { + var out []string + for _, m := range fontFaceFamilyRe.FindAllStringSubmatch(scss, -1) { + out = append(out, m[1]) + } + return out +} + +// unusedVendoredFamilies returns the vendored families the seeded font stacks no +// longer name. +// +// The test is on the SEEDED value only. A design that does not mention fonts at +// all seeds none, every family stays, and the theme keeps working — which is the +// right default, because dropping a font nobody asked to change would break the +// scaffold's own rendering. +func unusedVendoredFamilies(scss string, tokens *Tokens) []string { + if tokens == nil { + return nil + } + var stacks []string + for _, name := range []string{"--mxt-font", "--mxt-font-heading", "--mxt-font-mono"} { + if v, ok := tokens.Base[name]; ok { + stacks = append(stacks, strings.ToLower(v)) + } + } + if len(stacks) == 0 { + return nil // fonts were not seeded; nothing to drop + } + declared := strings.ToLower(strings.Join(stacks, " | ")) + + var unused []string + for _, fam := range vendoredFamilies(scss) { + if !strings.Contains(declared, strings.ToLower(fam)) { + unused = append(unused, fam) + } + } + return unused +} + +// dropFontFaces removes the @font-face blocks for the named families, and the +// section comment when nothing is left to explain. +func dropFontFaces(scss string, families []string) string { + for _, fam := range families { + re := regexp.MustCompile(`(?s)\n*@each\s+\$weight[^{]*\{\s*@font-face\s*\{[^}]*font-family:\s*"` + + regexp.QuoteMeta(fam) + `"[^}]*\}[^}]*\}`) + scss = re.ReplaceAllString(scss, "") + } + if len(vendoredFamilies(scss)) == 0 { + // The banner explains vendored fonts; with none left it describes + // nothing, and a comment about files the theme does not ship is how the + // next reader concludes the fonts went missing by accident. + scss = fontSectionCommentRe.ReplaceAllString(scss, "") + } + return strings.TrimRight(scss, "\n") + "\n" +} + +// fontSectionCommentRe matches the vendored-fonts banner comment. +var fontSectionCommentRe = regexp.MustCompile(`(?m)\n*^// -+\n(?:^// .*\n)*?^// Fonts\. Vendored.*\n(?:^//.*\n)*^// -+\n`) + +// isVendoredFontPath reports whether a scaffold-relative path is one of the +// vendored font files or their licence. +func isVendoredFontPath(rel string) bool { + return strings.Contains(rel, "mxcli-fonts/") +} + +// planFonts decides which vendored families the new theme keeps, before any +// file is written. +// +// It reads the base theme's partial — the one file that carries @font-face — +// rather than waiting to meet it during the walk, so the outcome does not +// depend on WalkDir reaching the partial before the woff2 files it loads. +func (r *rewriter) planFonts(src source, root string, tokens *Tokens) error { + if tokens == nil { + return nil // nothing seeded; the base theme's fonts stand + } + partial := root + "/theme/web/_mxcli-" + r.baseName + ".scss" + body, err := fs.ReadFile(src.fsys, partial) + if err != nil { + // A base theme with no such partial simply has no vendored fonts to + // reason about. That is not an error — it is a theme that already + // relies on system fonts. + return nil + } + scss := string(body) + r.droppedFonts = unusedVendoredFamilies(scss, tokens) + r.keptAnyFont = len(r.droppedFonts) < len(vendoredFamilies(scss)) + return nil +} + +// dropsFontFile reports whether a scaffold-relative path is a font file the new +// theme no longer loads. +// +// Per family, not all-or-nothing: a brand theme routinely changes its body font +// and keeps the mono one for code, and shipping four unused Sans weights +// because Mono survived is the same dead weight in smaller print. The licence +// and any other non-woff2 file under mxcli-fonts/ is kept as long as ANY family +// is still loaded, because it covers the ones that remain. +func (r *rewriter) dropsFontFile(rel string) bool { + if !isVendoredFontPath(rel) { + return false + } + if !r.keptAnyFont { + return true // nothing loads any of them, licence included + } + base := strings.ToLower(path.Base(rel)) + for _, fam := range r.droppedFonts { + if strings.HasPrefix(base, fontFileSlug(fam)+"-") { + return true + } + } + return false +} + +// fontFileSlug turns a family name into the filename stem the vendored files +// use: "IBM Plex Sans" -> "ibm-plex-sans". +func fontFileSlug(family string) string { + return strings.ToLower(strings.ReplaceAll(strings.TrimSpace(family), " ", "-")) +} diff --git a/cmd/mxcli/theme/create_seeded_test.go b/cmd/mxcli/theme/create_seeded_test.go new file mode 100644 index 0000000000..3eb8ca89e5 --- /dev/null +++ b/cmd/mxcli/theme/create_seeded_test.go @@ -0,0 +1,276 @@ +// SPDX-License-Identifier: Apache-2.0 + +package theme + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +// The partial's real shape: one `@each $weight` block per vendored family. +const probePartial = ` +// --------------------------------------------------------------------------- +// Fonts. Vendored (SIL OFL 1.1, see mxcli-fonts/OFL-signal.txt) rather than pulled +// from a CDN: no @import ordering trap, no third-party request at runtime. +// --------------------------------------------------------------------------- +@each $weight in (400, 500, 600, 700) { + @font-face { + font-family: "IBM Plex Sans"; + src: url("./mxcli-fonts/ibm-plex-sans-latin-#{$weight}-normal.woff2") format("woff2"); + font-weight: $weight; + } +} + +@each $weight in (400, 500, 600) { + @font-face { + font-family: "IBM Plex Mono"; + src: url("./mxcli-fonts/ibm-plex-mono-latin-#{$weight}-normal.woff2") format("woff2"); + font-weight: $weight; + } +} +` + +func tokensWith(pairs map[string]string) *Tokens { + set := TokenSet{} + for k, v := range pairs { + set[k] = v + } + return &Tokens{Source: "design.css", Base: set} +} + +func TestVendoredFamilies(t *testing.T) { + got := vendoredFamilies(probePartial) + want := []string{"IBM Plex Sans", "IBM Plex Mono"} + if len(got) != len(want) { + t.Fatalf("got %v, want %v", got, want) + } + for i := range want { + if got[i] != want[i] { + t.Errorf("family %d = %q, want %q", i, got[i], want[i]) + } + } +} + +func TestUnusedVendoredFamilies(t *testing.T) { + tests := []struct { + name string + tokens *Tokens + want []string + }{ + { + name: "fonts not seeded at all — nothing is dropped", + tokens: tokensWith(map[string]string{"--mxt-brand": "#10069F"}), + want: nil, + }, + { + name: "no tokens at all — scaffolding from a theme", + tokens: nil, + want: nil, + }, + { + name: "a brand font replaces both", + tokens: tokensWith(map[string]string{ + "--mxt-font": `"Neue Haas Grotesk", Helvetica, sans-serif`, + "--mxt-font-mono": `ui-monospace, Menlo, monospace`, + }), + want: []string{"IBM Plex Sans", "IBM Plex Mono"}, + }, + { + name: "keeping the mono font keeps only its faces", + tokens: tokensWith(map[string]string{ + "--mxt-font": `"Neue Haas Grotesk", Helvetica, sans-serif`, + "--mxt-font-mono": `"IBM Plex Mono", ui-monospace, monospace`, + }), + want: []string{"IBM Plex Sans"}, + }, + { + name: "a heading font that still names the family keeps it", + tokens: tokensWith(map[string]string{ + "--mxt-font-heading": `"IBM Plex Sans", sans-serif`, + }), + want: []string{"IBM Plex Mono"}, + }, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + got := unusedVendoredFamilies(probePartial, tc.tokens) + if strings.Join(got, ",") != strings.Join(tc.want, ",") { + t.Errorf("got %v, want %v", got, tc.want) + } + }) + } +} + +func TestDropFontFaces(t *testing.T) { + // Dropping one family leaves the other, and leaves the banner explaining it. + one := dropFontFaces(probePartial, []string{"IBM Plex Sans"}) + if fams := vendoredFamilies(one); len(fams) != 1 || fams[0] != "IBM Plex Mono" { + t.Errorf("after dropping Sans: %v, want [IBM Plex Mono]", fams) + } + if strings.Contains(one, "ibm-plex-sans-latin") { + t.Error("the dropped family's src urls must go with it") + } + if !strings.Contains(one, "Fonts. Vendored") { + t.Error("the banner still explains the family that remains") + } + + // Dropping all of them takes the banner too: a comment about vendored fonts + // a theme does not ship reads as fonts having gone missing by accident. + all := dropFontFaces(probePartial, []string{"IBM Plex Sans", "IBM Plex Mono"}) + if fams := vendoredFamilies(all); len(fams) != 0 { + t.Errorf("after dropping both: %v, want none", fams) + } + if strings.Contains(all, "Fonts. Vendored") { + t.Error("with no families left the banner describes nothing and must go") + } + if strings.Contains(all, "@font-face") { + t.Error("no @font-face may survive") + } +} + +// Per-family file dropping. Shipping four unused Sans weights because Mono +// survived is the same dead weight in smaller print. +func TestDropsFontFile(t *testing.T) { + r := &rewriter{droppedFonts: []string{"IBM Plex Sans"}, keptAnyFont: true} + for path, want := range map[string]bool{ + "theme/web/mxcli-fonts/ibm-plex-sans-latin-400-normal.woff2": true, + "theme/web/mxcli-fonts/ibm-plex-mono-latin-400-normal.woff2": false, + "theme/web/mxcli-fonts/OFL-signal.txt": false, // covers the family that stayed + "theme/web/custom-variables.scss": false, + } { + if got := r.dropsFontFile(path); got != want { + t.Errorf("dropsFontFile(%q) = %v, want %v", path, got, want) + } + } + + // With nothing loaded, everything under mxcli-fonts/ goes — licence too, + // since it covers fonts the theme no longer ships. + none := &rewriter{droppedFonts: []string{"IBM Plex Sans", "IBM Plex Mono"}, keptAnyFont: false} + for _, p := range []string{ + "theme/web/mxcli-fonts/ibm-plex-mono-latin-400-normal.woff2", + "theme/web/mxcli-fonts/OFL-signal.txt", + } { + if !none.dropsFontFile(p) { + t.Errorf("with no families kept, %q must be dropped", p) + } + } + if none.dropsFontFile("theme/web/main.scss") { + t.Error("a non-font file must never be dropped") + } +} + +// The colorway is what `mxcli theme list` shows, and an inherited one shows the +// base theme's colours for a theme that no longer has them. +func TestDeriveColorway(t *testing.T) { + base := &Theme{Colorway: []string{"#0F6E6B", "#1F5FA8", "#1F7A4D", "#B45309", "#B42318", "#5A6572"}} + + // A design that pins only some entries keeps the base's for the rest — a + // partial seed is normal, and a hole in the swatch row reads as a defect. + got := deriveColorway(base, tokensWith(map[string]string{ + "--mxt-brand": "#10069F", + "--mxt-info": "#1297E4", + })) + if got[0] != "#10069F" || got[1] != "#1297E4" { + t.Errorf("seeded entries not used: %v", got) + } + if got[4] != "#B42318" || got[5] != "#5A6572" { + t.Errorf("unseeded entries must fall back to the base: %v", got) + } + if len(got) != len(base.Colorway) { + t.Errorf("length changed: %d, want %d", len(got), len(base.Colorway)) + } + + // Scaffolding from a theme seeds nothing, so the base's colorway stands. + if same := deriveColorway(base, nil); len(same) != len(base.Colorway) || same[0] != base.Colorway[0] { + t.Errorf("with no tokens the base colorway must be kept, got %v", same) + } +} + +func TestSeededSummaryNamesTheSource(t *testing.T) { + s := seededSummary(tokensWith(map[string]string{"--mxt-brand": "#10069F"})) + if !strings.Contains(s, "design.css") { + t.Errorf("summary must name where the palette came from: %q", s) + } + if strings.Contains(strings.ToLower(s), "teal") || strings.Contains(strings.ToLower(s), "slate") { + t.Errorf("summary must not describe the BASE theme's colours: %q", s) + } + if seededSummary(nil) != "" { + t.Error("with no tokens there is nothing to say, and the base's summary stands") + } +} + +// The invariant that matters most, at the level it actually breaks: after a +// scaffold, every @font-face URL must resolve to a file the theme ships, and +// every file it ships must be loaded by one. +// +// Dropping a family touches BOTH halves — the @font-face block and the woff2 +// files — and getting either alone wrong is silent: a surviving rule for a +// deleted file 404s in the browser, and a surviving file nothing loads is the +// dead weight this fix exists to remove. Unit tests on each half cannot catch a +// mismatch between them. +func TestScaffoldedThemeShipsExactlyTheFontsItLoads(t *testing.T) { + tests := map[string]struct { + css string + wantLoaded []string + }{ + "a brand font replaces both": { + css: `:root { + --mxt-brand: #10069F; + --mxt-font: "Neue Haas Grotesk", Helvetica, sans-serif; + --mxt-font-mono: ui-monospace, Menlo, monospace; + }`, + wantLoaded: nil, + }, + "keeping mono keeps only its files": { + css: `:root { + --mxt-brand: #10069F; + --mxt-font: "Neue Haas Grotesk", Helvetica, sans-serif; + --mxt-font-mono: "IBM Plex Mono", ui-monospace, monospace; + }`, + wantLoaded: []string{"IBM Plex Mono"}, + }, + "a design that says nothing about fonts keeps both": { + css: `:root { --mxt-brand: #10069F; }`, + wantLoaded: []string{"IBM Plex Sans", "IBM Plex Mono"}, + }, + } + + for name, tc := range tests { + t.Run(name, func(t *testing.T) { + dir := newProject(t) + design := filepath.Join(dir, "design.css") + write(t, design, tc.css) + + if _, err := Create(dir, "probe", CreateOptions{From: design}); err != nil { + t.Fatal(err) + } + themeDir := filepath.Join(dir, "theme", "mxcli-themes", "probe", "files", "theme", "web") + partial := read(t, filepath.Join(themeDir, "_mxcli-probe.scss")) + + if got := vendoredFamilies(partial); strings.Join(got, ",") != strings.Join(tc.wantLoaded, ",") { + t.Errorf("families loaded = %v, want %v", got, tc.wantLoaded) + } + + // Every URL resolves to a shipped file. + shipped := map[string]bool{} + entries, _ := os.ReadDir(filepath.Join(themeDir, "mxcli-fonts")) + for _, e := range entries { + if strings.HasSuffix(e.Name(), ".woff2") { + shipped[e.Name()] = true + } + } + for _, name := range fontRefs(partial) { + if !shipped[name] { + t.Errorf("@font-face loads %s but the theme does not ship it", name) + } + delete(shipped, name) + } + // And nothing is shipped that no rule loads. + for leftover := range shipped { + t.Errorf("%s is shipped but no @font-face loads it", leftover) + } + }) + } +} diff --git a/docs-site/src/SUMMARY.md b/docs-site/src/SUMMARY.md index 992b04634f..b6ef406e30 100644 --- a/docs-site/src/SUMMARY.md +++ b/docs-site/src/SUMMARY.md @@ -258,6 +258,7 @@ - [GRANT](reference/security/grant.md) - [REVOKE](reference/security/revoke.md) - [CREATE DEMO USER](reference/security/create-demo-user.md) + - [UPDATE SECURITY](reference/security/update-security.md) - [Navigation Statements](reference/navigation/README.md) - [ALTER NAVIGATION](reference/navigation/alter-navigation.md) - [SHOW NAVIGATION](reference/navigation/show-navigation.md) diff --git a/docs-site/src/appendixes/error-messages.md b/docs-site/src/appendixes/error-messages.md index 3aa32df789..fe9a642e2e 100644 --- a/docs-site/src/appendixes/error-messages.md +++ b/docs-site/src/appendixes/error-messages.md @@ -50,9 +50,31 @@ See the debug workflow in `.claude/skills/debug-bson.md` for step-by-step instru CE0066: Entity access for 'MyModule.Customer' is out of date. ``` -**Cause:** Association MemberAccess entries were added to the wrong entity. In Mendix, association access rules must only be on the **FROM** entity (the one stored in `ParentPointer`), not the TO entity. +**Cause (1): the rules no longer cover the entity's members.** An access rule +names the members it governs, so an entity that has gained an attribute or an +association — or lost one — leaves every rule on it out of date. This is the +usual cause for a model that arrived from somewhere else: a module imported or +updated outside Studio Pro whose author never pressed **Update security**, a +hand-edited `.mpr`, a merge. -**Solution:** Ensure `MemberAccess` entries for associations are added only to the entity that owns the foreign key (the FROM side of the association). Remove any association MemberAccess entries from the TO entity. +**Solution:** run the headless equivalent of that button: + +```bash +mxcli -p app.mpr -c "update security MyModule" +``` + +It reports what it changed (`Reconciled 3 access rule(s) in module MyModule`) and +writes nothing when the rules already match. See +[UPDATE SECURITY](../reference/security/update-security.md). `mxcli marketplace +install` and `mxcli marketplace update` run it for the module they copy in. + +**Cause (2): association MemberAccess entries on the wrong entity.** In Mendix, +association access rules must only be on the **FROM** entity (the one stored in +`ParentPointer`), not the TO entity. + +**Solution:** Ensure `MemberAccess` entries for associations are added only to the +entity that owns the foreign key (the FROM side of the association). Remove any +association MemberAccess entries from the TO entity. ### System.ArgumentNullException (ValidationRule) diff --git a/docs-site/src/guides/marketplace.md b/docs-site/src/guides/marketplace.md index 4cdee5c873..69d40b3531 100644 --- a/docs-site/src/guides/marketplace.md +++ b/docs-site/src/guides/marketplace.md @@ -176,6 +176,19 @@ mxcli diff-local -p app.mpr A headless install or update leaves two repairs for Mendix's own tools: **CE0463** (the project's stored widget instances are older than the widget packages beside them) and **CE6087** (a module references design properties an older Atlas spelled differently). Both are expected, not faults in the install. See [`mxcli fix`](#repairing-the-model-mxcli-fix) below. +A third repair is **not** left to you: **CE0066** ("Entity access is out of date"). A transplant copies the incoming module's units in verbatim, so a package whose access rules do not cover every member of their entities would land that error in your project with nothing said about it (mendixlabs/mxcli#1085). Install and update now reconcile the module's rules for themselves and report the count: + +``` + 3 entity access rule(s) reconciled — the package's rules did not cover every + member of their entities, which Mendix reports as CE0066. +``` + +Nothing is written when the rules already match, so a well-formed package comes through untouched. When the line does appear, the module's domain model differs from the package — the same change Studio Pro's **Update security** button makes — and `mxcli marketplace diff` will read it as a local edit. To run the repair by hand, or on a project updated by an older mxcli, use [`UPDATE SECURITY`](../reference/security/update-security.md): + +```bash +mxcli -p app.mpr -c "update security UserCommons" +``` + Measured: Administration 4.3.2 → 4.5.0 and DataWidgets 3.5.0 → 3.11.3 both reach **0 errors** afterwards. ## Bundled widgets and install order diff --git a/docs-site/src/reference/page/create-page.md b/docs-site/src/reference/page/create-page.md index 2c5e5b48f8..459f081539 100644 --- a/docs-site/src/reference/page/create-page.md +++ b/docs-site/src/reference/page/create-page.md @@ -112,7 +112,9 @@ The `DataSource` property determines how a data widget obtains its data: ### Action Types -The `Action` property on buttons determines what happens when clicked: +The `Action` property determines what happens when the widget is clicked. It is +not a button-only property: a `CONTAINER` is clickable too and takes the same +values, including the argument list. | Action | Syntax | Description | |--------|--------|-------------| @@ -124,6 +126,26 @@ The `Action` property on buttons determines what happens when clicked: | Close | `Action: CLOSE_PAGE` | Closes the current page | | Delete | `Action: DELETE` | Deletes the context object | +A microflow or nanoflow action is a **call**: every parameter the flow declares +needs an argument, or Mendix rejects the page with **CE1571**. An enclosing data +container of the parameter's type supplies it without one — but a data grid's +**control bar** does not, because it is not row-scoped. Pass the grid's selection +there, addressed by the widget's own name: + +```sql +DATAGRID dgOrders (DataSource: DATABASE FROM Sales.Order, Selection: Single) { + COLUMN colNr (Attribute: Number, Caption: 'Order #') + CONTROLBAR cb { + CONTAINER cShip (Class: 'command', + Action: NANOFLOW Sales.ACT_Ship($Order = $dgOrders)) { + ACTIONBUTTON btnShip (Caption: 'Ship') + } + } +} +``` + +`mxcli check -p ` reports a missing argument before mxbuild does. + ### ButtonStyle Values `Primary`, `Default`, `Success`, `Danger`, `Warning`, `Info`. diff --git a/docs-site/src/reference/security/README.md b/docs-site/src/reference/security/README.md index 283436de9d..45a42e33e2 100644 --- a/docs-site/src/reference/security/README.md +++ b/docs-site/src/reference/security/README.md @@ -13,6 +13,7 @@ Mendix security operates at two levels. **Module roles** define permissions with | [GRANT](grant.md) | Grant entity, microflow, page, or nanoflow access to roles | | [REVOKE](revoke.md) | Remove previously granted access | | [CREATE DEMO USER](create-demo-user.md) | Create a demo user for development and testing | +| [UPDATE SECURITY](update-security.md) | Re-sync entity access rules with the domain model (CE0066) | ## Related Statements diff --git a/docs-site/src/reference/security/update-security.md b/docs-site/src/reference/security/update-security.md new file mode 100644 index 0000000000..2f517c1e70 --- /dev/null +++ b/docs-site/src/reference/security/update-security.md @@ -0,0 +1,92 @@ +# UPDATE SECURITY + +Bring entity access rules back into sync with their domain model — the headless +equivalent of Studio Pro's **Update security** button in the domain model editor. + +## Syntax + +```sql +UPDATE SECURITY; +UPDATE SECURITY ; +UPDATE SECURITY IN ; +``` + +With no module, every module in the project is reconciled. The two scoped forms +are the same thing; `IN` is optional. + +## What it does + +An access rule names the members it governs. When the entity gains an attribute +or an association, every rule on it needs an entry for the new member — and when +a member goes, its entries have to go with it. A rule that has fallen behind is +what Mendix reports as: + +``` +[error] [CE0066] "Entity access is out of date. Please update security by + clicking the 'Update security' button in the domain model editor." + at Domain model of module 'MyModule' +``` + +`UPDATE SECURITY` adds what is missing, removes what is stale, and downgrades +write rights on calculated attributes. It reports what it changed: + +``` +Reconciled 3 access rule(s) in module Administration +``` + +A project whose rules already cover their entities is **not written to** — the +command says `All entity access rules are up to date` and leaves the model alone. + +`System` is skipped: its entities are the platform's and its access rules are +not the project's to rewrite. Naming it explicitly is an error rather than a +silent no-op. + +## When you need it + +Rarely, for models mxcli writes. Every mxcli write path reconciles as it writes — +`GRANT`, `ALTER ENTITY`, `CREATE ASSOCIATION`, and the finalize step after any +script — so an MDL script leaves the rules correct. + +It is for a model that arrived from somewhere else: + +- a module imported or updated **outside Studio Pro**, whose author never pressed + Update security, so the package ships rules that do not cover every member; +- a hand-edited or merged `.mpr`; +- a project last touched by a Mendix version whose conversion added members. + +`mxcli marketplace install` and `mxcli marketplace update` run this for the module +they copy in, and report the count — a transplant copies the incoming units +verbatim, so without it the package's rules reach the project unchecked +(mendixlabs/mxcli#1085). + +## Examples + +```sql +-- After a headless module install or update +UPDATE SECURITY UserCommons; + +-- Every module in the project +UPDATE SECURITY; +``` + +From the shell, without writing a script: + +```bash +mxcli -p app.mpr -c "update security UserCommons" +``` + +## Notes + +- Reconciling a marketplace module's rules changes that module, so it shows up in + `mxcli diff-local` and `mxcli marketplace diff` reads it as a local edit. That + is unavoidable — it is the same change Studio Pro's button makes — and it only + happens when the rules were genuinely incomplete. +- The rules of an entity whose generalization lives in **another module** cannot + be fully checked here: those inherited members are neither added nor removed, + but existing entries for them are preserved rather than pruned. + +## See also + +- [GRANT](grant.md) — author entity access rules +- [REVOKE](revoke.md) — remove them +- [Error messages](../../appendixes/error-messages.md) — CE0066 diff --git a/docs-site/src/tools/run-local.md b/docs-site/src/tools/run-local.md index e64ba58243..5977fa1515 100644 --- a/docs-site/src/tools/run-local.md +++ b/docs-site/src/tools/run-local.md @@ -375,6 +375,59 @@ not just headless checks. re-bundles **only when the edit touched client source**: a microflow/entity edit skips the bundle and just hot-reloads. +**Mendix 11.14+ bundles the client itself**, so there is no rollup step to run and +no bundler to keep hot — `run --local` prints a line saying so and skips it. Both +paths work; the bundle is mxbuild's rather than mxcli's. + +### `--watch` on Mendix 11.14 + +**`--watch` is not usable on 11.14 yet.** It starts and the app boots, but every +*rebuild* fails inside mxbuild: + +``` +Could not find a part of the path '…/deployment/web/pages/MyModule.Home_Web.js' +``` + +or, when the change did not touch any page: + +``` +Compilation of the app bundle failed. +Cannot find module '…/deployment/web/rollup.config.mjs' + imported from …/modeler/tools/node/rollup-runner.mjs +``` + +The first build in an `mxbuild --serve` process does not leave the deployment in a +state its own incremental build can continue from — neither the bundler's config +file nor `web/pages/`/`web/layouts/` survive it — so the first build succeeds and +every later one fails. Measured against mxbuild 11.14.0 over its own HTTP API with +no mxcli involved: the same `/build` request POSTed twice, model untouched between +them, goes Success then Failure. + +No remedy from outside the process works, and three were measured: + +| Attempt | Result | +|---|---| +| Switch *App > Settings > Runtime > App bundler* to Rspack | Identical failure, naming `rspack.config.mjs` | +| Delete `deployment/` and start over | Next second build fails the same way | +| Restore the config (it exists for ~1.5s mid-build) | Rebuilds fine **while the model is unchanged**; fails as soon as a page changes | + +That last one is the interesting near-miss: the config carries nothing +model-specific, so it can be captured and put back — but doing so only rescues the +case a warm loop never needs. There are two regressions here, and the second one +(the per-document client export) has no external fix. `run --local` prints an +explanation when it sees either, so the failure does not read as a corrupt +`deployment/`. + +For contrast, a one-shot `mxbuild --target=deploy` run twice into the same +deployment directory succeeds both times — it is the serve process, not the 11.14 +deployment shape. + +Until mxbuild closes this, use a restart per change: + +```bash +mxcli run --local --screenshot -p app.mpr +``` + ## Pixel-perfect page loop Pass `--screenshot` and each applied change is captured to a PNG (default diff --git a/docs-site/src/tools/theme.md b/docs-site/src/tools/theme.md index 5696d0166e..6636339a2e 100644 --- a/docs-site/src/tools/theme.md +++ b/docs-site/src/tools/theme.md @@ -201,6 +201,22 @@ seeds one by declaring them. That is the whole contract — anywhere in a `.css` @media (prefers-color-scheme: dark) { :root { --mxt-ground: #16161a; } } ``` +Seeding also updates what the new theme **says about itself**, because a value +inherited from the base describes a palette that is no longer there: + +- the **colorway** `theme list` shows is derived from the seeded + brand/info/success/warning/danger/ink-muted, falling back to the base's for + any the design did not declare (a partial seed is normal); +- the **summary** names the design file it came from, rather than keeping the + base's description of its own colours. Pass `--summary` for a real one. + +And if the seeded fonts no longer name a family the base theme vendors, that +family's `@font-face` rules **and its `.woff2` files** are dropped together — +per family, so a theme that changes its body font and keeps IBM Plex Mono for +code ships only the mono weights. A design that says nothing about fonts keeps +them all, which is the right default: dropping a font nobody asked to change +would break the scaffold's own rendering. + Declarations inside a dark block (`prefers-color-scheme: dark`, `.theme-dark`, `[data-theme="dark"]`) seed the dark palette; everything else seeds the light one. Tokens the design does not name keep the base theme's value, so a diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index 25aee2929a..2a625246bc 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -637,6 +637,7 @@ Nested folders use `/` separator: `'Parent/Child/Grandchild'`. Missing folders a | Disable guest access | `alter project security guest access off;` | Keeps the stored role, so re-enabling needs no `role` clause | | Create demo user | `create demo user 'name' password 'pass' [entity Module.Entity] (UserRole, ...);` | | | Drop demo user | `drop demo user [if exists] 'name';` | `if exists` makes a cleanup script re-runnable | +| Update security | `update security [[in] Module];` | Re-syncs access rules with their domain model — Studio Pro's **Update security** button, headless. Repairs **CE0066** "Entity access is out of date", which a model authored elsewhere can carry (a module imported or updated outside Studio Pro). Not needed after mxcli's own writes: every write path reconciles as it writes. Writes nothing when the rules already match, and skips `System` | ## Workflows @@ -1334,7 +1335,8 @@ MDL uses explicit property declarations for pages: | Action binding | `action: type` | `actionbutton btn (caption: 'Save', action: save_changes)` | | Microflow action | `action: microflow Name(Param: val)` | `action: microflow Mod.ACT_Process(Order: $Order)` | | Button icon | `icon: 'Module.IconCollection.IconName'` | `linkbutton btn (caption: 'Edit', action: nothing, icon: 'Atlas_Core.Atlas_Filled.pencil')` — icon-collection icon; MxBuild rejects an unknown name (CE1613) | -| Clickable container | `onclick: action` (alias of `action:`) | `container card (onclick: microflow Mod.ACT_Open) { ... }` | +| Clickable container | `onclick: action` (alias of `action:`) | `container card (onclick: microflow Mod.ACT_Open) { ... }` — takes an argument list like a button: `action: nanoflow Mod.ACT_Ship($Order = $dgOrders)` | +| Action arguments | every parameter needs one | A flow action with an unfilled parameter is **CE1571**. An enclosing data container of its type supplies it; a data grid's **control bar** does not (not row-scoped) — pass the grid's selection, `$dgOrders` | | Database source | `datasource: database entity` | `datagrid dg (datasource: database Module.Entity)` | | Selection binding | `datasource: selection widget` | `dataview dv (datasource: selection galleryList)` | | Association source ("data from context") | `datasource: $currentObject/Module.Assoc` | nested `dataview dvCust (datasource: $currentObject/Order_Customer)` shows the to-one referenced object; a list widget shows the to-many collection | diff --git a/docs/11-proposals/PROPOSAL_soap_request_body.md b/docs/11-proposals/PROPOSAL_soap_request_body.md new file mode 100644 index 0000000000..4368f29a58 --- /dev/null +++ b/docs/11-proposals/PROPOSAL_soap_request_body.md @@ -0,0 +1,346 @@ +--- +title: The SOAP request body — the half of a web service call MDL cannot say +status: draft +date: 2026-09-11 +related: + - PROPOSAL_mapping_coverage.md + - PROPOSAL_first_class_expressions.md + - https://github.com/ako/TestApp + - docs/13-decisions/0003-mdl-is-sql-shaped.md + - docs/13-decisions/0005-semantic-model-interface-currency.md +--- + +# The SOAP request body + +> Pinned against `ako/TestApp` (Mendix 11.14.0), whose `Clients` module holds +> three Studio Pro-authored SOAP calls covering both request shapes — two with +> operation arguments, one with a send mapping. Every storage claim below is read +> off those documents rather than inferred from the metamodel, and every error +> number was produced by running `mx check` on a project mxcli wrote. + +## 1. Problem + +`CALL WEB SERVICE` can say who to call and what to do with the answer. It cannot +say **what to send**, and it does not admit that. + +Two clauses are involved, and they fail in opposite directions: + +**`send mapping` parses, is accepted, and is discarded.** The clause has been in +the grammar since SOAP support landed. Neither engine writes it. Measured — one +statement, `mxcli exec`, then `mx check` on a project whose baseline is 0 errors: + +``` +create or replace microflow Clients.MxcliSoapSend ($Order : Clients.Order) +begin + call web service Clients.OrderSoapClient + operation SaveOrder + send mapping Clients.SoapOrderExportMapping; + return; +end; +``` + +``` +Created microflow: Clients.MxcliSoapSend +[error] [CE0369] "Cannot use simple request body, as the operation's body is + complex" at Call web service activity 'Call web service 'SaveOrder'' +``` + +`SoapOrderExportMapping` appears **zero times** in the written document, on +`MXCLI_ENGINE=modelsdk` and on `legacy` alike. This is the #850 shape exactly: a +clause the parser accepts, the writer ignores, `exec` reports as success, and +mxbuild rejects several minutes later with an error naming neither the clause nor +the statement. + +**Operation arguments have no syntax at all**, so the failure is one step +earlier: there is nothing to write. A call to an operation that takes parameters +is refused by mxbuild as **CE0178**, and MDL offers the author no way to fix it. +That was the last unresolved error in the four-round chain recorded in +`.claude/skills/fix-issue/findings/mdl-executor.jsonl` (2026-09-10): + +``` +StorageLoadException → CE0386 → CE0243 + CE0366 → CE0178 +``` + +The first three are fixed. CE0178 is this proposal. + +The two are one problem, which is the point of proposing them together: **they +are the two branches of a single stored property**, and today a statement can ask +for both and get neither. + +### 1.1 Consequence: a SOAP call is currently unreadable as well as unwritable + +Related, and worth fixing in the same pass. All three of TestApp's SOAP actions +carry **15 keys**; `webServiceActionRequiresRawBSON` treats **9** as +representable, so anything else forces the opaque fallback. mxcli's own writer +emits the same 15. Measured on both engines: + +``` +$Orders = call web service raw 'KAYAAAUkSUQAEAAAAACTWjMVYM9yRZNCZQlJkWTA…' +``` + +Every SOAP call in every project — Studio Pro's and mxcli's — describes as +base64. The structured `call web service …` form the describer can produce is +**unreachable**, which is how five dead resolvers lived in it undetected +(bfb30c7c). Six of the seven unsupported keys are fixed boilerplate mxcli already +writes unconditionally; the seventh is `RequestBodyHandling`, i.e. this proposal. +Landing it is what makes the structured form reachable, and DESCRIBE→exec a real +copy operation for SOAP. + +## 2. What Mendix stores + +`Microflows$CallWebServiceAction.RequestBodyHandling` is a polymorphic child. +`generated/metamodel` declares six variants (`Advanced`, `Binary`, `Custom`, +`FormData`, `Mapping`, `Simple`); the two a SOAP call uses are below. **They are +alternatives** — a call has one request body, not two. + +### 2.1 Arguments — `Microflows$SimpleRequestHandling` + +From `Clients.GetOrders`: + +```json +{ + "$Type": "Microflows$SimpleRequestHandling", + "NullValueOption": "LeaveOutElement", + "ParameterMappings": [2, { + "$Type": "Microflows$WebServiceOperationSimpleParameterMapping", + "Argument": "2", + "IsChecked": true, + "ParameterName": "", + "ParameterPath": "http%3A//www.example.com/:GetOrder|OrderId" + }] +} +``` + +`Argument` is a **Mendix expression string**, not a literal — `"2"` here because +the reference call passes a constant, but `$Order/OrderId` belongs in the same +field. `ParameterMappings` is a typed array with marker **2**. + +`ParameterPath` is the load-bearing one, and it is **derivable**: + +``` +ParameterPath = escape(operation.RequestBodyElementName) + "|" + parameterName +``` + +The imported service document carries `RequestBodyElementName` per operation — +`"http://www.example.com/:GetOrder"` on `GetOrder` — in +`Description.Services[].Operations[]`, structured, alongside the `Name` the +CE0386 fix already reads. Escaping is per path segment: `:` becomes `%3A` inside +a segment, `/` is left alone, and the segment separators `:` and `|` are not +escaped. So the author supplies `OrderId` and mxcli builds the rest. + +That the path is derivable is what makes a readable syntax possible at all; the +alternative is making users paste `http%3A//www.example.com/:GetOrder|OrderId` +into a script, which fails principle 1 on sight. + +**The parameter names themselves are not in the model.** The operation document +stops at the body element name; `OrderId` lives in the WSDL's inline XSD, stored +as raw text in `Description.WsdlContentss[]`. mxcli does not parse WSDL and this +proposal does not add that. Consequence, stated plainly: a misspelled parameter +name cannot be caught by `mxcli check`, and reaches the author as CE0178 from +mxbuild. Section 6 says what could change that later. + +### 2.2 Send mapping — `Microflows$MappingRequestHandling` + +From `Clients.SaveOrder`: + +```json +{ + "$Type": "Microflows$MappingRequestHandling", + "ContentType": "Json", + "MappingId": "Clients.SoapOrderExportMapping", + "MappingVariableName": "NewSaveOrder" +} +``` + +Three facts, all consequential: + +1. **`MappingVariableName` is the missing half of today's clause.** An export + mapping maps *from an object*, and the call has to say which variable holds + it. The current `send mapping X` cannot express it — which is a second reason + the clause could not have been written correctly even if the writer tried. +2. **Both keys are already known-wrong in `modelsdk/gen`.** The key audit lists + them: `Mapping`→`MappingId` and `MappingArgumentVariableName`→ + `MappingVariableName` (`modelsdk/gen/keyaudit_test.go:168-169`). Writing + through the gen accessors produces a document mxbuild tolerates and Studio Pro + cannot open. Two `STORAGE-NAME OVERRIDE` patches in `init` **and** + `InitFromRaw`, per CLAUDE.md's two rules, and struck off the audit ledger. +3. **`ContentType` is `"Json"`** on this reference — on a SOAP call, whose + request is XML, and whose *receive* side writes `"Xml"`. Surprising enough to + be worth naming as a risk rather than a finding: it is one document, and the + honest move is to write what Studio Pro wrote and get a second reference + before treating it as the rule. See §6. + +## 3. Proposed syntax + +Reuse `callArgumentList` — the named-argument form every other call statement in +MDL already uses (`call microflow`, `call nanoflow`, `call java action`, +`call external action`, `execute database query`). No new spelling for a concept +MDL has spelled five times. + +**Arguments attach to `operation`**, because they are the operation's parameters +and `ParameterPath` is built from the operation: + +``` +$Orders = call web service Clients.OrderSoapClient + operation GetOrder (OrderId = $Customer/OrderId) + receive mapping Clients.SoapOrdersImportMapping; +``` + +**The send mapping gains the variable it maps from**, with `from` — the +preposition MDL already uses for a source: + +``` +call web service Clients.OrderSoapClient + operation SaveOrder + send mapping Clients.SoapOrderExportMapping from $NewSaveOrder; +``` + +Grammar delta, confined to one rule: + +```antlr +callWebServiceStatement + : (VARIABLE EQUALS)? CALL WEB SERVICE + (RAW STRING_LITERAL + | webServiceReference + (OPERATION webServiceReference (LPAREN callArgumentList? RPAREN)?)? + (SEND MAPPING webServiceReference (FROM VARIABLE)?)? + (RECEIVE MAPPING webServiceReference)? + (TIMEOUT expression)?) + onErrorClause? + ; +``` + +`FROM` and `LPAREN`/`RPAREN` are existing tokens; no lexer change. + +Against the design checklist: it reads as English (*call this service, operation +GetOrder with OrderId …*); it adds no verb; both clauses are optional, so every +script that parses today still parses; one argument is a one-line diff; and an +LLM that has seen `call microflow Mod.Flow (Name = $x)` generates this correctly +from the shape alone. + +### 3.1 The clauses are mutually exclusive — and that is the rule to enforce + +`RequestBodyHandling` holds one variant. So: + +| Statement | Written | +|---|---| +| `operation X (a = …)` | `SimpleRequestHandling` with parameter mappings | +| `send mapping M from $v` | `MappingRequestHandling` | +| `operation X` alone | `SimpleRequestHandling`, empty — today's behaviour | +| both | **refused by `mxcli check`** | + +The last row is the one worth having. Today both clauses can be written, neither +is stored, and the author learns something is wrong from CE0369 — an error about +a "simple request body" on a statement that asked for a mapping. The refusal +should name both clauses and say that a call sends either arguments or a mapping. + +Per the repo's rule, `check` and `exec` must call the **same** function, so a +script cannot pass one and fail the other. + +### 3.2 DESCRIBE + +Round-trippable, per the layouts precedent — describe → edit → exec is how a SOAP +call gets copied. Once `RequestBodyHandling` is representable, remove it from the +`webServiceActionRequiresRawBSON` unsupported set along with the six boilerplate +keys, and a real call describes as the readable form instead of base64 (§1.1). + +An argument whose `ParameterPath` does not decompose into a known operation's +body element name is the honest edge: emit the raw form for that action rather +than a structured line that would lose the path. Unknown-shape → raw, never +unknown-shape → dropped. + +## 4. Alternatives considered + +**`arguments (…)` as its own clause.** Symmetrical with `send mapping`, and +readable. Rejected: it introduces a second way to spell a call's arguments, when +five statements already use `(Name = value)`. Principle 2. + +**Expose `ParameterPath` verbatim.** Honest about storage, and would survive a +WSDL whose parameters mxcli cannot enumerate. Rejected as the primary form: it +puts a percent-encoded URI in a script aimed at business analysts. Worth +reconsidering only as an escape hatch if a real WSDL turns up whose paths are not +`element|parameter` — none of TestApp's three are, but three is not many. + +**Write the send mapping now and leave arguments for later.** Tempting, since the +mapping is four keys. Rejected because the mutual exclusivity in §3.1 is only +enforceable once both exist; implementing one alone means `check` can refuse a +combination it cannot yet offer an alternative to. + +**Parse the WSDL to validate parameter names.** See §6 — deliberately out of +scope, not rejected. + +## 5. Implementation sketch + +Roughly the shape the CE0386/CE0243 fixes took, and mostly reusing their parts. + +1. **Semantic model** (`sdk/microflows`): `WebServiceCallAction` gains + `Arguments []WebServiceArgument{Name, Expression}` and `SendMappingVariable + string`. Per ADR-0005 the backend interface speaks this, not gen or BSON. +2. **Operation lookup** (`mdl/executor/webservice_names.go`): one more reader + beside `resolveWebServiceName`, returning the chosen operation's + `RequestBodyElementName`. Same document, same walk, no new backend method — + and the same discipline: **`""` when it cannot be established, never a guess**, + since a fabricated path reproduces CE0178 with different text in it. +3. **Writers**: `RequestBodyHandling` stops being an unconditional + `simpleRequestHandlingToGen()` and branches. Both engines + (`mdl/backend/modelsdk/microflow_webservice_write.go`, + `sdk/mpr/writer_microflow_actions.go`), key-for-key in the same order. +4. **`MappingRequestHandling` needs the two storage-name overrides** from §2.2 + before it can be written at all — both sides, `gofmt`, ledger struck. +5. **Reader**: `RequestBodyHandling` → arguments or send mapping. Note the read + path currently looks for `RequestHandling` → `ExportMappingCall` → `Mapping`, + a key **no TestApp document carries**, which is why `SendMappingID` was never + populated from a real project either. +6. **Validation**: §3.1, one function, called by `check` and `exec`. +7. **Version gating**: none expected — SOAP calls predate the supported range — + but confirm against `sdk/versions/mendix-{9,10,11}.yaml` before merging. + +## 6. What this does not settle + +Listed because each is a place where the next person will otherwise assume the +question was answered. + +- **`ContentType` on the send mapping.** One reference says `"Json"` (§2.2). + Write that; get a second Studio Pro-authored send mapping before calling it a + rule. If a second says `"Xml"`, the property is probably following the + *mapping's* source, not the call's protocol. +- **`ParameterName`** is `""` in both reference parameter mappings, and gen + declares it beside `ParameterPath`. What fills it is unmeasured — plausibly + RPC-style bindings, which neither TestApp operation uses. Write `""`. +- **`IsChecked`** is `true` in both. Presumably Studio Pro's per-parameter + checkbox. There is no observed `false`, so no syntax is proposed for it; a + stored `false` must be preserved on rewrite rather than normalised to `true`. +- **`Microflows$WebServiceOperationAdvancedParameterMapping`** — a per-parameter + export mapping, a third shape between §2.1 and §2.2. No reference document. + Out of scope; a stored one must make the action refuse a rewrite + (guard-don't-drop), not silently become a simple mapping. +- **Nested parameters.** Both references pass one scalar. A WSDL with a nested + complex parameter may produce a `ParameterPath` with more segments, which would + change the derivation in §2.1. Unknown, and the reason §4 keeps the verbatim + form as a possible escape hatch. +- **Validating parameter names against the WSDL.** Would need the XSD in + `Description.WsdlContentss[]` parsed. Real value — it turns a CE0178 from + mxbuild into a `check` error naming the parameter — but it is a separate piece + of work with its own risk surface, and it is not needed for the write path to + be correct. + +## 7. Proving it + +The bar is the one the CE0386 chain set: **measured against a real project, with +a control.** + +- Unit, both engines: encode an action with arguments and one with a send + mapping; assert the key sets and order against §2.1/§2.2 verbatim. +- Unit: `ParameterPath` derivation, including the escaping — a test asserting + `http%3A//www.example.com/:GetOrder|OrderId` character for character, since a + plausible-looking wrong escaping is exactly what mxbuild would accept and + Studio Pro would not. +- Unit: the §3.1 refusal, and that `check` and `exec` reject the same script. +- `mdl-examples/doctype-tests/06b-soap-examples.mdl` extended with both forms. +- **Integration, against ako/TestApp**: rewrite `Clients.GetOrders` and + `Clients.SaveOrder` from MDL and get **0 errors** from `mx check`. The control + is the pair of numbers this proposal opened with — the same two statements + today give CE0178 and CE0369. A test that only passes against fixed code has + not been shown to detect anything. +- DESCRIBE round trip: describe both, re-exec into a copy, `mx check` clean — + which also demonstrates §1.1, since neither describes as `raw` any more. diff --git a/mdl-examples/bug-tests/controlbar-action-argument-ce1571.mdl b/mdl-examples/bug-tests/controlbar-action-argument-ce1571.mdl new file mode 100644 index 0000000000..fc358b287e --- /dev/null +++ b/mdl-examples/bug-tests/controlbar-action-argument-ce1571.mdl @@ -0,0 +1,95 @@ +-- mendixlabs/mxcli#1082 — a CONTAINER action calling a flow with a parameter. +-- +-- POSITIVE test (plain .mdl): this script must PASS `mxcli check`. The rule it +-- guards needs a project (it compares a flow's stored signature against the +-- arguments), so the negative case cannot be a .fail.mdl — `make check-mdl` +-- runs check with no project and would report a working rule as regressed +-- (#891, #892). The rejection is covered by unit tests in +-- mdl/executor/validate_flow_args_action_test.go and by the doctype gate. +-- +-- REPORTED SYMPTOM +-- +-- A container inside a DataGrid 2 control bar, with +-- +-- Action: nanoflow CustomModule.ACT_UnLink +-- +-- where the nanoflow declares a required entity parameter, built to +-- +-- [error] [CE1571] "No argument has been selected for parameter +-- 'LogisticWhitelist' and no default is available." at Container +-- 'containerUnlinkMat' +-- +-- …while `mxcli check --references` printed "Check passed!". +-- +-- WHAT THE REPORT GOT WRONG, AND WHAT WAS ACTUALLY BROKEN +-- +-- The report concluded that MDL had no syntax for passing an argument through +-- a container action ("It accepts only Action: nanoflow Module.NF"). It has: +-- `actionExprV3` carries `NANOFLOW qualifiedName microflowArgsV3?`, the +-- visitor reads the args, and buildContainerV3 hands the action to the same +-- buildClientActionV3 an actionbutton uses. Measured on a Mendix 11.12.0 app, +-- three pages in one script, one `mx check`: +-- +-- container, Action: nanoflow M.NF → CE1571 +-- container, Action: nanoflow M.NF ($P = $dgMaterials) → 0 errors +-- actionbutton, same binding → 0 errors +-- +-- What WAS broken is that mxcli never said so. The CE1571 argument check +-- walked a widget's data source only, so the identical fault on an action +-- passed check and failed the build. It now covers both. +-- +-- A control bar is also not row-scoped, which is why the argument is required +-- here and not on the column button below: measured, the same container is +-- CE1571 in the control bar and clean in a column. +-- +-- VERIFY IN STUDIO PRO +-- +-- exec this against an app, open the page, and check the container's +-- "On click" — the nanoflow's parameter is bound to the grid's selection. + +create module BugTest1082; + +create persistent entity BugTest1082.LogisticWhitelist ( + Name: String(200), + IsLinked: Boolean +); + +create nanoflow BugTest1082.ACT_UnLink + ($LogisticWhitelist : BugTest1082.LogisticWhitelist) + folder 'Actions' +begin + change $LogisticWhitelist (IsLinked = false) refresh; +end; + +create or replace page BugTest1082.MaterialsOverview +( + title: 'Material definitions', + layout: Atlas_Core.Atlas_Default, + url: 'bugtest1082_materials' +) +{ + datagrid dgMaterials ( + datasource: database from BugTest1082.LogisticWhitelist, + selection: single + ) { + column colName (attribute: Name, caption: 'Name') + + -- Row-scoped: the grid's row supplies the parameter, no argument needed. + column colRow (caption: 'Row') { + container cRowUnlink (action: nanoflow BugTest1082.ACT_UnLink, class: 'command') { + actionbutton btnRowUnlink (caption: 'Unlink') + } + } + + controlbar controlBar1 { + -- The reported shape, written correctly. Remove the argument list and + -- `mxcli check -p ` reports CE1571 before mxbuild ever runs. + container containerUnlinkMat ( + class: 'command', + action: nanoflow BugTest1082.ACT_UnLink ($LogisticWhitelist = $dgMaterials) + ) { + actionbutton btnUnlink (caption: 'Unlink', icon: 'Atlas_Core.Atlas_Filled.trash-can') + } + } + } +} diff --git a/mdl-examples/bug-tests/marketplace-1085-security-after-transplant.mdl b/mdl-examples/bug-tests/marketplace-1085-security-after-transplant.mdl new file mode 100644 index 0000000000..166aab7d45 --- /dev/null +++ b/mdl-examples/bug-tests/marketplace-1085-security-after-transplant.mdl @@ -0,0 +1,77 @@ +-- ============================================================================ +-- CE0066 left behind by `mxcli marketplace update` (mendixlabs/mxcli#1085) +-- ============================================================================ +-- +-- Symptom, reported against 0.21.0 on a Mendix 11.12.4 project: +-- +-- $ mxcli marketplace update 223053 --to 2.2.0 --no-baseline -p App.mpr +-- UserCommons updated 2.1.3 → 2.2.0 +-- 81 units copied, 45 element identities preserved, 6 role grant(s) restored. +-- $ mx check -p App.mpr +-- [error] [CE0066] "Entity access is out of date. Please update security by +-- clicking the 'Update security' button in the domain model editor." +-- at Domain model of module 'UserCommons' +-- +-- The update exits 0 and says nothing about it. Its "Next, repair what a +-- headless update leaves behind" block named CE0463 and CE6087 only, so the +-- error reads as needing Studio Pro — which is how it was reported: "no +-- headless fix available". +-- +-- Root cause: +-- A transplant is the one write path in mxcli that does not reconcile as it +-- writes. TransplantModule copies the incoming module's units in VERBATIM, +-- and RestoreRoleGrants runs its statements one at a time rather than as a +-- program, so the executor's finalize step — the one that calls +-- ReconcileMemberAccesses — never runs. An access rule that does not cover +-- every member of its entity therefore arrives exactly as the package +-- shipped it, and Mendix rejects that model with CE0066. +-- +-- Measured on the vendored 11.6.6 app (Administration 4.3.2, mxbuild +-- 11.14.0), removing ONE member entry from the incoming version's rules: +-- +-- | update transplants | mx check | +-- |-------------------------------------|---------------------------------| +-- | the module UNCHANGED (control)| 0 errors | +-- | rules missing one member | CE0066 at the domain model | +-- | the same, with this fix | 0 errors | +-- +-- The control is what localises it: the machinery does not CREATE the stale +-- rule, it transmits one. So the fix is to reconcile the module after it is +-- in place — PerformUpdate and PerformInstall both do now — and to report +-- the count, because repairing a marketplace module's rules is a change the +-- operator will see in `diff-local`. +-- +-- WHAT THIS EXAMPLE DOES NOT SHOW, and why. +-- It cannot reproduce the bug. Producing a stale access rule requires a +-- module arriving from outside mxcli; every MDL write path reconciles as it +-- writes, so no script can leave one behind. Reproduction lives in +-- cmd/mxcli/marketplace/security_test.go, which edits the stored BSON +-- directly to build the state a package can be in and a script cannot. +-- +-- What this script covers is the OTHER half of the contract: the repair is +-- idempotent. `update security` is what the update now runs for itself, and +-- on a project whose rules are complete it must change nothing and say so. +-- A reconcile that rewrites a healthy marketplace module on every install +-- would be read back by `marketplace diff` as a local edit — and a local +-- edit is what makes the NEXT update refuse. +-- +-- Verify (needs no fixture — it exercises the command against any project): +-- +-- mxcli exec marketplace-1085-security-after-transplant.mdl -p app.mpr +-- mxcli docker check -p app.mpr -- unchanged from before the run +-- +-- Expected output from both statements on a healthy project: +-- +-- All entity access rules are up to date +-- +-- Anything else means a module's rules did not cover their entities' members, +-- which is the CE0066 this fix exists to close. + +-- Project-wide: every module's rules against its own domain model. +update security; +/ + +-- Scoped to one module, which is what the update runs for the module it just +-- replaced. +update security Administration; +/ diff --git a/mdl-examples/bug-tests/wf-415-alter-outcome-activity-kind.mdl b/mdl-examples/bug-tests/wf-415-alter-outcome-activity-kind.mdl new file mode 100644 index 0000000000..4b4ac5d0be --- /dev/null +++ b/mdl-examples/bug-tests/wf-415-alter-outcome-activity-kind.mdl @@ -0,0 +1,75 @@ +-- ako/mxcli#415 — an ALTER WORKFLOW insert op must match the activity's kind. +-- +-- An activity's outcome list is TYPED, and each inserting op writes exactly one +-- outcome type into it: +-- +-- INSERT OUTCOME -> Workflows$UserTaskOutcome (user task only) +-- INSERT PATH -> Workflows$ParallelSplitOutcome (parallel split only) +-- INSERT CONDITION -> Workflows$…ConditionOutcome (decision / call microflow) +-- INSERT BOUNDARY EVENT -> a boundary event (user task, call microflow, +-- call workflow, wait for notification) +-- +-- Before the fix none of them checked what they were pointed at. Measured on +-- mxbuild 11.10.0 against this project sitting at 0 errors, each of these six +-- statements left a model Mendix cannot LOAD — `mx check` dies at "Loading the +-- mpr file" with a .NET exception, before validating anything, and Studio Pro +-- will not open the project: +-- +-- alter workflow WF415.WF insert outcome 'X' on decision9 { }; -- InvalidCastException +-- alter workflow WF415.WF insert outcome 'X' on split1 { }; -- InvalidCastException +-- alter workflow WF415.WF insert path on decision9 { }; -- InvalidOperationException +-- alter workflow WF415.WF insert path on task1 { }; -- InvalidOperationException +-- alter workflow WF415.WF insert condition 'WF415.Kind.Standard' on task1 { }; -- InvalidOperationException +-- alter workflow WF415.WF insert condition 'WF415.Kind.Standard' on split1 { }; -- InvalidOperationException +-- +-- They are now refused, by the one function `check --references` and `exec` +-- share, and the refusal names the op that fits the target. +-- +-- This file is NOT a .fail.mdl: the guard resolves the target against the +-- STORED workflow, so it needs a project, and `make check-mdl` runs `check` +-- with none. What follows is the correct form of each op — run it against a +-- project with `mxcli exec` and `mx check` should stay at 0 errors. + +create module WF415; + +create enumeration WF415.Kind (Standard 'Standard', Priority 'Priority'); + +create persistent entity WF415.Ctx ( + Kind: Enumeration(WF415.Kind) +); + +create page WF415.TaskPage + ( + Title: 'Task', + Layout: Atlas_Core.Atlas_Default, + Params: { $TaskPageParam: System.WorkflowUserTask } + ) + { + CONTAINER c1 { } + } + +create workflow WF415.WF parameter $WorkflowContext: WF415.Ctx +begin + user task task1 'Review' + page WF415.TaskPage + outcomes 'OK' { }; + + -- An enumeration decision carries one outcome per value plus the empty one + -- (MDL-WF06 / CE6686). + decision decision9 '$WorkflowContext/Kind' + outcomes + 'WF415.Kind.Standard' -> { } + 'WF415.Kind.Priority' -> { } + '' -> { } + ; + + parallel split split1 + path 1 { } + path 2 { } + ; +end workflow; + +-- Each op aimed at the activity kind whose list can hold what it writes. +alter workflow WF415.WF insert outcome 'Rejected' on task1 { }; +alter workflow WF415.WF insert path on split1 { }; +alter workflow WF415.WF insert boundary event on task1 timer '1h' { }; diff --git a/mdl-examples/bug-tests/xpath-unqualified-association-ok.mdl b/mdl-examples/bug-tests/xpath-unqualified-association-ok.mdl new file mode 100644 index 0000000000..fe9221648d --- /dev/null +++ b/mdl-examples/bug-tests/xpath-unqualified-association-ok.mdl @@ -0,0 +1,43 @@ +-- ============================================================================ +-- ako/ChipCoV1 FINDINGS.md — the qualified form, which must PASS +-- ============================================================================ +-- +-- The positive control for MDL-XPATH01. The rule rejects an association named +-- BARE in an XPath constraint; this file is the same model with it qualified, +-- and must PASS. Without a control like this the rule could degrade to "reject +-- every XPath constraint" and its negative tests would still look green. +-- +-- The NEGATIVE case is a Go test, not a .fail.mdl, because MDL-XPATH01 needs a +-- project: it fires only when it can tell an attribute from an association, and +-- `make check-mdl` runs `mxcli check` with no project. A .fail.mdl here would +-- pass silently and assert nothing. See mdl/executor/validate_xpath_association_test.go. +-- +-- Verified on Mendix 11.14.0: this form builds at BUILD SUCCEEDED where the +-- unqualified one fails with "Error(s) in XPath constraint". +-- ============================================================================ + +CREATE MODULE BugXPathAssocOK; + +CREATE OR MODIFY PERSISTENT ENTITY BugXPathAssocOK.Ticket ( Title: String(100) ); + +CREATE OR MODIFY PERSISTENT ENTITY BugXPathAssocOK.Equipment ( Code: String(50) ); + +CREATE OR MODIFY ASSOCIATION BugXPathAssocOK.Ticket_Equipment + FROM BugXPathAssocOK.Ticket TO BugXPathAssocOK.Equipment TYPE Reference; + +CREATE OR MODIFY MICROFLOW BugXPathAssocOK.ACT_ForEquipment ($Eq: BugXPathAssocOK.Equipment) +RETURNS list of BugXPathAssocOK.Ticket +BEGIN + retrieve $Tickets from BugXPathAssocOK.Ticket + where [BugXPathAssocOK.Ticket_Equipment = $Eq]; + return $Tickets; +END; + +-- An attribute stays bare — the rule must not "qualify" one. +CREATE OR MODIFY MICROFLOW BugXPathAssocOK.ACT_ByTitle () +RETURNS list of BugXPathAssocOK.Ticket +BEGIN + retrieve $Tickets from BugXPathAssocOK.Ticket + where [Title = 'urgent']; + return $Tickets; +END; diff --git a/mdl-examples/doctype-tests/03-page-examples.mdl b/mdl-examples/doctype-tests/03-page-examples.mdl index a2549d50e4..b1bfa17bf1 100644 --- a/mdl-examples/doctype-tests/03-page-examples.mdl +++ b/mdl-examples/doctype-tests/03-page-examples.mdl @@ -949,6 +949,75 @@ create page PgTest.P013d_Container_Clickable } } +/** + * A clickable container in a DATA GRID CONTROL BAR, calling a nanoflow that + * takes a parameter (mendixlabs/mxcli#1082). + * + * Three things meet here that nothing else in this gate exercised together, and + * the gap let a real defect through: + * + * 1. A container's action can take an ARGUMENT LIST, exactly like a button's. + * The issue was filed on the belief that it could not — `Action: nanoflow + * Module.NF` was assumed to be the only accepted form — so the page below is + * the counter-example, and `mx check` reporting 0 errors on it is the claim. + * + * 2. A control bar is NOT row-scoped. Measured on mxbuild 11.12.0 with six + * containers carrying one identical fault: inside a COLUMN the grid's row + * supplies the argument (no error), inside the CONTROL BAR it does not + * (CE1571). So the argument here is not decoration — drop the `($Product = + * $ProductGrid)` and `mx check` goes red. + * + * 3. The grid's SELECTION is what fills it. `$ProductGrid` is the selection of + * the widget named `ProductGrid`, which is why `selection: single` below is + * load-bearing rather than cosmetic. + * + * The column button beside it is the control for (2): same nanoflow, same + * missing-argument shape, and it needs no argument because a column is + * row-scoped. + */ +create nanoflow PgTest.ACT_ToggleProductActive + ($Product : PgTest.Product) + folder 'Actions' +begin + change $Product (IsActive = false) refresh; +end; + +create page PgTest.P013e_ControlBar_Container_Action +( + title: 'Control-bar container action', + layout: Atlas_Core.Atlas_Default, + url: 'p013e_controlbar_container_action', + folder: 'Container' +) +{ + datagrid ProductGrid ( + datasource: database from PgTest.Product, + selection: single + ) { + column colName (attribute: Name, caption: 'Product Name') + + -- ROW-SCOPED: the grid's row supplies the nanoflow's parameter, so this + -- one needs no argument. The control for the control-bar case below. + column colRowAction (caption: 'Row action') { + container cRowToggle (action: nanoflow PgTest.ACT_ToggleProductActive, class: 'command') { + actionbutton btnRowToggle (caption: 'Deactivate') + } + } + + controlbar controlBar1 { + -- NOT row-scoped: the argument is required, and it is the grid's + -- selection. Both spellings of an argument list are accepted; this is + -- the `$Param = value` one, matching how a microflow call is written. + container cToggleSelected ( + class: 'command', + action: nanoflow PgTest.ACT_ToggleProductActive ($Product = $ProductGrid) + ) { + actionbutton btnToggleSelected (caption: 'Deactivate selected') + } + } + } +} + -- MARK: DataView -- ============================================================================ diff --git a/mdl/backend/modelsdk/backend.go b/mdl/backend/modelsdk/backend.go index 43deb9b392..2bbdc73877 100644 --- a/mdl/backend/modelsdk/backend.go +++ b/mdl/backend/modelsdk/backend.go @@ -117,6 +117,25 @@ func (b *Backend) IsConnected() bool { return b.reader != nil } func (b *Backend) Path() string { return b.path } +// ContentsDir is the mprcontents/ directory of an MPR v2 project, empty for v1. +func (b *Backend) ContentsDir() string { + if b.reader == nil { + return "" + } + return b.reader.ContentsDir() +} + +// InvalidateCache drops the reader's unit cache. Nothing routes through the +// backend interface to reach it today, but the generated stub for a method with +// no results at all is a panic, so leaving it unimplemented parks a crash in the +// default engine against the day something does. +func (b *Backend) InvalidateCache() { + if b.reader == nil { + return + } + b.reader.InvalidateCache() +} + func (b *Backend) Version() types.MPRVersion { if b.reader == nil { return 0 diff --git a/mdl/backend/modelsdk/diff_local_support_test.go b/mdl/backend/modelsdk/diff_local_support_test.go new file mode 100644 index 0000000000..6a7f287741 --- /dev/null +++ b/mdl/backend/modelsdk/diff_local_support_test.go @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: Apache-2.0 + +// The two FullBackend methods `mxcli diff-local` needs from the engine, and +// which the modelsdk engine did not have (mendixlabs/mxcli#1080). Both return +// no error, so both failed silently rather than loudly: +// +// - ContentsDir() returned "" — read by diff-local as "not a v2 project", +// reported as "mprcontents directory not found" on a project whose +// mprcontents/ was populated. +// - ParseMicroflowFromRaw() returned nil — which the caller DOES guard, by +// substituting a "-- parse failed --" stub. The stub is a constant, so both +// sides of the diff got the same text and the microflow's diff came out +// EMPTY while the summary still counted the unit as modified. +package modelsdkbackend + +import ( + "os" + "strings" + "testing" + + "go.mongodb.org/mongo-driver/bson" +) + +// TestContentsDirResolvesForV2 pins the defect itself: a v2 project must report +// a real, existing directory. +func TestContentsDirResolvesForV2(t *testing.T) { + b := New() + if dir := b.ContentsDir(); dir != "" { + t.Errorf("ContentsDir() on an unconnected backend = %q, want empty", dir) + } + if err := b.Connect(fixture); err != nil { + t.Fatalf("Connect(%s): %v", fixture, err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + if b.Version() != 2 { + t.Fatalf("fixture is MPR v%d; this test needs a v2 project", b.Version()) + } + dir := b.ContentsDir() + if dir == "" { + t.Fatal("ContentsDir() = \"\" for a v2 project — diff-local reports " + + "\"mprcontents directory not found\" on exactly this") + } + if !strings.HasSuffix(dir, "mprcontents") { + t.Errorf("ContentsDir() = %q, want a path ending in mprcontents", dir) + } + if st, err := os.Stat(dir); err != nil || !st.IsDir() { + t.Errorf("ContentsDir() = %q, which is not a directory (err %v)", dir, err) + } +} + +// TestParseMicroflowFromRawDecodesRealUnit feeds the method what diff-local +// feeds it: a unit's raw BSON, unmarshalled to a map, with no unit to look up +// (the real caller's bytes come from `git show`, so they are not in storage at +// all). Asserting on the activities is the point — a microflow parsed down to +// its name only would still diff to nothing. +func TestParseMicroflowFromRawDecodesRealUnit(t *testing.T) { + b := New() + if err := b.Connect(fixture); err != nil { + t.Fatalf("Connect(%s): %v", fixture, err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + refs, err := b.reader.ListUnitsByType("Microflows$Microflow") + if err != nil { + t.Fatalf("ListUnitsByType: %v", err) + } + var picked map[string]any + var pickedName string + for _, ref := range refs { + var raw map[string]any + if err := bson.Unmarshal(ref.Contents, &raw); err != nil { + continue + } + if name, _ := raw["Name"].(string); name == "NewAccount" { + picked, pickedName = raw, name + break + } + } + if picked == nil { + t.Skip("fixture has no Administration.NewAccount microflow") + } + + mf := b.ParseMicroflowFromRaw(picked, "some-unit-id", "some-container") + if mf == nil { + t.Fatal("ParseMicroflowFromRaw returned nil — diff-local renders both " + + "sides as the same '-- parse failed --' stub and shows no diff at all") + } + if mf.Name != pickedName { + t.Errorf("Name = %q, want %q", mf.Name, pickedName) + } + if mf.ObjectCollection == nil || len(mf.ObjectCollection.Objects) == 0 { + t.Error("microflow decoded with no flow objects — a body-less microflow " + + "diffs to nothing, which is the same silent miss wearing a different hat") + } +} diff --git a/mdl/backend/modelsdk/microflow_parse_raw.go b/mdl/backend/modelsdk/microflow_parse_raw.go new file mode 100644 index 0000000000..3e3a713994 --- /dev/null +++ b/mdl/backend/modelsdk/microflow_parse_raw.go @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + bsonv1 "go.mongodb.org/mongo-driver/bson" + bsonv2 "go.mongodb.org/mongo-driver/v2/bson" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/codec" + genMf "github.com/mendixlabs/mxcli/modelsdk/gen/microflows" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// ParseMicroflowFromRaw builds a Microflow from an already-unmarshalled BSON +// document that did NOT come from this project's storage — diff-local hands it +// a historical unit read back with `git show`, so there is no unit to look up. +// +// The codec decodes bytes, not maps, so the map is re-marshalled first. That is +// safe here because the result is read-only (it is rendered as MDL and thrown +// away): nothing this produces is written back, so the field reordering a map +// round-trip causes cannot reach storage. +func (b *Backend) ParseMicroflowFromRaw(raw map[string]any, unitID, containerID model.ID) *microflows.Microflow { + data, err := bsonv1.Marshal(raw) + if err != nil { + return nil + } + elem, err := codec.NewDecoder(codec.DefaultRegistry).Decode(bsonv2.Raw(data)) + if err != nil { + return nil + } + mf, ok := elem.(*genMf.Microflow) + if !ok { + return nil + } + out := microflowFromGen(mf, containerID) + if out != nil && unitID != "" { + out.ID = unitID + } + return out +} diff --git a/mdl/backend/modelsdk/microflow_webservice_write.go b/mdl/backend/modelsdk/microflow_webservice_write.go index 2b3d80922a..26d597ee2b 100644 --- a/mdl/backend/modelsdk/microflow_webservice_write.go +++ b/mdl/backend/modelsdk/microflow_webservice_write.go @@ -30,12 +30,51 @@ import ( // dependency that keeps the legacy engine alive. // // The target shape is byte-parity with the legacy serializer -// (sdk/mpr.serializeWebServiceCallAction), not an independent reading of the -// metamodel. There is no Studio Pro-authored SOAP document in this repo to pin -// against, so legacy's output is the only reference that exists — and it is what -// users' projects already contain. TestWebServiceCallAction_MatchesLegacyBSON -// holds the two engines together; a discrepancy is a test failure, not a silent -// divergence. +// (sdk/mpr.serializeWebServiceCallAction), so that fixing the silent drop +// changes nothing else. TestWebServiceCallAction_MatchesLegacyDocument holds the +// two engines together; a discrepancy is a test failure, not a silent divergence. +// +// PARITY WITH LEGACY IS NOT FIDELITY TO STUDIO PRO, and the difference is now +// measured rather than assumed. This file first claimed no Studio Pro-authored +// SOAP document existed to pin against; one does — ako/TestApp carries three +// (Clients.GetOrders / GetCustomerOrders / SaveOrder, Mendix 11.14.0), and +// against them legacy was wrong in six places. FOUR are now fixed in both +// engines, each verified by writing a call against TestApp's real service and +// running mx check — the errors came one at a time, each hidden by the one +// before it (StorageLoadException -> CE0386 -> CE0243+CE0366 -> CE0178): +// +// - ServiceName is the WSDL SERVICE name ("OrdersWS"), not the local part of +// the imported service's qualified name ("OrderSoapClient"). Deriving it +// gave CE0386 "Operation 'GetOrder' does not exist in consumed web service". +// Now resolved from the document by the executor (resolveWebServiceName), +// with the old derivation kept as a fallback. +// - ImportMappingCall.ContentType is "Xml", not the hardcoded "Json". +// - ReturnValueMapping is the mapping's QUALIFIED NAME. The executor used to +// resolve it to the mapping's unit `$ID`, which made the project impossible +// to LOAD — mx check stopped at a StorageLoadException before validation. +// That one only fired when the reference RESOLVED, so the only SOAP fixture +// (whose mappings deliberately do not exist) never triggered it. +// - VariableType is the result's REAL type, read off the receive mapping's +// root ObjectMappingElement (resolveImportMappingEntity). DataTypes$VoidType +// said the call returns nothing, which contradicted the mapping (CE0243) and +// made assigning the result its own error (CE0366). VoidType stays the +// fallback for a mapping that cannot be resolved. +// +// Two remain: +// +// - Range.SingleObject follows the operation's cardinality; both reference +// calls write false where both engines write true. No error has been +// measured from it, so it is left until one is — the reference roots carry +// MaxOccurs 1 while the calls carry SingleObject false, so it is NOT simply +// the mapping's cardinality and would be a guess today. +// - Operation ARGUMENTS are Microflows$WebServiceOperationSimpleParameterMapping +// entries inside RequestBodyHandling.ParameterMappings, keyed by an escaped +// ParameterPath ("http%3A//www.example.com/:GetOrder|OrderId" — the +// operation's RequestBodyElementName, escaped, plus "|" plus the parameter). +// Writing that list empty gives CE0178 "Body parameter mapping needs to be +// refreshed" — now the ONLY error left on a real call. It needs MDL SYNTAX +// before it can be written at all: callWebServiceStatement has no argument +// list, so there is nothing to serialize yet. // // Two shapes are deliberately NOT re-derived here: // @@ -49,12 +88,10 @@ import ( // registered as 2 in microflow_write.go and as 3 in odata_write.go, and // which one wins is decided by file order.) // - RequestBodyHandling is always SimpleRequestHandling, even when the -// statement carries a SEND MAPPING. Legacy does the same and says why: the -// advanced form needs a Studio Pro-generated example to establish its type -// storage name. Writing a guessed $Type is the failure mode that makes a -// project impossible to OPEN rather than merely invalid, so the send mapping -// stays unwritten here exactly as it does on legacy. `call web service raw` -// is the escape hatch for operations that need it. +// statement carries a SEND MAPPING — matching legacy, and WRONG: the real +// type is Microflows$MappingRequestHandling (see above). Until that is +// implemented the send mapping is silently dropped on both engines, and +// `call web service raw` is the only way to author one. // // Every null the document carries is written IN KEY POSITION rather than through // NullFields, for the same reason and with the same consequence — see addNull. @@ -88,7 +125,7 @@ func webServiceCallActionToGen(a *microflows.WebServiceCallAction) element.Eleme addPart(g, "RequestBodyHandling", simpleRequestHandlingToGen()) addPart(g, "RequestHeaderHandling", simpleRequestHandlingToGen()) addStr(g, "RequestProxyType", "DefaultProxy") - addStr(g, "ServiceName", webServiceLocalName(string(a.ServiceID))) + addStr(g, "ServiceName", webServiceName(a)) addStr(g, "TimeOutExpression", orDefault(a.TimeoutExpression, "300")) addBool(g, "UseRequestTimeOut", true) return g @@ -123,7 +160,10 @@ func webServiceResultHandlingToGen(a *microflows.WebServiceCallAction) element.E if a.ReceiveMappingID != "" { imc := newElem("Microflows$ImportMappingCall", "") addStr(imc, "Commit", "YesWithoutEvents") - addStr(imc, "ContentType", "Json") + // Xml, not Json: a SOAP response IS XML, and Studio Pro writes "Xml" in + // both reference calls that carry an import mapping (ako/TestApp, + // Clients.GetOrders and GetCustomerOrders, 11.14.0). + addStr(imc, "ContentType", "Xml") addBool(imc, "ForceSingleOccurrence", false) addStr(imc, "ObjectHandlingBackup", "Create") addStr(imc, "ParameterVariableName", "") @@ -139,10 +179,26 @@ func webServiceResultHandlingToGen(a *microflows.WebServiceCallAction) element.E } addStr(rh, "ResultVariableName", a.OutputVariable) - addPart(rh, "VariableType", newElem("DataTypes$VoidType", "")) + addPart(rh, "VariableType", webServiceVariableType(a)) return rh } +// webServiceVariableType is the type of the value the call returns: the entity +// the receive mapping produces. VoidType — what both engines wrote +// unconditionally — says the call returns NOTHING, which contradicts the mapping +// (CE0243 "the mapping used to return a value of type 'Nothing'") and makes +// assigning the result an error in its own right (CE0366 "cannot store in +// variable when there is no return value"). Kept as the fallback for a mapping +// that could not be resolved. +func webServiceVariableType(a *microflows.WebServiceCallAction) element.Element { + if a.ResultEntity == "" { + return newElem("DataTypes$VoidType", "") + } + vt := newElem("DataTypes$ObjectType", "") + addStr(vt, "Entity", a.ResultEntity) + return vt +} + // simpleRequestHandlingToGen builds the Microflows$SimpleRequestHandling used for // both the body and the header handling. func simpleRequestHandlingToGen() element.Element { @@ -152,9 +208,21 @@ func simpleRequestHandlingToGen() element.Element { return rh } -// webServiceLocalName is the service's local name — the part after the last dot -// of the qualified name. Mendix stores both: ImportedService is qualified, -// ServiceName is not. +// webServiceName is the WSDL Mendix resolves the operation +// within. The executor reads it off the imported service document; the fallback +// below is what BOTH engines used to do unconditionally, and it is right only +// when the document happens to be named after the service — otherwise Mendix +// reports CE0386 "Operation … does not exist in consumed web service …". Kept as +// a fallback rather than an error because it is what ships today, and a call +// against an unresolvable service is no worse than before. +func webServiceName(a *microflows.WebServiceCallAction) string { + if a.ServiceName != "" { + return a.ServiceName + } + return webServiceLocalName(string(a.ServiceID)) +} + +// webServiceLocalName is the part after the last dot of a qualified name. func webServiceLocalName(qualified string) string { if i := strings.LastIndex(qualified, "."); i >= 0 { return qualified[i+1:] diff --git a/mdl/backend/modelsdk/microflow_webservice_write_test.go b/mdl/backend/modelsdk/microflow_webservice_write_test.go index 639853dd69..2a0aa2239a 100644 --- a/mdl/backend/modelsdk/microflow_webservice_write_test.go +++ b/mdl/backend/modelsdk/microflow_webservice_write_test.go @@ -62,11 +62,27 @@ func TestWebServiceCallAction_IsWritten(t *testing.T) { // TestWebServiceCallAction_MatchesLegacyDocument pins the whole document against // the shape the legacy serializer writes. // -// Legacy is the reference on purpose: there is no Studio Pro-authored SOAP -// document in this repo, and legacy's output is both the documented fallback and -// what users' projects already contain. The values below were read off a real -// legacy-written project (`mxcli bson dump`, Mendix 11.13.0), not off the -// serializer's source. +// Legacy is the reference for this test on purpose — the change it guards is +// "stop dropping the action", so reproducing what ships is what makes it safe. +// The values below were read off a real legacy-written project (`mxcli bson +// dump`, Mendix 11.13.0), not off the serializer's source. +// +// It is NOT a fidelity test, and the difference matters for reading the +// expectations below. Studio Pro-authored SOAP documents exist (ako/TestApp, +// 11.14.0); measured against them, four of legacy's six divergences have since +// been fixed in BOTH engines, so most of this test now agrees with Studio Pro +// too. The exceptions are deliberate: +// +// - ServiceName "OrderService" here is the FALLBACK, exercised because this +// action carries no resolved ServiceName. The resolved path has its own +// test (TestWebServiceCallAction_ServiceNameIsTheWsdlService). +// - Range.SingleObject and the send-mapping request handling are the two +// divergences still open; the header comment in +// microflow_webservice_write.go says what each one costs and why it has not +// been changed on a guess. +// +// When those are fixed these expectations change with them, which is the point +// of recording which reference each one came from. func TestWebServiceCallAction_MatchesLegacyDocument(t *testing.T) { doc := encodeMicroflowAction(t, fullWebServiceCall()) @@ -149,6 +165,12 @@ func TestWebServiceCallAction_ResultHandlingBindsTheReceiveMapping(t *testing.T) if got := docGet(imc, "ReturnValueMapping"); got != "SampleSOAP.OrderResponse" { t.Errorf("ReturnValueMapping = %#v, want the qualified mapping name", got) } + // Xml, not Json. A SOAP response is XML, and Studio Pro writes "Xml" in both + // reference calls carrying an import mapping (ako/TestApp, 11.14.0). Legacy + // hardcoded "Json"; both engines now write Xml. + if got := docGet(imc, "ContentType"); got != "Xml" { + t.Errorf("ContentType = %#v, want Xml", got) + } rng, ok := docGet(imc, "Range").(bsonv1.D) if !ok { t.Fatalf("Range = %#v, want a document", docGet(imc, "Range")) @@ -158,6 +180,60 @@ func TestWebServiceCallAction_ResultHandlingBindsTheReceiveMapping(t *testing.T) } } +// TestWebServiceCallAction_ServiceNameIsTheWsdlService — ServiceName is the +// WSDL , resolved by the executor off the imported service +// document, NOT the local part of the qualified document name. Writing the +// derived name made Mendix look for the operation in a service that does not +// exist: CE0386, measured on 11.14.0 against ako/TestApp. +func TestWebServiceCallAction_ServiceNameIsTheWsdlService(t *testing.T) { + a := fullWebServiceCall() + a.ServiceName = "OrdersWS" + + if got := docGet(encodeMicroflowAction(t, a), "ServiceName"); got != "OrdersWS" { + t.Errorf("ServiceName = %#v, want the resolved WSDL service name", got) + } + + // Control: unresolved, the writer falls back to the derivation that ships + // today rather than writing nothing. A call against a service mxcli cannot + // resolve is then no worse off than before. + a.ServiceName = "" + if got := docGet(encodeMicroflowAction(t, a), "ServiceName"); got != "OrderService" { + t.Errorf("fallback ServiceName = %#v, want the derived OrderService", got) + } +} + +// TestWebServiceCallAction_VariableTypeIsTheMappingsEntity — the result's type +// is the entity the receive mapping produces. VoidType says the call returns +// nothing: CE0243 and CE0366, measured on 11.14.0 against ako/TestApp. +func TestWebServiceCallAction_VariableTypeIsTheMappingsEntity(t *testing.T) { + a := fullWebServiceCall() + a.ResultEntity = "Clients.Order" + + rh, ok := docGet(encodeMicroflowAction(t, a), "NewResultHandling").(bsonv1.D) + if !ok { + t.Fatal("NewResultHandling missing") + } + vt, ok := docGet(rh, "VariableType").(bsonv1.D) + if !ok { + t.Fatalf("VariableType = %#v, want a document", docGet(rh, "VariableType")) + } + if got := docGet(vt, "$Type"); got != "DataTypes$ObjectType" { + t.Errorf("VariableType.$Type = %#v, want DataTypes$ObjectType", got) + } + if got := docGet(vt, "Entity"); got != "Clients.Order" { + t.Errorf("VariableType.Entity = %#v, want Clients.Order", got) + } + + // Control: unresolved, it stays VoidType — wrong, but what ships, so an + // unresolvable mapping is no worse off than before. + a.ResultEntity = "" + rh2, _ := docGet(encodeMicroflowAction(t, a), "NewResultHandling").(bsonv1.D) + vt2, _ := docGet(rh2, "VariableType").(bsonv1.D) + if got := docGet(vt2, "$Type"); got != "DataTypes$VoidType" { + t.Errorf("fallback VariableType.$Type = %#v, want DataTypes$VoidType", got) + } +} + // TestWebServiceCallAction_NoOutputVariable — a call that binds nothing writes // Bind false and an explicitly null ImportMappingCall, rather than omitting the // result handling. diff --git a/mdl/backend/modelsdk/unimplemented_silent_test.go b/mdl/backend/modelsdk/unimplemented_silent_test.go new file mode 100644 index 0000000000..c24b6ca606 --- /dev/null +++ b/mdl/backend/modelsdk/unimplemented_silent_test.go @@ -0,0 +1,112 @@ +// SPDX-License-Identifier: Apache-2.0 + +// gen_unimplemented.go promises that a method the modelsdk Backend does not +// override "fails loudly rather than silently dropping data". That promise is +// conditional: the generated stub reports failure by returning +// errUnimplemented, so it only holds for methods that HAVE an error to fail +// through. A method returning a bare value gets `var r0 T; return r0`. +// +// mendixlabs/mxcli#1080 is what that costs. ContentsDir() returns a plain +// string, was never overridden, and the zero value is not nonsense — "" is the +// in-band answer for "MPR v1, no mprcontents/". So `mxcli diff-local` on the +// default engine reported "mprcontents directory not found" for a v2 project +// whose directory was right there, and the missing implementation was +// indistinguishable from a v1 project. +package modelsdkbackend + +import ( + "go/ast" + "go/parser" + "go/token" + "os" + "path/filepath" + "reflect" + "sort" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend" +) + +// methodsDeclaredOnBackend parses the package's own sources and returns the +// names of every method declared on Backend. Reading the source is the point: +// a promoted method and a declared one are indistinguishable through reflection +// (Go synthesises a wrapper named (*Backend).X for both), so runtime.FuncForPC +// cannot tell an override from the embedded stub. +func methodsDeclaredOnBackend(t *testing.T) map[string]bool { + t.Helper() + entries, err := os.ReadDir(".") + if err != nil { + t.Fatalf("read package dir: %v", err) + } + out := map[string]bool{} + for _, e := range entries { + name := e.Name() + if e.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") { + continue + } + f, err := parser.ParseFile(token.NewFileSet(), filepath.Clean(name), nil, 0) + if err != nil { + t.Fatalf("parse %s: %v", name, err) + } + for _, d := range f.Decls { + fd, ok := d.(*ast.FuncDecl) + if !ok || fd.Recv == nil || len(fd.Recv.List) != 1 { + continue + } + star, ok := fd.Recv.List[0].Type.(*ast.StarExpr) + if !ok { + continue + } + if id, ok := star.X.(*ast.Ident); ok && id.Name == "Backend" { + out[fd.Name.Name] = true + } + } + } + if len(out) == 0 { + t.Fatal("found no methods on *Backend — the guard would pass vacuously") + } + return out +} + +// TestErrorlessBackendMethodsAreImplemented is the guard for the whole class: +// every FullBackend method that cannot report failure must be implemented by +// hand, because for those the generated stub is silent by construction. +// +// A method listed here is a decision, not an omission — record why. +func TestErrorlessBackendMethodsAreImplemented(t *testing.T) { + exempt := map[string]string{} + + declared := methodsDeclaredOnBackend(t) + iface := reflect.TypeOf((*backend.FullBackend)(nil)).Elem() + + var errorless, missing []string + errType := reflect.TypeOf((*error)(nil)).Elem() + for i := 0; i < iface.NumMethod(); i++ { + m := iface.Method(i) + hasErr := false + for j := 0; j < m.Type.NumOut(); j++ { + if m.Type.Out(j) == errType { + hasErr = true + break + } + } + if hasErr { + continue + } + errorless = append(errorless, m.Name) + if !declared[m.Name] && exempt[m.Name] == "" { + missing = append(missing, m.Name) + } + } + if len(errorless) == 0 { + t.Fatal("no errorless methods found on FullBackend — the guard would pass vacuously") + } + sort.Strings(missing) + if len(missing) > 0 { + t.Errorf("these FullBackend methods return no error, so the generated stub\n"+ + "returns a zero value silently, and *Backend does not override them:\n %s\n"+ + "Implement each one, or add it to `exempt` with the reason the zero value is correct.", + strings.Join(missing, "\n ")) + } +} diff --git a/mdl/executor/alter_workflow_outcome_kind_test.go b/mdl/executor/alter_workflow_outcome_kind_test.go new file mode 100644 index 0000000000..46f7c93fe6 --- /dev/null +++ b/mdl/executor/alter_workflow_outcome_kind_test.go @@ -0,0 +1,192 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/workflows" +) + +// ako/mxcli#415. Three ALTER WORKFLOW ops each write one hard-coded outcome +// type into the target activity's Outcomes list: +// +// INSERT OUTCOME -> Workflows$UserTaskOutcome +// INSERT PATH -> Workflows$ParallelSplitOutcome +// INSERT CONDITION -> Workflows$*ConditionOutcome +// +// The lists are not interchangeable — the metamodel types them per activity +// (UserTask/MultiUserTask hold UserTaskOutcome, Decision/CallMicroflowTask hold +// ConditionOutcome, ParallelSplit holds ParallelSplitOutcome) — and none of the +// three checked what it was pointed at. Measured on mxbuild 11.10.0, every +// mismatch leaves a project that cannot be LOADED: `mx check` dies at "Loading +// the mpr file" with a .NET exception, before it validates anything, so Studio +// Pro will not open the project either. + +// wfKindFixture returns a mock backend serving one stored workflow with a user +// task, a decision, a parallel split and a call microflow — one of every +// activity kind these ops can legitimately or illegitimately target. +func wfKindFixture(t *testing.T) (*ExecContext, *mock.MockBackend) { + t.Helper() + mod := mkModule("Sales") + wf := mkWorkflow(mod.ID, "WF") + wf.Flow = &workflows.Flow{ + Activities: []workflows.WorkflowActivity{ + &workflows.UserTask{BaseWorkflowActivity: workflows.BaseWorkflowActivity{ + BaseElement: model.BaseElement{ID: "a1"}, Name: "task1"}}, + &workflows.ExclusiveSplitActivity{BaseWorkflowActivity: workflows.BaseWorkflowActivity{ + BaseElement: model.BaseElement{ID: "a2"}, Name: "decision9"}}, + &workflows.ParallelSplitActivity{BaseWorkflowActivity: workflows.BaseWorkflowActivity{ + BaseElement: model.BaseElement{ID: "a3"}, Name: "split1"}}, + &workflows.CallMicroflowTask{BaseWorkflowActivity: workflows.BaseWorkflowActivity{ + BaseElement: model.BaseElement{ID: "a4"}, Name: "callMicroflow1"}}, + }, + } + + h := mkHierarchy(mod) + withContainer(h, wf.ContainerID, mod.ID) + + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListWorkflowsFunc: func() ([]*workflows.Workflow, error) { return []*workflows.Workflow{wf}, nil }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + return ctx, mb +} + +// alterWfRefErrors runs the guard both passes share. +func alterWfRefErrors(t *testing.T, ctx *ExecContext, src string) []string { + t.Helper() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse %q: %v", src, errs) + } + stmt, ok := prog.Statements[0].(*ast.AlterWorkflowStmt) + if !ok { + t.Fatalf("not an ALTER WORKFLOW statement: %T", prog.Statements[0]) + } + return validateAlterWorkflowRefs(ctx, stmt, nil) +} + +func TestAlterWorkflow_OutcomeOpOnWrongActivityKindIsRefused(t *testing.T) { + for _, tc := range []struct { + name string + src string + refused bool + }{ + // INSERT OUTCOME writes a UserTaskOutcome. + {"insert outcome on a decision", `alter workflow Sales.WF insert outcome 'X' on decision9 { };`, true}, + {"insert outcome on a parallel split", `alter workflow Sales.WF insert outcome 'X' on split1 { };`, true}, + {"insert outcome on a call microflow", `alter workflow Sales.WF insert outcome 'X' on callMicroflow1 { };`, true}, + {"insert outcome on a user task", `alter workflow Sales.WF insert outcome 'X' on task1 { };`, false}, + + // INSERT PATH writes a ParallelSplitOutcome. + {"insert path on a decision", `alter workflow Sales.WF insert path on decision9 { };`, true}, + {"insert path on a user task", `alter workflow Sales.WF insert path on task1 { };`, true}, + {"insert path on a parallel split", `alter workflow Sales.WF insert path on split1 { };`, false}, + + // INSERT CONDITION writes a ConditionOutcome. + {"insert condition on a user task", `alter workflow Sales.WF insert condition 'Sales.K.A' on task1 { };`, true}, + {"insert condition on a parallel split", `alter workflow Sales.WF insert condition 'Sales.K.A' on split1 { };`, true}, + {"insert condition on a decision", `alter workflow Sales.WF insert condition 'Sales.K.A' on decision9 { };`, false}, + {"insert condition on a call microflow", `alter workflow Sales.WF insert condition 'Sales.K.A' on callMicroflow1 { };`, false}, + + // A boundary event is a separate list, and only four activity kinds + // declare one. On a decision the write was silently dropped — mxcli + // printed "Altered workflow" and the document was byte-identical. + {"boundary event on a decision", `alter workflow Sales.WF insert boundary event on decision9 timer '1h' { };`, true}, + {"boundary event on a parallel split", `alter workflow Sales.WF insert boundary event on split1 timer '1h' { };`, true}, + {"boundary event on a user task", `alter workflow Sales.WF insert boundary event on task1 timer '1h' { };`, false}, + {"boundary event on a call microflow", `alter workflow Sales.WF insert boundary event on callMicroflow1 timer '1h' { };`, false}, + } { + t.Run(tc.name, func(t *testing.T) { + ctx, _ := wfKindFixture(t) + errs := alterWfRefErrors(t, ctx, tc.src) + got := false + for _, e := range errs { + if strings.Contains(e, "cannot hold") { + got = true + } + } + if got != tc.refused { + t.Fatalf("refused = %v, want %v (errors: %v)", got, tc.refused, errs) + } + }) + } +} + +// The message has to name the op that IS right for the target, or the author is +// told no without being told what to write instead. +func TestAlterWorkflow_RefusalNamesTheRightOp(t *testing.T) { + ctx, _ := wfKindFixture(t) + errs := alterWfRefErrors(t, ctx, `alter workflow Sales.WF insert outcome 'X' on decision9 { };`) + if len(errs) == 0 { + t.Fatal("expected a refusal") + } + joined := strings.Join(errs, "\n") + for _, want := range []string{"INSERT CONDITION", "decision"} { + if !strings.Contains(joined, want) { + t.Errorf("refusal does not mention %q: %s", want, joined) + } + } +} + +// An unresolvable or ambiguous reference must NOT be turned into a refusal by +// this guard — the mutator already reports those, with better information. A +// guard that refused on a failed lookup would reject valid scripts whose +// activity is addressed in a way only the stored document can settle. +func TestAlterWorkflow_UnknownActivityIsLeftToTheMutator(t *testing.T) { + ctx, _ := wfKindFixture(t) + errs := alterWfRefErrors(t, ctx, `alter workflow Sales.WF insert outcome 'X' on noSuchActivity { };`) + for _, e := range errs { + if strings.Contains(e, "cannot hold") { + t.Fatalf("guard fired on an activity it could not resolve: %v", errs) + } + } +} + +// The load-bearing assertion: exec must not reach the mutator. A refusal that +// still wrote would leave exactly the corrupt project the guard exists to +// prevent — and exec is reachable without `check` ever running. +func TestAlterWorkflow_RefusedOpNeverReachesTheMutator(t *testing.T) { + mod := mkModule("Sales") + wf := mkWorkflow(mod.ID, "WF") + wf.Flow = &workflows.Flow{Activities: []workflows.WorkflowActivity{ + &workflows.ExclusiveSplitActivity{BaseWorkflowActivity: workflows.BaseWorkflowActivity{ + BaseElement: model.BaseElement{ID: "a2"}, Name: "decision9"}}, + }} + h := mkHierarchy(mod) + withContainer(h, wf.ContainerID, mod.ID) + + inserted := false + mut := &mock.MockWorkflowMutator{ + InsertOutcomeFunc: func(string, int, string, []workflows.WorkflowActivity) error { + inserted = true + return nil + }, + } + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListWorkflowsFunc: func() ([]*workflows.Workflow, error) { return []*workflows.Workflow{wf}, nil }, + OpenWorkflowForMutationFunc: func(model.ID) (backend.WorkflowMutator, error) { return mut, nil }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + prog, perrs := visitor.Build(`alter workflow Sales.WF insert outcome 'X' on decision9 { };`) + if len(perrs) > 0 { + t.Fatalf("parse: %v", perrs) + } + err := execAlterWorkflow(ctx, prog.Statements[0].(*ast.AlterWorkflowStmt)) + if err == nil { + t.Fatal("exec accepted an INSERT OUTCOME on a decision") + } + if inserted { + t.Fatal("exec wrote the outcome anyway — the project would be unloadable") + } +} diff --git a/mdl/executor/cmd_diff_local.go b/mdl/executor/cmd_diff_local.go index b8f53f9230..f60bb8025a 100644 --- a/mdl/executor/cmd_diff_local.go +++ b/mdl/executor/cmd_diff_local.go @@ -437,8 +437,16 @@ func attributeBsonToMDL(_ *ExecContext, raw map[string]any) string { } } - // Check for type object which contains the actual type - if typeObj, ok := raw["Type"].(map[string]any); ok { + // Check for the type object, which carries the actual type. Mendix stores + // it under the STORAGE name "NewType"; "Type" is the SDK name and is not + // a key any real document has. Reading only "Type" left every attribute + // rendering as "Unknown", so a String(200) -> String(50) change diffed to + // nothing at all while the summary still counted the unit as modified. + typeObj, ok := raw["NewType"].(map[string]any) + if !ok { + typeObj, ok = raw["Type"].(map[string]any) + } + if ok { if typeType := extractString(typeObj["$Type"]); typeType != "" { switch { case strings.Contains(typeType, "StringAttributeType"): diff --git a/mdl/executor/cmd_diff_local_attrtype_test.go b/mdl/executor/cmd_diff_local_attrtype_test.go new file mode 100644 index 0000000000..2365b493cc --- /dev/null +++ b/mdl/executor/cmd_diff_local_attrtype_test.go @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: Apache-2.0 + +// attributeBsonToMDL read the attribute's type object from raw["Type"]. That is +// the SDK name; Mendix stores it under "NewType" (the storage-name split in +// CLAUDE.md, per property rather than per $Type). No real document carries a +// "Type" key, so the lookup never hit and EVERY attribute rendered as +// "Unknown" — on both engines, since this renderer does not touch the backend. +// +// The cost is not the ugly word. It is that "Unknown" == "Unknown": narrowing +// FullName from String(200) to String(50) produced "1 modified" and an empty +// diff, because both sides rendered identically. Found alongside +// mendixlabs/mxcli#1080. +package executor + +import ( + "strings" + "testing" +) + +func attrRaw(name string, typeObj map[string]any) map[string]any { + return map[string]any{ + "$Type": "DomainModels$Attribute", + "Name": name, + "NewType": typeObj, + } +} + +func TestAttributeBsonToMDL_ReadsStorageNameNewType(t *testing.T) { + cases := []struct { + name string + typeObj map[string]any + want string + }{ + {"string with length", map[string]any{"$Type": "DomainModels$StringAttributeType", "Length": int32(200)}, "Name: String(200)"}, + {"unbounded string", map[string]any{"$Type": "DomainModels$StringAttributeType", "Length": int32(0)}, "Name: String"}, + {"boolean", map[string]any{"$Type": "DomainModels$BooleanAttributeType"}, "Name: Boolean"}, + {"integer", map[string]any{"$Type": "DomainModels$IntegerAttributeType"}, "Name: Integer"}, + {"enumeration", map[string]any{"$Type": "DomainModels$EnumerationAttributeType"}, "Name: Enumeration"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := attributeBsonToMDL(nil, attrRaw("Name", tc.typeObj)) + if got != tc.want { + t.Errorf("attributeBsonToMDL = %q, want %q", got, tc.want) + } + if strings.Contains(got, "Unknown") { + t.Errorf("attribute rendered as Unknown: %q — two attributes of "+ + "different types then diff to nothing", got) + } + }) + } +} + +// The whole point, stated as a diff: a length change must not render the same +// on both sides. +func TestAttributeBsonToMDL_LengthChangeIsVisible(t *testing.T) { + before := attributeBsonToMDL(nil, attrRaw("FullName", + map[string]any{"$Type": "DomainModels$StringAttributeType", "Length": int32(200)})) + after := attributeBsonToMDL(nil, attrRaw("FullName", + map[string]any{"$Type": "DomainModels$StringAttributeType", "Length": int32(50)})) + if before == after { + t.Fatalf("String(200) and String(50) both render as %q — diff-local "+ + "counts the unit as modified and shows no change", before) + } +} + +// Documents written before the rename still carry "Type"; keep reading it, so +// the fix is additive rather than a swap. +func TestAttributeBsonToMDL_StillReadsLegacyTypeKey(t *testing.T) { + raw := map[string]any{ + "$Type": "DomainModels$Attribute", + "Name": "Legacy", + "Type": map[string]any{"$Type": "DomainModels$IntegerAttributeType"}, + } + if got := attributeBsonToMDL(nil, raw); got != "Legacy: Integer" { + t.Errorf("attributeBsonToMDL = %q, want %q", got, "Legacy: Integer") + } +} diff --git a/mdl/executor/cmd_microflows_builder_calls.go b/mdl/executor/cmd_microflows_builder_calls.go index 0665a606b0..35b7ab302b 100644 --- a/mdl/executor/cmd_microflows_builder_calls.go +++ b/mdl/executor/cmd_microflows_builder_calls.go @@ -548,9 +548,36 @@ func (fb *flowBuilder) addCallWebServiceAction(s *ast.CallWebServiceStmt) model. BaseElement: model.BaseElement{ID: model.ID(types.GenerateID())}, ErrorHandlingType: convertErrorHandlingType(s.ErrorHandling), ServiceID: model.ID(s.ServiceID), - OperationName: s.OperationName, - SendMappingID: model.ID(fb.resolveMappingRefForWrite(s.SendMappingID, true)), - ReceiveMappingID: model.ID(fb.resolveMappingRefForWrite(s.ReceiveMappingID, false)), + // The WSDL service name, read off the imported service document. Empty + // when it cannot be established (a dangling reference, a backend that + // cannot list raw units), and the writers then derive it as before. + ServiceName: resolveWebServiceName(fb.backend, s.ServiceID, s.OperationName), + OperationName: s.OperationName, + // The QUALIFIED NAME, verbatim. + // + // Both are BY_NAME_REFERENCEs: the stored ImportMappingCall's + // ReturnValueMapping is an ImportMappingIdentifier, and Studio Pro + // writes "Clients.SoapOrdersImportMapping" there. These used to be + // resolved to the mapping's unit `$ID`, which does not merely fail + // validation — it makes the project impossible to LOAD. Measured on + // 11.14.0 against ako/TestApp (baseline 0 errors): one + // `receive mapping Clients.SoapOrdersImportMapping` written by mxcli + // and `mx check` stops before validation with + // StorageLoadException, "The text 'c2d1682f-…' is not a valid + // ImportMappingIdentifier." + // + // Why it survived is worth more than the fix: the substitution only + // happened when the lookup SUCCEEDED, and the only SOAP fixture + // (06b-soap-examples.mdl) names mappings that do not exist, on + // purpose, to show dangling refs. It took the fallback every time and + // passed. The gate was green BECAUSE the fixture was broken — a valid + // reference was the one input that triggered the defect, and nothing + // tested one. + SendMappingID: model.ID(s.SendMappingID), + ReceiveMappingID: model.ID(s.ReceiveMappingID), + // The entity the receive mapping produces, which types the result + // variable. Empty when unresolvable, and the writers keep VoidType. + ResultEntity: resolveImportMappingEntity(fb.backend, s.ReceiveMappingID), OutputVariable: s.OutputVariable, UseReturnVariable: s.OutputVariable != "", } @@ -595,32 +622,6 @@ func (fb *flowBuilder) addCallWebServiceAction(s *ast.CallWebServiceStmt) model. return activity.ID } -func (fb *flowBuilder) resolveMappingRefForWrite(ref string, preferExport bool) string { - if ref == "" || !strings.Contains(ref, ".") || fb.backend == nil { - return ref - } - moduleName, name, ok := strings.Cut(ref, ".") - if !ok || moduleName == "" || name == "" { - return ref - } - if preferExport { - if mapping, err := fb.backend.GetExportMappingByQualifiedName(moduleName, name); err == nil && mapping != nil { - return string(mapping.ID) - } - if mapping, err := fb.backend.GetImportMappingByQualifiedName(moduleName, name); err == nil && mapping != nil { - return string(mapping.ID) - } - } else { - if mapping, err := fb.backend.GetImportMappingByQualifiedName(moduleName, name); err == nil && mapping != nil { - return string(mapping.ID) - } - if mapping, err := fb.backend.GetExportMappingByQualifiedName(moduleName, name); err == nil && mapping != nil { - return string(mapping.ID) - } - } - return ref -} - // resolveExternalActionReturnKind looks up the called OData action in the // consumed service's cached $metadata and returns the Mendix kind name // ("Boolean", "String", "Integer", "Long", "Decimal", "DateTime", "Binary", diff --git a/mdl/executor/cmd_microflows_builder_webservice_test.go b/mdl/executor/cmd_microflows_builder_webservice_test.go index 22324ef020..2a24e54dec 100644 --- a/mdl/executor/cmd_microflows_builder_webservice_test.go +++ b/mdl/executor/cmd_microflows_builder_webservice_test.go @@ -70,12 +70,24 @@ func TestBuildFlowGraph_WebServiceCallCreatesRealAction(t *testing.T) { if action.ServiceID != "SampleSOAP.OrderService" { t.Errorf("ServiceID = %q, want SampleSOAP.OrderService", action.ServiceID) } - if action.SendMappingID != sendMappingID { - t.Errorf("SendMappingID = %q, want %q", action.SendMappingID, sendMappingID) + // The QUALIFIED NAMES, not the mappings' unit ids. + // + // This test used to assert the ids, locking in a defect: the stored + // ImportMappingCall's ReturnValueMapping is an ImportMappingIdentifier, and a + // UUID there makes the project impossible to LOAD — `mx check` stops with a + // StorageLoadException before validation. Measured on 11.14.0 against + // ako/TestApp. The mock resolving these names is what made the old + // expectation reachable at all; the only SOAP fixture names mappings that do + // not exist, so exec never took the resolving branch and nothing else noticed. + if string(action.SendMappingID) != "SampleSOAP.OrderRequest" { + t.Errorf("SendMappingID = %q, want the qualified name", action.SendMappingID) } - if action.ReceiveMappingID != receiveMappingID { - t.Errorf("ReceiveMappingID = %q, want %q", action.ReceiveMappingID, receiveMappingID) + if string(action.ReceiveMappingID) != "SampleSOAP.OrderResponse" { + t.Errorf("ReceiveMappingID = %q, want the qualified name", action.ReceiveMappingID) } + // Unused now that the ids must not reach the action, but kept wired into the + // mock above: a future change that starts resolving them again fails here. + _, _ = sendMappingID, receiveMappingID if action.OutputVariable != "Root" || !action.UseReturnVariable { t.Errorf("output = %q/%v", action.OutputVariable, action.UseReturnVariable) } diff --git a/mdl/executor/cmd_microflows_format_action.go b/mdl/executor/cmd_microflows_format_action.go index e6ca3518b1..bbd8c29134 100644 --- a/mdl/executor/cmd_microflows_format_action.go +++ b/mdl/executor/cmd_microflows_format_action.go @@ -10,7 +10,6 @@ import ( "sort" "strings" - mdltypes "github.com/mendixlabs/mxcli/mdl/types" "github.com/mendixlabs/mxcli/mdl/visitor" "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/microflows" @@ -1010,15 +1009,19 @@ func formatWebServiceCallAction(ctx *ExecContext, a *microflows.WebServiceCallAc return prefix + "call web service raw " + mdlQuote(raw) + ";" } - parts := []string{prefix + "call web service " + formatWebServiceReference(resolveWebServiceReference(ctx, a.ServiceID))} + // The service and the two mappings are BY_NAME_REFERENCE properties: what is + // stored IS the qualified name, so there is nothing to resolve. See + // TestFormatAction_WebServiceCallRendersStoredQualifiedNames for what the + // three resolvers that used to stand here actually did. + parts := []string{prefix + "call web service " + formatWebServiceReference(string(a.ServiceID))} if a.OperationName != "" { parts = append(parts, "operation "+formatWebServiceReference(a.OperationName)) } if a.SendMappingID != "" { - parts = append(parts, "send mapping "+formatWebServiceReference(resolveWebServiceMappingReference(ctx, a.SendMappingID, true))) + parts = append(parts, "send mapping "+formatWebServiceReference(string(a.SendMappingID))) } if a.ReceiveMappingID != "" { - parts = append(parts, "receive mapping "+formatWebServiceReference(resolveWebServiceMappingReference(ctx, a.ReceiveMappingID, false))) + parts = append(parts, "receive mapping "+formatWebServiceReference(string(a.ReceiveMappingID))) } if a.TimeoutExpression != "" { parts = append(parts, "timeout "+strings.TrimRight(a.TimeoutExpression, " \t\n\r")) @@ -1063,107 +1066,6 @@ func isBareIdentifier(part string) bool { return true } -func resolveWebServiceReference(ctx *ExecContext, id model.ID) string { - raw := string(id) - if raw == "" || ctx == nil || ctx.Backend == nil { - return raw - } - units, err := ctx.Backend.ListRawUnitsByType("WebServices$ImportedWebService") - if err != nil { - return raw - } - h, err := getHierarchy(ctx) - if err != nil { - return raw - } - for _, unit := range units { - if unit == nil || unit.ID != id { - continue - } - return qualifiedRawUnitName(h, unit, raw) - } - return raw -} - -func qualifiedRawUnitName(h *ContainerHierarchy, unit *mdltypes.RawUnit, fallback string) string { - name := rawUnitName(unit.Contents) - if name == "" { - return fallback - } - if h == nil { - return name - } - if qn := h.GetQualifiedName(unit.ContainerID, name); qn != "." && qn != "" { - return qn - } - return name -} - -func resolveWebServiceMappingReference(ctx *ExecContext, id model.ID, preferExport bool) string { - if preferExport { - if qn := resolveExportMappingReference(ctx, id); qn != "" { - return qn - } - if qn := resolveImportMappingReference(ctx, id); qn != "" { - return qn - } - } else { - if qn := resolveImportMappingReference(ctx, id); qn != "" { - return qn - } - if qn := resolveExportMappingReference(ctx, id); qn != "" { - return qn - } - } - return string(id) -} - -func resolveImportMappingReference(ctx *ExecContext, id model.ID) string { - if id == "" || ctx == nil || ctx.Backend == nil { - return "" - } - mappings, err := ctx.Backend.ListImportMappings() - if err != nil { - return "" - } - for _, mapping := range mappings { - if mapping != nil && mapping.ID == id { - return qualifiedNameForContainer(ctx, mapping.ContainerID, mapping.Name) - } - } - return "" -} - -func resolveExportMappingReference(ctx *ExecContext, id model.ID) string { - if id == "" || ctx == nil || ctx.Backend == nil { - return "" - } - mappings, err := ctx.Backend.ListExportMappings() - if err != nil { - return "" - } - for _, mapping := range mappings { - if mapping != nil && mapping.ID == id { - return qualifiedNameForContainer(ctx, mapping.ContainerID, mapping.Name) - } - } - return "" -} - -func qualifiedNameForContainer(ctx *ExecContext, containerID model.ID, name string) string { - if name == "" { - return "" - } - h, err := getHierarchy(ctx) - if err != nil || h == nil { - return name - } - if qn := h.GetQualifiedName(containerID, name); qn != "." && qn != "" { - return qn - } - return name -} - func rawUnitName(contents []byte) string { var raw map[string]any if err := bson.Unmarshal(contents, &raw); err != nil { diff --git a/mdl/executor/cmd_microflows_format_action_test.go b/mdl/executor/cmd_microflows_format_action_test.go index 8aee2fb237..4e96c4d102 100644 --- a/mdl/executor/cmd_microflows_format_action_test.go +++ b/mdl/executor/cmd_microflows_format_action_test.go @@ -1270,76 +1270,64 @@ func TestGetActionErrorHandlingType_JavaScriptActionCallAction(t *testing.T) { } } -func TestFormatAction_WebServiceCallResolvesKnownReferences(t *testing.T) { - moduleID := mkID("soap-module") - serviceID := mkID("soap-service") - sendMappingID := mkID("soap-send") - receiveMappingID := mkID("soap-receive") - serviceContents, err := bson.Marshal(bson.M{"Name": "OrderService"}) - if err != nil { - t.Fatal(err) - } - +// TestFormatAction_WebServiceCallRendersStoredQualifiedNames pins that DESCRIBE +// prints the three names a SOAP call stores, verbatim, and consults nothing. +// +// All three are BY_NAME_REFERENCE properties — qualified-name strings, never +// element ids. Measured on ako/TestApp (11.14.0), Clients.GetOrders stores +// ImportedService "Clients.OrderSoapClient" and ReturnValueMapping +// "Clients.SoapOrdersImportMapping"; modelsdk/gen agrees, registering +// ImportedWebService and ImportMappingCall.Mapping as codec.RefByName. +// +// Three resolvers used to sit here turning those strings into qualified names +// by looking each one up in the project. They could not match on two counts at +// once — the id compared was already a qualified name, and the unit type asked +// for ("WebServices$ImportedWebService") is stored as +// "WebServices$ImportedServiceImpl" — so every call fell through to its +// fallback, which returned the stored string. The right answer, by accident. +// +// The backend below fails the test if it is asked anything: a lookup returning +// the correct value is indistinguishable from no lookup at all in the output, +// which is precisely how this survived. +func TestFormatAction_WebServiceCallRendersStoredQualifiedNames(t *testing.T) { backend := &mock.MockBackend{ IsConnectedFunc: func() bool { return true }, ListRawUnitsByTypeFunc: func(typePrefix string) ([]*mdltypes.RawUnit, error) { - if typePrefix != "WebServices$ImportedWebService" { - t.Fatalf("unexpected type prefix %q", typePrefix) - } - return []*mdltypes.RawUnit{{ - ID: serviceID, - ContainerID: moduleID, - Type: "WebServices$ImportedWebService", - Contents: serviceContents, - }}, nil + t.Fatalf("DESCRIBE consulted the project for %q; the stored names are already qualified", typePrefix) + return nil, nil }, ListExportMappingsFunc: func() ([]*model.ExportMapping, error) { - return []*model.ExportMapping{{ - BaseElement: model.BaseElement{ID: sendMappingID}, - ContainerID: moduleID, - Name: "OrderRequest", - }}, nil + t.Fatal("DESCRIBE listed export mappings; the send mapping is stored by qualified name") + return nil, nil }, ListImportMappingsFunc: func() ([]*model.ImportMapping, error) { - return []*model.ImportMapping{{ - BaseElement: model.BaseElement{ID: receiveMappingID}, - ContainerID: moduleID, - Name: "OrderResponse", - }}, nil + t.Fatal("DESCRIBE listed import mappings; the receive mapping is stored by qualified name") + return nil, nil }, } - h := mkHierarchy(&model.Module{BaseElement: model.BaseElement{ID: moduleID}, Name: "SyntheticSOAP"}) - ctx, _ := newMockCtx(t, withBackend(backend), withHierarchy(h)) + ctx, _ := newMockCtx(t, withBackend(backend), withHierarchy(mkHierarchy())) action := µflows.WebServiceCallAction{ - ServiceID: serviceID, - OperationName: "FetchOrders", - SendMappingID: sendMappingID, - ReceiveMappingID: receiveMappingID, - OutputVariable: "Root", + ServiceID: "Clients.OrderSoapClient", + OperationName: "GetOrder", + SendMappingID: "Clients.SoapOrderExportMapping", + ReceiveMappingID: "Clients.SoapOrdersImportMapping", + OutputVariable: "Orders", UseReturnVariable: true, } got := formatAction(ctx, action, nil, nil) - want := "$Root = call web service SyntheticSOAP.OrderService\noperation FetchOrders\nsend mapping SyntheticSOAP.OrderRequest\nreceive mapping SyntheticSOAP.OrderResponse;" + want := "$Orders = call web service Clients.OrderSoapClient\noperation GetOrder\n" + + "send mapping Clients.SoapOrderExportMapping\nreceive mapping Clients.SoapOrdersImportMapping;" if got != want { t.Errorf("got %q, want %q", got, want) } } -func TestFormatAction_WebServiceCallKeepsRawReferencesWhenUnknown(t *testing.T) { - backend := &mock.MockBackend{ - IsConnectedFunc: func() bool { return true }, - ListRawUnitsByTypeFunc: func(typePrefix string) ([]*mdltypes.RawUnit, error) { - return nil, nil - }, - ListExportMappingsFunc: func() ([]*model.ExportMapping, error) { - return nil, nil - }, - ListImportMappingsFunc: func() ([]*model.ImportMapping, error) { - return nil, nil - }, - } - ctx, _ := newMockCtx(t, withBackend(backend), withHierarchy(mkHierarchy())) +// TestFormatAction_WebServiceCallQuotesNamesThatAreNotIdentifiers — a dangling +// or malformed reference is still printed, quoted so the line parses back. +func TestFormatAction_WebServiceCallQuotesNamesThatAreNotIdentifiers(t *testing.T) { + ctx, _ := newMockCtx(t, withBackend(&mock.MockBackend{IsConnectedFunc: func() bool { return true }}), + withHierarchy(mkHierarchy())) action := µflows.WebServiceCallAction{ ServiceID: "dangling-service-id", diff --git a/mdl/executor/validate.go b/mdl/executor/validate.go index fc507b3689..6bccfff1c8 100644 --- a/mdl/executor/validate.go +++ b/mdl/executor/validate.go @@ -46,6 +46,16 @@ type scriptContext struct { // script-defined flow is what puts an entity into context for the widgets // nested in it. flowParams map[string]*flowSignature // Module.Flow (lower-cased) -> signature + + // Associations and entity attributes declared in the script, for + // MDL-XPATH01. Same reason as flowParams above: the overwhelmingly common + // shape is ONE script that creates the entity, the association and the + // microflow constraining on it, so a rule that could only see a stored + // association would fire on the minority case only — and the majority case + // is exactly the one that reaches a build half-written. + associations map[string]string // Association (unqualified) -> Module.Association + entityAttrs map[string]map[string]bool // Module.Entity -> attribute names + ambiguousAssc map[string]bool // names defined in more than one module } // newScriptContext creates a new script context. @@ -64,10 +74,44 @@ func newScriptContext() *scriptContext { javaActions: make(map[string][]string), javaScriptActions: make(map[string][]string), + associations: map[string]string{}, + entityAttrs: map[string]map[string]bool{}, + ambiguousAssc: map[string]bool{}, flowParams: make(map[string]*flowSignature), } } +// recordEntityAttrs stores a script-declared entity's attribute names, for the +// rules that need to tell an attribute from an association (MDL-XPATH01). +func (sc *scriptContext) recordEntityAttrs(s *ast.CreateEntityStmt) { + attrs := make(map[string]bool, len(s.Attributes)) + for _, a := range s.Attributes { + attrs[a.Name] = true + } + sc.entityAttrs[s.Name.String()] = attrs +} + +// recordAssociation stores a script-declared association under its UNQUALIFIED +// name, because that is the spelling an XPath constraint gets wrong. A name +// declared in two modules is recorded as ambiguous and then dropped by the +// rule: naming the right spelling is the whole value, and offering one of two +// would be wrong half the time. +// +// Both collectDefinitions and collectSingle call this. They are two parallel +// switches over the same statement types, so a case added to one and not the +// other is collected on one path only — which is how this rule first shipped +// firing against stored associations but not script-declared ones. +func (sc *scriptContext) recordAssociation(s *ast.CreateAssociationStmt) { + if s.Name.Module == "" || s.Name.Name == "" { + return + } + if prev, ok := sc.associations[s.Name.Name]; ok && prev != s.Name.String() { + sc.ambiguousAssc[s.Name.Name] = true + return + } + sc.associations[s.Name.Name] = s.Name.String() +} + // codeActionParamNames returns the declared parameter names of a CREATE JAVA // ACTION / CREATE JAVASCRIPT ACTION statement, in declaration order. func codeActionParamNames(params []ast.JavaActionParam) []string { @@ -87,7 +131,10 @@ func (sc *scriptContext) collectDefinitions(prog *ast.Program) { case *ast.CreateEntityStmt: if s.Name.Module != "" { sc.entities[s.Name.String()] = true + sc.recordEntityAttrs(s) } + case *ast.CreateAssociationStmt: + sc.recordAssociation(s) case *ast.CreateViewEntityStmt: if s.Name.Module != "" { sc.entities[s.Name.String()] = true @@ -150,7 +197,10 @@ func (sc *scriptContext) collectSingle(stmt ast.Statement) { case *ast.CreateEntityStmt: if s.Name.Module != "" { sc.entities[s.Name.String()] = true + sc.recordEntityAttrs(s) } + case *ast.CreateAssociationStmt: + sc.recordAssociation(s) case *ast.CreateViewEntityStmt: if s.Name.Module != "" { sc.entities[s.Name.String()] = true @@ -587,9 +637,10 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext return mdlerrors.NewValidationf("page '%s' has context errors:\n - %s", s.Name.String(), strings.Join(ctxErrors, "\n - ")) } - // CE1571: a microflow data source must be given an argument per parameter. - if argErrors := validateDataSourceArguments(ctx, s.Parameters, s.Widgets, sc); len(argErrors) > 0 { - return mdlerrors.NewValidationf("page '%s' has data source errors:\n - %s", + // CE1571: a microflow call must be given an argument per parameter — + // as a data source and as an action alike (mendixlabs/mxcli#1082). + if argErrors := validateFlowArguments(ctx, s.Parameters, s.Widgets, sc); len(argErrors) > 0 { + return mdlerrors.NewValidationf("page '%s' has argument errors:\n - %s", s.Name.String(), strings.Join(argErrors, "\n - ")) } case *ast.CreateSnippetStmtV3: @@ -603,10 +654,10 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext return mdlerrors.NewValidationf("snippet '%s' has reference errors:\n - %s", s.Name.String(), strings.Join(refErrors, "\n - ")) } - // A snippet takes the same data sources a page does, and CE1571 does not - // care which document the widget lives in. - if argErrors := validateDataSourceArguments(ctx, s.Parameters, s.Widgets, sc); len(argErrors) > 0 { - return mdlerrors.NewValidationf("snippet '%s' has data source errors:\n - %s", + // A snippet takes the same data sources and actions a page does, and + // CE1571 does not care which document the widget lives in. + if argErrors := validateFlowArguments(ctx, s.Parameters, s.Widgets, sc); len(argErrors) > 0 { + return mdlerrors.NewValidationf("snippet '%s' has argument errors:\n - %s", s.Name.String(), strings.Join(argErrors, "\n - ")) } // Validate snippet context tree (parameter/selection/attribute bindings) @@ -877,6 +928,7 @@ func validateFlowBodyReferences(ctx *ExecContext, body []ast.MicroflowStatement, if len(refs.retrieves) > 0 { errors = append(errors, validateRetrieveConstraints(ctx, refs.retrieves)...) + errors = append(errors, validateXPathAssociations(ctx, refs.retrieves, sc)...) } return errors diff --git a/mdl/executor/validate_datasource_args.go b/mdl/executor/validate_datasource_args.go index 2c8b04394f..a89e329ee6 100644 --- a/mdl/executor/validate_datasource_args.go +++ b/mdl/executor/validate_datasource_args.go @@ -40,6 +40,45 @@ // Where a context's entity type cannot be resolved (an association or selection // source), nothing is reported: guessing wrong in that direction is what // rejected the working page in the first place. +// +// # The same CE, the other property (mendixlabs/mxcli#1082) +// +// CE1571 is not a data-source error. It is raised for any call whose parameters +// are not all supplied, and a widget's ACTION is such a call — so the rule now +// covers `Action:` / `OnClick:` / `OnChange:` and a pluggable widget's named +// action slots as well. Before this it walked GetDataSource() alone, and the +// identical fault on an action passed `mxcli check --references` clean and then +// failed the build. Measured on mxbuild 11.12.0, one project, one `mx check`: +// +// CONTAINER action, no argument, no enclosing data context → CE1571 +// the same page put through `mxcli check` → "Check passed!" +// +// The action obeys the same context rule as the data source, measured the same +// way — six containers carrying the identical fault, one `mx check`: +// +// nested in a dataview of the parameter's type → no error +// nested in a dataview of a DIFFERENT type → CE1571 +// page parameter of the exact type, no dataview → CE1571 +// inside a data grid COLUMN (row-scoped) → no error +// no enclosing data context at all → CE1571 +// inside a data grid CONTROL BAR → CE1571 +// +// # A control bar is not row-scoped +// +// The last row is the reported shape, and it is the one the context walk got +// wrong: a control bar is a child of the data widget, so it used to inherit the +// row context and the check stayed silent. It applies to data sources too — +// same project, same run: +// +// dataview whose microflow source needs an argument, +// placed in the grid's CONTROL BAR → CE1571 +// the same dataview placed in a COLUMN → no error +// +// So the fix belongs in the walk rather than in either rule, and it drops only +// the data widget's OWN object: an outer context still reaches the control bar +// (dataview > datagrid > controlbar > action, no argument → no error), which is +// why the control bar is walked with the context its parent was walked with +// rather than with an empty one. package executor import ( @@ -140,8 +179,9 @@ func (c dataContext) describe() string { } } -// validateDataSourceArguments reports microflow/nanoflow data sources whose -// argument list does not match the flow's parameters. +// validateFlowArguments reports microflow/nanoflow calls on a widget — its data +// source and its action slots alike — whose argument list does not match the +// flow's parameters. // // Two distinct faults, both provable: // @@ -153,7 +193,7 @@ func (c dataContext) describe() string { // validateWidgetReferences' job, and reporting it twice would be noise. A flow // created earlier in the SAME script is likewise skipped rather than guessed at, // since its signature is not in the project yet. -func validateDataSourceArguments(ctx *ExecContext, params []ast.PageParameter, widgets []*ast.WidgetV3, sc *scriptContext) []string { +func validateFlowArguments(ctx *ExecContext, params []ast.PageParameter, widgets []*ast.WidgetV3, sc *scriptContext) []string { if !ctx.Connected() || len(widgets) == 0 { return nil } @@ -170,13 +210,13 @@ func validateDataSourceArguments(ctx *ExecContext, params []ast.PageParameter, w if len(sigs) == 0 { return nil } - return validateDataSourceArgumentsIn(params, widgets, sigs, entityCompatibility(ctx)) + return validateFlowArgumentsIn(params, widgets, sigs, entityCompatibility(ctx)) } -// validateDataSourceArgumentsIn is the whole rule with its two project-dependent +// validateFlowArgumentsIn is the whole rule with its two project-dependent // inputs — the flow signatures and the entity-compatibility test — handed in, so // it can be measured against the shapes `mx check` was run on. -func validateDataSourceArgumentsIn( +func validateFlowArgumentsIn( pageParams []ast.PageParameter, widgets []*ast.WidgetV3, sigs map[string]*flowSignature, @@ -190,25 +230,49 @@ func validateDataSourceArgumentsIn( } var errs []string - var walk func(ws []*ast.WidgetV3, enclosing dataContext) - walk = func(ws []*ast.WidgetV3, enclosing dataContext) { - for _, w := range ws { - if w == nil { + var walk func(w *ast.WidgetV3, enclosing dataContext, controlBarOf string) + walk = func(w *ast.WidgetV3, enclosing dataContext, controlBarOf string) { + if w == nil { + return + } + ds := w.GetDataSource() + if ds != nil { + // The context checked is what ENCLOSES the widget: a data source + // cannot supply its own parameter. + errs = append(errs, dataSourceArgErrors(w.Name, ds, sigs, enclosing, compatible)...) + } + errs = append(errs, actionArgErrors(w, sigs, enclosing, compatible, controlBarOf)...) + + inner := childContext(enclosing, ds, sigs, paramEntities) + // A widget that establishes its own data context ends the control bar's + // reach: inside a dataview placed in a control bar, $currentObject is + // that dataview's object like anywhere else. + childControlBarOf := controlBarOf + if ds != nil { + childControlBarOf = "" + } + for _, c := range w.Children { + if isControlBar(c) { + // Not row-scoped: the data widget's own object is out of scope + // here, everything above it is not. Measured — see the header. + walk(c, enclosing, w.Name) continue } - ds := w.GetDataSource() - if ds != nil { - // The context checked is what ENCLOSES the widget: a data source - // cannot supply its own parameter. - errs = append(errs, dataSourceArgErrors(w.Name, ds, sigs, enclosing, compatible)...) - } - walk(w.Children, childContext(enclosing, ds, sigs, paramEntities)) + walk(c, inner, childControlBarOf) } } - walk(widgets, dataContext{}) + for _, w := range widgets { + walk(w, dataContext{}, "") + } return errs } +// isControlBar reports whether the widget is a data widget's control bar, the +// one child that does not inherit its parent's row context. +func isControlBar(w *ast.WidgetV3) bool { + return w != nil && strings.EqualFold(w.Type, "controlbar") +} + // childContext extends the enclosing context with what this widget's data source // puts in scope for its children. // @@ -258,8 +322,46 @@ func dataSourceArgErrors( return nil // resolution is validateWidgetReferences' job } - given := make(map[string]bool, len(ds.Args)) - for _, a := range ds.Args { + missing, unknown, missingEntity := argDiff(ds.Args, sig, enclosing, compatible) + + var out []string + if len(missing) > 0 { + msg := fmt.Sprintf( + "widget '%s': %s data source %s has no argument for %s %s — Mendix rejects this with CE1571. "+ + "Write it as `%s(%s: $Value)`", + widgetName, ds.Type, ds.Reference, + plural(len(missing), "parameter", "parameters"), + quoteJoin(missing), ds.Reference, missing[0]) + if hint := contextHint(enclosing, missingEntity); hint != "" { + msg += hint + } + out = append(out, msg) + } + if len(unknown) > 0 { + out = append(out, fmt.Sprintf( + "widget '%s': %s data source %s has no %s %s (it declares %s)", + widgetName, ds.Type, ds.Reference, + plural(len(unknown), "parameter", "parameters"), + quoteJoin(unknown), quoteJoin(sig.paramNames()))) + } + return out +} + +// argDiff compares a call's arguments against a flow's parameters, treating the +// enclosing data context as supplying what it supplies. +// +// Shared by the data source and the action, because CE1571 does not distinguish +// them: both are a call whose parameters must all be filled. Returns the +// parameters nothing fills, the arguments naming no parameter, and the entity of +// the first missing parameter (what the context hint is written against). +func argDiff( + args []ast.FlowArgV3, + sig *flowSignature, + enclosing dataContext, + compatible func(contextQN, paramQN string) bool, +) (missing []string, unknown []string, missingEntity string) { + given := make(map[string]bool, len(args)) + for _, a := range args { given[strings.ToLower(a.Name)] = true } wanted := make(map[string]bool, len(sig.Params)) @@ -267,8 +369,6 @@ func dataSourceArgErrors( wanted[strings.ToLower(p.Name)] = true } - var missing []string - var missingEntity string for _, p := range sig.Params { if given[strings.ToLower(p.Name)] || enclosing.supplies(p, compatible) { continue @@ -278,38 +378,122 @@ func dataSourceArgErrors( missingEntity = p.Entity } } - var unknown []string - for _, a := range ds.Args { + for _, a := range args { if !wanted[strings.ToLower(a.Name)] { unknown = append(unknown, a.Name) } } sort.Strings(missing) sort.Strings(unknown) + return missing, unknown, missingEntity +} - var out []string - if len(missing) > 0 { - msg := fmt.Sprintf( - "widget '%s': %s data source %s has no argument for %s %s — Mendix rejects this with CE1571. "+ - "Write it as `%s(%s: $Value)`", - widgetName, ds.Type, ds.Reference, - plural(len(missing), "parameter", "parameters"), - quoteJoin(missing), ds.Reference, missing[0]) - if hint := contextHint(enclosing, missingEntity); hint != "" { - msg += hint +// widgetActions returns the widget's microflow and nanoflow actions, each with +// the property key it was written under. +// +// Every action slot goes through one Properties sweep rather than a list of +// known keys: `Action:`/`OnClick:` land on "Action", `OnChange:` on "OnChange", +// and a pluggable widget's named slot on its own key (`createFileAction:`), all +// as *ast.ActionV3. A list of keys would have covered the reported one and +// quietly missed the rest — the same shape as the bug being fixed. +// +// Keys are sorted so a widget with two faulty slots reports them in a stable +// order. A chained THEN action is followed: `create_object E then show_page P` +// carries a second call, and only the flow ones are kept here. +func widgetActions(w *ast.WidgetV3) []namedAction { + if w == nil || len(w.Properties) == 0 { + return nil + } + keys := make([]string, 0, len(w.Properties)) + for k := range w.Properties { + keys = append(keys, k) + } + sort.Strings(keys) + + var out []namedAction + for _, k := range keys { + a, ok := w.Properties[k].(*ast.ActionV3) + if !ok { + continue + } + for ; a != nil; a = a.ThenAction { + if a.Type == "microflow" || a.Type == "nanoflow" { + out = append(out, namedAction{Property: k, Action: a}) + } } - out = append(out, msg) } - if len(unknown) > 0 { - out = append(out, fmt.Sprintf( - "widget '%s': %s data source %s has no %s %s (it declares %s)", - widgetName, ds.Type, ds.Reference, - plural(len(unknown), "parameter", "parameters"), - quoteJoin(unknown), quoteJoin(sig.paramNames()))) + return out +} + +// namedAction is one action slot: the property it was written under, and the +// call in it. +type namedAction struct { + Property string + Action *ast.ActionV3 +} + +// actionArgErrors reports a widget's action slots whose arguments do not match +// the flow's parameters — CE1571 and the silent-typo argument, the same two +// faults dataSourceArgErrors reports for a data source. +// +// controlBarOf names the data widget whose control bar this widget sits in, and +// is "" everywhere else. It only changes the advice: the remedy there is the +// grid's selection, which is the one thing a control-bar author needs told and +// the thing the reported issue went looking for in the grammar. +func actionArgErrors( + w *ast.WidgetV3, + sigs map[string]*flowSignature, + enclosing dataContext, + compatible func(contextQN, paramQN string) bool, + controlBarOf string, +) []string { + var out []string + for _, na := range widgetActions(w) { + sig, known := sigs[strings.ToLower(na.Action.Target)] + if !known || sig == nil { + continue // resolution is validateWidgetReferences' job + } + missing, unknown, missingEntity := argDiff(na.Action.Args, sig, enclosing, compatible) + + if len(missing) > 0 { + msg := fmt.Sprintf( + "widget '%s': %s action %s (%s:) has no argument for %s %s — Mendix rejects this with CE1571. "+ + "Write it as `%s: %s %s(%s: $Value)`", + w.Name, na.Action.Type, na.Action.Target, na.Property, + plural(len(missing), "parameter", "parameters"), + quoteJoin(missing), + na.Property, na.Action.Type, na.Action.Target, missing[0]) + if hint := controlBarHint(controlBarOf); hint != "" { + msg += hint + } else if hint := contextHint(enclosing, missingEntity); hint != "" { + msg += hint + } + out = append(out, msg) + } + if len(unknown) > 0 { + out = append(out, fmt.Sprintf( + "widget '%s': %s action %s (%s:) has no %s %s (it declares %s)", + w.Name, na.Action.Type, na.Action.Target, na.Property, + plural(len(unknown), "parameter", "parameters"), + quoteJoin(unknown), quoteJoin(sig.paramNames()))) + } } return out } +// controlBarHint is the advice for an action in a control bar, which replaces +// the ordinary "nest it in a data container" hint: a control bar is not +// row-scoped, so nesting is not the remedy — the grid's selection is. +func controlBarHint(controlBarOf string) string { + if controlBarOf == "" { + return "" + } + return fmt.Sprintf( + ". A control bar is not row-scoped, so no enclosing row fills it in: pass the selection of `%s` "+ + "(`$%s`, once the widget has `Selection:` set), or move the widget into a column, which is row-scoped", + controlBarOf, controlBarOf) +} + // contextHint says why the enclosing context did not fill the parameter in. // Without it the reader of a nested widget's error has to work out for themselves // that the surrounding data container is the wrong type. diff --git a/mdl/executor/validate_datasource_context_test.go b/mdl/executor/validate_datasource_context_test.go index 5adcf904b6..0562f921d1 100644 --- a/mdl/executor/validate_datasource_context_test.go +++ b/mdl/executor/validate_datasource_context_test.go @@ -77,7 +77,7 @@ func TestDataSourceArg_MatchingEnclosingContextSuppliesTheArgument(t *testing.T) dsWidget("dgMatrix", "DS.DS_Rows"), ), } - if errs := validateDataSourceArgumentsIn(nil, page, f147Sigs(), sameName); len(errs) != 0 { + if errs := validateFlowArgumentsIn(nil, page, f147Sigs(), sameName); len(errs) != 0 { t.Errorf("mxbuild accepts this page; the check must not reject it:\n %s", strings.Join(errs, "\n ")) } @@ -92,7 +92,7 @@ func TestDataSourceArg_MatchingContextTwoLevelsUp(t *testing.T) { ), ), } - if errs := validateDataSourceArgumentsIn(nil, page, f147Sigs(), sameName); len(errs) != 0 { + if errs := validateFlowArgumentsIn(nil, page, f147Sigs(), sameName); len(errs) != 0 { t.Errorf("a matching context two levels up still supplies the argument:\n %s", strings.Join(errs, "\n ")) } @@ -101,7 +101,7 @@ func TestDataSourceArg_MatchingContextTwoLevelsUp(t *testing.T) { // CONTROL 1 (dgLoose): no enclosing data context at all — still CE1571. func TestDataSourceArg_NoContextIsStillReported(t *testing.T) { page := []*ast.WidgetV3{dsWidget("dgLoose", "DS.DS_Rows")} - errs := validateDataSourceArgumentsIn(nil, page, f147Sigs(), sameName) + errs := validateFlowArgumentsIn(nil, page, f147Sigs(), sameName) if len(errs) != 1 { t.Fatalf("got %d errors, want 1: %v", len(errs), errs) } @@ -118,7 +118,7 @@ func TestDataSourceArg_MismatchedContextIsStillReported(t *testing.T) { ), } params := []ast.PageParameter{{Name: "Row", EntityType: ast.QualifiedName{Module: "DS147", Name: "Row"}}} - errs := validateDataSourceArgumentsIn(params, page, f147Sigs(), sameName) + errs := validateFlowArgumentsIn(params, page, f147Sigs(), sameName) if len(errs) != 1 { t.Fatalf("got %d errors, want 1: %v", len(errs), errs) } @@ -140,7 +140,7 @@ func TestDataSourceArg_PageParameterAloneDoesNotSupplyIt(t *testing.T) { params := []ast.PageParameter{ {Name: "Ctx", EntityType: ast.QualifiedName{Module: "DS147", Name: "ReportContext"}}, } - errs := validateDataSourceArgumentsIn(params, page, f147Sigs(), sameName) + errs := validateFlowArgumentsIn(params, page, f147Sigs(), sameName) if len(errs) != 1 { t.Fatalf("a page parameter of the right type is NOT a data context (measured: CE1571); got %d errors: %v", len(errs), errs) @@ -158,7 +158,7 @@ func TestDataSourceArg_UnresolvableContextSuppresses(t *testing.T) { } { t.Run(ds.Type+" "+ds.Reference, func(t *testing.T) { page := []*ast.WidgetV3{dvWidget("dv", ds, dsWidget("dgUnknown", "DS.DS_Rows"))} - if errs := validateDataSourceArgumentsIn(nil, page, f147Sigs(), sameName); len(errs) != 0 { + if errs := validateFlowArgumentsIn(nil, page, f147Sigs(), sameName); len(errs) != 0 { t.Errorf("an unresolvable context must not be reported as a mismatch: %v", errs) } }) @@ -178,7 +178,7 @@ func TestDataSourceArg_PrimitiveParameterIsAlwaysReported(t *testing.T) { dsWidget("dgSearch", "DS.DS_Search"), ), } - if errs := validateDataSourceArgumentsIn(nil, page, sigs, sameName); len(errs) != 1 { + if errs := validateFlowArgumentsIn(nil, page, sigs, sameName); len(errs) != 1 { t.Fatalf("a String parameter is never filled in from a data context; got %d errors: %v", len(errs), errs) } @@ -192,7 +192,7 @@ func TestDataSourceArg_UnknownArgumentIsReportedInsideAContext(t *testing.T) { dsWidget("dgTypo", "DS.DS_Rows", "Contxet"), ), } - errs := validateDataSourceArgumentsIn(nil, page, f147Sigs(), sameName) + errs := validateFlowArgumentsIn(nil, page, f147Sigs(), sameName) if len(errs) != 1 || !strings.Contains(errs[0], "'Contxet'") { t.Fatalf("the unknown-argument half must survive; got %v", errs) } @@ -207,7 +207,7 @@ func TestDataSourceArg_ExplicitArgumentInsideAContextIsClean(t *testing.T) { dsWidget("dgOK", "DS.DS_Rows", "Context"), ), } - if errs := validateDataSourceArgumentsIn(nil, page, f147Sigs(), sameName); len(errs) != 0 { + if errs := validateFlowArgumentsIn(nil, page, f147Sigs(), sameName); len(errs) != 0 { t.Errorf("explicit arguments reported: %v", errs) } } @@ -223,7 +223,7 @@ func TestDataSourceArg_AWidgetDoesNotSupplyItsOwnArgument(t *testing.T) { }, } page := []*ast.WidgetV3{dsWidget("dgSelf", "DS.DS_Self")} - if errs := validateDataSourceArgumentsIn(nil, page, sigs, sameName); len(errs) != 1 { + if errs := validateFlowArgumentsIn(nil, page, sigs, sameName); len(errs) != 1 { t.Fatalf("a widget must not supply its own argument; got %d errors: %v", len(errs), errs) } } @@ -231,7 +231,7 @@ func TestDataSourceArg_AWidgetDoesNotSupplyItsOwnArgument(t *testing.T) { // The message must no longer assert what measurement falsified. func TestDataSourceArg_MessageDoesNotClaimMendixNeverFillsItIn(t *testing.T) { page := []*ast.WidgetV3{dsWidget("dgLoose", "DS.DS_Rows")} - errs := validateDataSourceArgumentsIn(nil, page, f147Sigs(), sameName) + errs := validateFlowArgumentsIn(nil, page, f147Sigs(), sameName) if len(errs) != 1 { t.Fatalf("got %v", errs) } @@ -255,7 +255,7 @@ func TestDataSourceArg_DatabaseContextResolves(t *testing.T) { dsWidget("dgLines", "DS.DS_Lines"), ), } - if errs := validateDataSourceArgumentsIn(nil, page, sigs, sameName); len(errs) != 0 { + if errs := validateFlowArgumentsIn(nil, page, sigs, sameName); len(errs) != 0 { t.Errorf("a database data source of the parameter's entity supplies it: %v", errs) } } @@ -275,11 +275,11 @@ func TestDataSourceArg_CompatibilityIsDelegated(t *testing.T) { return sameName(ctxQN, paramQN) || (strings.EqualFold(ctxQN, "HR.Employee") && strings.EqualFold(paramQN, "HR.Person")) } - if errs := validateDataSourceArgumentsIn(nil, page, sigs, compat); len(errs) != 0 { + if errs := validateFlowArgumentsIn(nil, page, sigs, compat); len(errs) != 0 { t.Errorf("a specialization in context satisfies a generalization parameter: %v", errs) } // Control: without the generalization it is a mismatch. - if errs := validateDataSourceArgumentsIn(nil, page, sigs, sameName); len(errs) != 1 { + if errs := validateFlowArgumentsIn(nil, page, sigs, sameName); len(errs) != 1 { t.Errorf("the control did not report the mismatch: %v", errs) } } diff --git a/mdl/executor/validate_flow_args_action_test.go b/mdl/executor/validate_flow_args_action_test.go new file mode 100644 index 0000000000..595bfec802 --- /dev/null +++ b/mdl/executor/validate_flow_args_action_test.go @@ -0,0 +1,295 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// mendixlabs/mxcli#1082: CE1571 raised on a CONTAINER's action while +// `mxcli check --references` said "Check passed!". +// +// The rule walked GetDataSource() only, so the identical missing-argument fault +// on an ACTION slot was silent. Each test below mirrors one row of the mxbuild +// 11.12.0 measurement in the file header; every "no error" case is paired with a +// control that still reports, so a green run cannot come from the rule having +// been blunted. + +// actionWidget is a widget carrying one action slot under the given property. +func slotActionWidget(name, property, flowType, target string, args ...string) *ast.WidgetV3 { + a := &ast.ActionV3{Type: flowType, Target: target} + for _, arg := range args { + a.Args = append(a.Args, ast.FlowArgV3{Name: arg, Value: "$x"}) + } + return &ast.WidgetV3{ + Name: name, + Type: "container", + Properties: map[string]any{property: a}, + } +} + +// gridWidget is a data grid bound to an entity, with the children given. +func slotGridWidget(name, entity string, children ...*ast.WidgetV3) *ast.WidgetV3 { + return &ast.WidgetV3{ + Name: name, + Type: "datagrid", + Properties: map[string]any{"DataSource": &ast.DataSourceV3{Type: "database", Reference: entity}}, + Children: children, + } +} + +func slotWidget(name, typ string, children ...*ast.WidgetV3) *ast.WidgetV3 { + return &ast.WidgetV3{Name: name, Type: typ, Children: children} +} + +// a1082Sigs is one microflow taking one object parameter — the reported shape. +func a1082Sigs() map[string]*flowSignature { + return map[string]*flowSignature{ + "m.act_unlink": objSig("LogisticWhitelist", "M.LogisticWhitelist"), + "m.mf_onerow": {Returns: "M.LogisticWhitelist"}, + "m.mf_other": {Returns: "M.OtherThing"}, + } +} + +// THE REPORTED CASE, reduced: an action with no argument and nothing enclosing +// it. Measured CE1571; the rule was silent. +func TestActionArg_NoContextIsReported(t *testing.T) { + page := []*ast.WidgetV3{slotActionWidget("cLoose", "Action", "microflow", "M.ACT_UnLink")} + errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName) + if len(errs) != 1 { + t.Fatalf("got %d errors, want 1: %v", len(errs), errs) + } + if !strings.Contains(errs[0], "cLoose") || !strings.Contains(errs[0], "CE1571") { + t.Errorf("message should name the widget and CE1571: %s", errs[0]) + } + // The remedy has to be spellable from the message alone — the issue was + // filed because its author concluded the syntax did not exist. + if !strings.Contains(errs[0], "Action: microflow M.ACT_UnLink(LogisticWhitelist: $Value)") { + t.Errorf("message should spell the argument form: %s", errs[0]) + } +} + +// An explicit argument clears it — the fix the reporter needed, and proof the +// rule is not simply rejecting every action. +func TestActionArg_ExplicitArgumentIsClean(t *testing.T) { + page := []*ast.WidgetV3{ + slotActionWidget("cBound", "Action", "microflow", "M.ACT_UnLink", "LogisticWhitelist"), + } + if errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName); len(errs) != 0 { + t.Errorf("an explicitly bound action reported: %v", errs) + } +} + +// Measured: an enclosing data context of the parameter's type supplies an +// action's argument exactly as it supplies a data source's. +func TestActionArg_MatchingEnclosingContextSuppliesIt(t *testing.T) { + page := []*ast.WidgetV3{ + dvWidget("dvW", mfDS("M.MF_OneRow"), + slotActionWidget("cM1", "Action", "microflow", "M.ACT_UnLink"), + ), + } + if errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName); len(errs) != 0 { + t.Errorf("mxbuild accepts this page; the check must not reject it: %v", errs) + } +} + +// …and a context of a DIFFERENT type does not. The control for the test above. +func TestActionArg_MismatchedContextIsReported(t *testing.T) { + page := []*ast.WidgetV3{ + dvWidget("dvO", mfDS("M.MF_Other"), + slotActionWidget("cM2", "Action", "microflow", "M.ACT_UnLink"), + ), + } + errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName) + if len(errs) != 1 { + t.Fatalf("got %d errors, want 1: %v", len(errs), errs) + } + if !strings.Contains(errs[0], "M.OtherThing") { + t.Errorf("message should name the enclosing context type: %s", errs[0]) + } +} + +// A page PARAMETER of the exact type is not a data context — the same +// distinction the data-source half rests on, re-measured for the action. +func TestActionArg_PageParameterAloneDoesNotSupplyIt(t *testing.T) { + page := []*ast.WidgetV3{slotActionWidget("cM3", "Action", "microflow", "M.ACT_UnLink")} + params := []ast.PageParameter{ + {Name: "W", EntityType: ast.QualifiedName{Module: "M", Name: "LogisticWhitelist"}}, + } + if errs := validateFlowArgumentsIn(params, page, a1082Sigs(), sameName); len(errs) != 1 { + t.Fatalf("a page parameter is not a data context (measured: CE1571); got %v", errs) + } +} + +// A data grid COLUMN is row-scoped, so the grid's object supplies the argument. +func TestActionArg_GridColumnIsRowScoped(t *testing.T) { + page := []*ast.WidgetV3{ + slotGridWidget("dgM4", "M.LogisticWhitelist", + slotWidget("colM4", "column", + slotActionWidget("cM4", "Action", "microflow", "M.ACT_UnLink"), + ), + ), + } + if errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName); len(errs) != 0 { + t.Errorf("a column is row-scoped (measured: no error); got %v", errs) + } +} + +// …and a CONTROL BAR is not. This is the reported shape, and the pair with the +// column test above is the whole point: the two differ only in which child of +// the same grid the widget sits in. +func TestActionArg_ControlBarIsNotRowScoped(t *testing.T) { + page := []*ast.WidgetV3{ + slotGridWidget("dgMaterials", "M.LogisticWhitelist", + slotWidget("cb1", "controlbar", + slotActionWidget("cM6", "Action", "microflow", "M.ACT_UnLink"), + ), + ), + } + errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName) + if len(errs) != 1 { + t.Fatalf("a control bar is not row-scoped (measured: CE1571); got %v", errs) + } + // The remedy differs from every other context miss, so the advice must too: + // nesting does not help here, the grid's selection does. + if !strings.Contains(errs[0], "$dgMaterials") { + t.Errorf("message should offer the grid's selection: %s", errs[0]) + } + if strings.Contains(errs[0], "nest the widget in a data container") { + t.Errorf("nesting is not the remedy inside a control bar: %s", errs[0]) + } +} + +// The control bar drops only the grid's OWN object. Measured: an outer context +// still reaches it (dataview > datagrid > controlbar > action → no error), so +// the walk must not hand the control bar an empty context. +func TestActionArg_OuterContextStillReachesTheControlBar(t *testing.T) { + page := []*ast.WidgetV3{ + dvWidget("dvM9", mfDS("M.MF_OneRow"), + slotGridWidget("dgM9", "M.LogisticWhitelist", + slotWidget("cbM9", "controlbar", + slotActionWidget("cM9", "Action", "microflow", "M.ACT_UnLink"), + ), + ), + ), + } + if errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName); len(errs) != 0 { + t.Errorf("an outer context reaches into a control bar (measured: no error); got %v", errs) + } +} + +// A data container INSIDE a control bar ends the control bar's reach: its own +// object is in scope for its children like anywhere else. +func TestActionArg_DataContainerInsideAControlBarRestoresScope(t *testing.T) { + page := []*ast.WidgetV3{ + slotGridWidget("dgInner", "M.LogisticWhitelist", + slotWidget("cbInner", "controlbar", + dvWidget("dvInner", mfDS("M.MF_OneRow"), + slotActionWidget("cInner", "Action", "microflow", "M.ACT_UnLink"), + ), + ), + ), + } + if errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName); len(errs) != 0 { + t.Errorf("a dataview inside a control bar establishes context: %v", errs) + } +} + +// The control-bar rule is in the shared WALK, not in either rule, so a DATA +// SOURCE placed in a control bar is reported too. Measured on the same project: +// the same dataview is CE1571 in the control bar and clean in a column. +func TestDataSourceArg_ControlBarIsNotRowScoped(t *testing.T) { + sigs := map[string]*flowSignature{"m.mf_needsone": objSig("LogisticWhitelist", "M.LogisticWhitelist")} + inControlBar := []*ast.WidgetV3{ + slotGridWidget("dgM7", "M.LogisticWhitelist", + slotWidget("cbM7", "controlbar", + dvWidget("dvM7", mfDS("M.MF_NeedsOne")), + ), + ), + } + if errs := validateFlowArgumentsIn(nil, inControlBar, sigs, sameName); len(errs) != 1 { + t.Fatalf("a data source in a control bar is CE1571 (measured); got %v", errs) + } + // CONTROL: the identical data source in a column is accepted. + inColumn := []*ast.WidgetV3{ + slotGridWidget("dgM8", "M.LogisticWhitelist", + slotWidget("colM8", "column", + dvWidget("dvM8", mfDS("M.MF_NeedsOne")), + ), + ), + } + if errs := validateFlowArgumentsIn(nil, inColumn, sigs, sameName); len(errs) != 0 { + t.Errorf("the control failed: a column is row-scoped; got %v", errs) + } +} + +// Every action slot is covered, not just the reported one. `OnClick:` is stored +// under "Action" by the visitor, so the distinct keys worth proving are +// OnChange and a pluggable widget's named slot. +func TestActionArg_EveryActionSlotIsChecked(t *testing.T) { + for _, property := range []string{"Action", "OnChange", "createFileAction"} { + t.Run(property, func(t *testing.T) { + page := []*ast.WidgetV3{slotActionWidget("w", property, "nanoflow", "M.ACT_UnLink")} + errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName) + if len(errs) != 1 { + t.Fatalf("slot %s not checked; got %v", property, errs) + } + if !strings.Contains(errs[0], "("+property+":)") { + t.Errorf("message should name the slot it is about: %s", errs[0]) + } + }) + } +} + +// A widget with two faulty slots reports both, in a stable order — the +// Properties map is iterated, and a map is not ordered. +func TestActionArg_TwoFaultySlotsReportDeterministically(t *testing.T) { + w := slotActionWidget("wBoth", "Action", "microflow", "M.ACT_UnLink") + w.Properties["OnChange"] = &ast.ActionV3{Type: "microflow", Target: "M.ACT_UnLink"} + page := []*ast.WidgetV3{w} + + first := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName) + if len(first) != 2 { + t.Fatalf("got %d errors, want 2: %v", len(first), first) + } + for i := 0; i < 20; i++ { + again := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName) + if strings.Join(again, "|") != strings.Join(first, "|") { + t.Fatalf("order is not stable across runs:\n%v\n%v", first, again) + } + } + if !strings.Contains(first[0], "(Action:)") { + t.Errorf("sorted by property key, Action comes first: %v", first) + } +} + +// The unknown-argument half applies to actions too: an argument naming no +// parameter binds nothing and is silent otherwise. +func TestActionArg_UnknownArgumentIsReported(t *testing.T) { + page := []*ast.WidgetV3{ + slotActionWidget("cTypo", "Action", "microflow", "M.ACT_UnLink", "LogisticWhitelist", "Whitelst"), + } + errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName) + if len(errs) != 1 || !strings.Contains(errs[0], "'Whitelst'") { + t.Fatalf("the unknown-argument half must cover actions; got %v", errs) + } +} + +// A non-flow action carries no parameters to miss, and an unresolved flow is +// validateWidgetReferences' business — neither may be reported here. +func TestActionArg_NonFlowAndUnknownFlowAreSilent(t *testing.T) { + page := []*ast.WidgetV3{ + {Name: "cSave", Type: "actionbutton", Properties: map[string]any{ + "Action": &ast.ActionV3{Type: "save", ClosePage: true}, + }}, + {Name: "cGone", Type: "actionbutton", Properties: map[string]any{ + "Action": &ast.ActionV3{Type: "microflow", Target: "M.DoesNotExist"}, + }}, + } + if errs := validateFlowArgumentsIn(nil, page, a1082Sigs(), sameName); len(errs) != 0 { + t.Errorf("neither a non-flow action nor an unresolved one belongs here: %v", errs) + } +} diff --git a/mdl/executor/validate_page_button_context.go b/mdl/executor/validate_page_button_context.go index 871ca00b64..2c302a3fb7 100644 --- a/mdl/executor/validate_page_button_context.go +++ b/mdl/executor/validate_page_button_context.go @@ -24,34 +24,47 @@ func ValidatePageButtonContext(prog *ast.Program) []linter.Violation { for _, stmt := range prog.Statements { switch s := stmt.(type) { case *ast.CreatePageStmtV3: - out = append(out, checkButtonContextTree(s.Widgets, false, "page "+s.Name.String())...) + out = append(out, checkButtonContextTree(s.Widgets, "", "page "+s.Name.String())...) case *ast.CreateSnippetStmtV3: - out = append(out, checkButtonContextTree(s.Widgets, false, "snippet "+s.Name.String())...) + out = append(out, checkButtonContextTree(s.Widgets, "", "snippet "+s.Name.String())...) } } return out } -func checkButtonContextTree(widgets []*ast.WidgetV3, underControlBar bool, locationPrefix string) []linter.Violation { +// checkButtonContextTree walks the widget tree, carrying the name of the data +// widget whose control bar it is inside ("" when it is not inside one). +// +// The name is carried rather than just a flag because it IS the remedy: a data +// widget's selection is addressed by the widget's own name, so `$dgMaterials` is +// only spellable from here. Advice that stops at "move it into a column" sends +// an author looking for syntax that does not need to exist — which is how +// mendixlabs/mxcli#1082 was filed. +func checkButtonContextTree(widgets []*ast.WidgetV3, controlBarOf, locationPrefix string) []linter.Violation { var out []linter.Violation for _, w := range widgets { if w == nil { continue } - if underControlBar { + if controlBarOf != "" { if a := w.GetAction(); a != nil { - out = append(out, checkControlBarAction(a, w.Name, locationPrefix)...) + out = append(out, checkControlBarAction(a, w.Name, controlBarOf, locationPrefix)...) } } - childUnder := underControlBar || strings.EqualFold(w.Type, "controlbar") - out = append(out, checkButtonContextTree(w.Children, childUnder, locationPrefix)...) + for _, c := range w.Children { + childOf := controlBarOf + if c != nil && strings.EqualFold(c.Type, "controlbar") { + childOf = w.Name + } + out = append(out, checkButtonContextTree([]*ast.WidgetV3{c}, childOf, locationPrefix)...) + } } return out } // checkControlBarAction flags any $currentObject argument on an action (and its // chained THEN action) that sits inside a control bar. -func checkControlBarAction(a *ast.ActionV3, widgetName, locationPrefix string) []linter.Violation { +func checkControlBarAction(a *ast.ActionV3, widgetName, controlBarOf, locationPrefix string) []linter.Violation { var out []linter.Violation for a != nil { for _, arg := range a.Args { @@ -62,7 +75,7 @@ func checkControlBarAction(a *ast.ActionV3, widgetName, locationPrefix string) [ Message: fmt.Sprintf( "%s: control-bar button `%s` passes $currentObject to its %s action, but a control bar is not row-scoped — $currentObject is unbound there (CE1571)", locationPrefix, widgetName, a.Type), - Suggestion: "Move the button into a grid column (row-scoped) so it has a current row, or pass a page parameter instead of $currentObject.", + Suggestion: controlBarSuggestion(controlBarOf), }) } } @@ -70,3 +83,18 @@ func checkControlBarAction(a *ast.ActionV3, widgetName, locationPrefix string) [ } return out } + +// controlBarSuggestion names the remedy that actually applies to a control bar. +// +// The selection comes first because it is the one that keeps the button where +// the author put it: a data widget with `Selection:` set exposes the selected +// object as `$`, which an action takes as an ordinary argument. +func controlBarSuggestion(controlBarOf string) string { + if controlBarOf == "" { + return "Move the button into a grid column (row-scoped) so it has a current row, or pass a page parameter instead of $currentObject." + } + return fmt.Sprintf( + "Pass the selection of `%s` instead — `Action: microflow M.F($Param = $%s)`, with `Selection:` set on the widget. "+ + "Or move the button into a grid column (row-scoped) so it has a current row, or pass a page parameter.", + controlBarOf, controlBarOf) +} diff --git a/mdl/executor/validate_page_button_context_test.go b/mdl/executor/validate_page_button_context_test.go index 597532d45a..6663d5fa73 100644 --- a/mdl/executor/validate_page_button_context_test.go +++ b/mdl/executor/validate_page_button_context_test.go @@ -67,3 +67,37 @@ func TestValidatePageButtonContext_ControlBarNoCurrentObjectClean(t *testing.T) t.Fatalf("control-bar button without $currentObject should be clean, got %v", msgs) } } + +// The advice has to name the remedy that applies where the button IS. +// mendixlabs/mxcli#1082 was filed by an author who followed the old suggestion, +// found neither a column nor a page parameter usable, and concluded MDL had no +// syntax for passing the grid's selection — which it has had all along. +func TestValidatePageButtonContext_SuggestionNamesTheGridSelection(t *testing.T) { + src := `create page P.Grid ( Title: 'Orders', Layout: Atlas_Core.Atlas_Default ) { + datagrid dgMaterials ( DataSource: database from P.Order, Selection: Single ) { + controlbar cb1 { + container cUnlink (Action: nanoflow P.ACT_UnLink ($Order = $currentObject)) { + actionbutton btnUnlink (Caption: 'Unlink') + } + } + column OrderNumber (Attribute: OrderNumber) { } + } +};` + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse errors: %v", errs) + } + vs := ValidatePageButtonContext(prog) + if len(vs) != 1 { + t.Fatalf("expected one violation, got %d: %v", len(vs), vs) + } + // The selection is addressed by the DATA WIDGET's name, so the suggestion is + // only useful if it carries that name through from the walk. + if !strings.Contains(vs[0].Suggestion, "$dgMaterials") { + t.Errorf("suggestion should name the grid's selection variable:\n%s", vs[0].Suggestion) + } + // …and the container is the widget at fault, not the button inside it. + if !strings.Contains(vs[0].Message, "cUnlink") { + t.Errorf("message should name the widget carrying the action:\n%s", vs[0].Message) + } +} diff --git a/mdl/executor/validate_workflow_activity_kind.go b/mdl/executor/validate_workflow_activity_kind.go new file mode 100644 index 0000000000..fb96622b67 --- /dev/null +++ b/mdl/executor/validate_workflow_activity_kind.go @@ -0,0 +1,312 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/sdk/workflows" +) + +// An activity's outcome list is TYPED, and the three inserting ALTER WORKFLOW +// ops each write exactly one outcome type into it: +// +// INSERT OUTCOME -> Workflows$UserTaskOutcome +// INSERT PATH -> Workflows$ParallelSplitOutcome +// INSERT CONDITION -> Workflows$BooleanConditionOutcome / VoidConditionOutcome / +// EnumerationValueConditionOutcome +// +// and `generated/metamodel` types the receiving list per activity: +// +// SingleUserTaskActivity.Outcomes []*WorkflowsUserTaskOutcome +// MultiUserTaskActivity.Outcomes []*WorkflowsUserTaskOutcome +// ExclusiveSplitActivity.Outcomes []*WorkflowsConditionOutcome +// CallMicroflowTask.Outcomes []*WorkflowsConditionOutcome +// ParallelSplitActivity.Outcomes []*WorkflowsParallelSplitOutcome +// +// None of the three ops checked what it was pointed at, so any of the six +// mismatches wrote an outcome the list cannot hold (ako/mxcli#415). Measured on +// mxbuild 11.10.0 against a project sitting at 0 errors, every one of them +// leaves a model Mendix cannot LOAD — `mx check` dies at "Loading the mpr file" +// before validating anything, and Studio Pro will not open the project: +// +// INSERT OUTCOME on a decision / parallel split -> System.InvalidCastException +// INSERT PATH / INSERT CONDITION on a wrong kind -> System.InvalidOperationException +// +// This is the MDL-WF04 class, and the remedy is the one that class always +// takes: refuse. Writing "the right type instead" is not available here — the +// author asked for a specific branch shape, and on a decision an added +// enumeration outcome would have to complete the whole value set or MxBuild +// reports CE6686 (MDL-WF06). There is already a correct op for each activity +// kind, so the refusal names it. +// +// BoundaryEvents is a second typed list on the same activities, declared by +// only four types (CallMicroflowTask, CallWorkflowActivity, MultiUserTask, +// SingleUserTask, WaitForNotification). That one did not corrupt: measured, an +// INSERT BOUNDARY EVENT on a decision printed "Altered workflow" and left the +// document byte-identical — a silent no-op rather than a load failure. It is +// refused here for the other reason: a statement that reports success and +// changes nothing is the "no silent side effects" rule, and the honest answer +// is that a decision has nowhere to put it. +// +// The DROP ops are NOT covered, and that is measured rather than assumed: +// removing an element cannot write a wrong type. DROP OUTCOME and DROP PATH on +// a decision each leave the project loadable at 1 ordinary build error, which +// is the correct outcome of removing a branch. + +// workflowOutcomeSlot is the list an ALTER WORKFLOW op writes into. +type workflowOutcomeSlot int + +const ( + slotUserTaskOutcome workflowOutcomeSlot = iota + slotParallelSplitOutcome + slotConditionOutcome + slotBoundaryEvent +) + +// workflowOpSlot describes one inserting op for the diagnostic. +type workflowOpSlot struct { + op string // the MDL keywords, as the author wrote them + slot workflowOutcomeSlot // which typed list it writes into + writes string // the BSON $Type it writes + accepts string // the activity kinds whose list holds that type +} + +var workflowOpSlots = map[workflowOutcomeSlot]workflowOpSlot{ + slotUserTaskOutcome: { + op: "INSERT OUTCOME", slot: slotUserTaskOutcome, + writes: "Workflows$UserTaskOutcome", accepts: "a user task", + }, + slotParallelSplitOutcome: { + op: "INSERT PATH", slot: slotParallelSplitOutcome, + writes: "Workflows$ParallelSplitOutcome", accepts: "a parallel split", + }, + slotConditionOutcome: { + op: "INSERT CONDITION", slot: slotConditionOutcome, + writes: "Workflows$…ConditionOutcome", accepts: "a decision or a call microflow", + }, + slotBoundaryEvent: { + op: "INSERT BOUNDARY EVENT", slot: slotBoundaryEvent, + writes: "a boundary event", accepts: "a user task, call microflow, call workflow or wait for notification", + }, +} + +// activityAcceptsSlot reports whether this activity's document declares the +// list the op writes into. Typed against the semantic model rather than a table +// of storage-name strings, so a new activity kind cannot silently inherit the +// wrong answer by being absent from a list. +func activityAcceptsSlot(a workflows.WorkflowActivity, slot workflowOutcomeSlot) bool { + switch a.(type) { + case *workflows.UserTask: + return slot == slotUserTaskOutcome || slot == slotBoundaryEvent + case *workflows.ExclusiveSplitActivity: + return slot == slotConditionOutcome + case *workflows.CallMicroflowTask, *workflows.SystemTask: + return slot == slotConditionOutcome || slot == slotBoundaryEvent + case *workflows.ParallelSplitActivity: + return slot == slotParallelSplitOutcome + case *workflows.CallWorkflowActivity, *workflows.WaitForNotificationActivity: + return slot == slotBoundaryEvent + } + // An activity kind this build does not model (GenericWorkflowActivity, or + // one added by a newer Mendix) is left alone: refusing on a kind we cannot + // reason about would reject scripts that are fine. + return true +} + +// opForActivity names the inserting op that DOES write into this activity's +// outcome list. A refusal that only says what the author's op takes leaves them +// to work out the remedy; every one of these targets has a correct op. +func opForActivity(a workflows.WorkflowActivity) string { + switch a.(type) { + case *workflows.UserTask: + return "INSERT OUTCOME ''" + case *workflows.ExclusiveSplitActivity, *workflows.CallMicroflowTask, *workflows.SystemTask: + return "INSERT CONDITION ''" + case *workflows.ParallelSplitActivity: + return "INSERT PATH" + } + return "" +} + +// describeActivityKind names an activity the way an author would. +func describeActivityKind(a workflows.WorkflowActivity) string { + switch t := a.(type) { + case *workflows.UserTask: + if t.IsMulti { + return "a multi user task" + } + return "a user task" + case *workflows.ExclusiveSplitActivity: + return "a decision" + case *workflows.CallMicroflowTask, *workflows.SystemTask: + return "a call microflow" + case *workflows.ParallelSplitActivity: + return "a parallel split" + case *workflows.CallWorkflowActivity: + return "a call workflow" + case *workflows.WaitForNotificationActivity: + return "a wait for notification" + } + return "a " + a.ActivityType() + " activity" +} + +// validateAlterWorkflowActivityKinds refuses an inserting op aimed at an +// activity whose document has nowhere to put what the op writes. +// +// It is deliberately silent when the target cannot be resolved: an activity +// this pass cannot find is one the mutator reports on, with better information +// (not found, or ambiguous with the @N remedy). Refusing on a failed lookup +// would turn a resolution problem into a type problem and reject valid scripts. +func validateAlterWorkflowActivityKinds(ctx *ExecContext, s *ast.AlterWorkflowStmt) []string { + wf := findStoredWorkflow(ctx, s.Name) + if wf == nil || wf.Flow == nil { + return nil + } + + var errs []string + check := func(ref string, atPos int, slot workflowOutcomeSlot) { + target := resolveStoredActivity(wf.Flow, ref, atPos) + if target == nil || activityAcceptsSlot(target, slot) { + return + } + spec := workflowOpSlots[slot] + kind := describeActivityKind(target) + remedy := fmt.Sprintf("%s takes %s", spec.op, spec.accepts) + if alt := opForActivity(target); alt != "" && slot != slotBoundaryEvent { + remedy = fmt.Sprintf("use `%s ON %s { … }` to add a branch to %s (%s takes %s)", + alt, ref, kind, spec.op, spec.accepts) + } + errs = append(errs, fmt.Sprintf( + "%s on '%s' is refused: it writes %s, and %s cannot hold one — the project would not LOAD "+ + "(Studio Pro will not open it and `mx check` dies before validating anything); %s", + spec.op, ref, spec.writes, kind, remedy)) + } + + for _, op := range s.Operations { + switch o := op.(type) { + case *ast.InsertOutcomeOp: + check(o.ActivityRef, o.AtPosition, slotUserTaskOutcome) + case *ast.InsertPathOp: + check(o.ActivityRef, o.AtPosition, slotParallelSplitOutcome) + case *ast.InsertBranchOp: + check(o.ActivityRef, o.AtPosition, slotConditionOutcome) + case *ast.InsertBoundaryEventOp: + check(o.ActivityRef, o.AtPosition, slotBoundaryEvent) + } + } + return errs +} + +// findStoredWorkflow returns the stored workflow for a qualified name, or nil. +func findStoredWorkflow(ctx *ExecContext, qn ast.QualifiedName) *workflows.Workflow { + if ctx == nil || !ctx.Connected() { + return nil + } + all, err := ctx.Backend.ListWorkflows() + if err != nil { + return nil + } + h, err := getHierarchy(ctx) + if err != nil { + return nil + } + for _, wf := range all { + if wf.Name != qn.Name { + continue + } + if h.GetModuleName(h.FindModuleID(wf.ContainerID)) == qn.Module { + return wf + } + } + return nil +} + +// resolveStoredActivity mirrors the mutator's own addressing +// (wfmutator.findActivityByCaption): match on Name or Caption, search nested +// flows too, and use @N to pick among several. It returns nil rather than +// guessing when the reference is ambiguous, so the mutator's message wins. +func resolveStoredActivity(flow *workflows.Flow, ref string, atPos int) workflows.WorkflowActivity { + var matches []workflows.WorkflowActivity + collectStoredActivities(flow, ref, &matches) + switch { + case len(matches) == 0: + return nil + case atPos > 0: + if atPos > len(matches) { + return nil + } + return matches[atPos-1] + case len(matches) == 1: + return matches[0] + } + return nil +} + +func collectStoredActivities(flow *workflows.Flow, ref string, out *[]workflows.WorkflowActivity) { + if flow == nil { + return + } + for _, a := range flow.Activities { + if a == nil { + continue + } + if a.GetName() == ref || a.GetCaption() == ref { + *out = append(*out, a) + } + for _, nested := range storedNestedFlows(a) { + collectStoredActivities(nested, ref, out) + } + } +} + +// storedNestedFlows returns every sub-flow an activity owns. Condition outcomes +// go through the ConditionOutcome interface rather than a type switch over the +// three implementations — the same reasoning as the auto-bind walk, where a +// per-variant switch is how the enumeration branch got missed (ako/mxcli#417). +func storedNestedFlows(a workflows.WorkflowActivity) []*workflows.Flow { + var out []*workflows.Flow + add := func(f *workflows.Flow) { + if f != nil { + out = append(out, f) + } + } + switch t := a.(type) { + case *workflows.UserTask: + for _, o := range t.Outcomes { + add(o.Flow) + } + for _, b := range t.BoundaryEvents { + add(b.Flow) + } + case *workflows.ExclusiveSplitActivity: + for _, o := range t.Outcomes { + add(o.GetFlow()) + } + case *workflows.CallMicroflowTask: + for _, o := range t.Outcomes { + add(o.GetFlow()) + } + for _, b := range t.BoundaryEvents { + add(b.Flow) + } + case *workflows.SystemTask: + for _, o := range t.Outcomes { + add(o.GetFlow()) + } + case *workflows.ParallelSplitActivity: + for _, o := range t.Outcomes { + add(o.Flow) + } + case *workflows.CallWorkflowActivity: + for _, b := range t.BoundaryEvents { + add(b.Flow) + } + case *workflows.WaitForNotificationActivity: + for _, b := range t.BoundaryEvents { + add(b.Flow) + } + } + return out +} diff --git a/mdl/executor/validate_workflow_refs.go b/mdl/executor/validate_workflow_refs.go index ab6f5ff182..350fd1ebf8 100644 --- a/mdl/executor/validate_workflow_refs.go +++ b/mdl/executor/validate_workflow_refs.go @@ -225,5 +225,11 @@ func validateAlterWorkflowRefs(ctx *ExecContext, s *ast.AlterWorkflowStmt, sc *s } } } + // An inserting op must also be aimed at an activity whose document can hold + // what it writes — ako/mxcli#415, where it could not and the project stopped + // loading. Same function for both passes, so `check --references` and `exec` + // cannot drift. + errs = append(errs, validateAlterWorkflowActivityKinds(ctx, s)...) + return append(errs, validateWorkflowReferences(ctx, added, sc)...) } diff --git a/mdl/executor/validate_xpath_association.go b/mdl/executor/validate_xpath_association.go new file mode 100644 index 0000000000..a9cf9c04fc --- /dev/null +++ b/mdl/executor/validate_xpath_association.go @@ -0,0 +1,269 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "regexp" + "sort" + "strings" + + "github.com/mendixlabs/mxcli/mdl/linter" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// An association named in an XPath constraint must be written QUALIFIED +// (`Module.Association`); an attribute is written bare. Get it wrong and mxcli +// stores the constraint faithfully, `mxcli check --references` passes, `exec` +// reports success, and mxbuild fails the build with +// +// ERROR at , Microflow '', Retrieve object(s) activity +// 'Retrieve list of from database': Error(s) in XPath constraint. +// +// Measured on a blank Mendix 11.14.0 app, with the qualification as the only +// variable between two runs of the same script: +// +// [Ticket_Reporter = $currentUser] check passed → build exit 3 +// [MyFirstModule.Ticket_Reporter = $currentUser] check passed → BUILD SUCCEEDED +// +// This is the expensive shape rather than a cosmetic one, because `exec` applies +// statements one at a time and cannot roll back: a script carrying this in the +// middle writes everything before it and stops at the build, leaving the model +// half-updated with nothing having reported a problem. +// +// The rule is deliberately narrow. It fires only on a bare name that is NOT an +// attribute of the constrained entity AND IS a known association, so it can say +// which spelling to use instead of merely suspecting one. A name that is neither +// is left alone — that is a different error (an unknown member), and guessing at +// it here would produce false positives on every XPath function and keyword. +// That same narrowness is why no keyword list is needed: `and`, `or`, `not` and +// `contains` are not association names, so they cannot match. + +// xpathIdentRe matches an identifier and the character before it, so the caller +// can reject a qualified tail (`.Name`) or a variable (`$var`). RE2 has no +// lookahead, so the character AFTER the identifier — which distinguishes a +// module prefix from a member — is inspected by index instead. +var xpathIdentRe = regexp.MustCompile(`[A-Za-z_][A-Za-z0-9_]*`) + +// xpathAssocHit is one bare association name found in a constraint. +type xpathAssocHit struct { + Name string // as written, e.g. "Ticket_Reporter" + Qualified string // what it must be, e.g. "MyFirstModule.Ticket_Reporter" +} + +// blankXPathLiterals replaces the contents of single-quoted string literals with +// spaces, preserving length so match indices still line up with the original. +// +// Without this an association name mentioned inside a literal is flagged, and +// the fix offered is one that would corrupt the literal. Mendix escapes a quote +// inside a literal by doubling it, so a closing quote immediately followed by +// another is an escape and not the end of the string. +func blankXPathLiterals(s string) string { + out := []byte(s) + inLiteral := false + for i := 0; i < len(out); i++ { + if out[i] == '\'' { + if inLiteral && i+1 < len(out) && out[i+1] == '\'' { + out[i+1] = ' ' // the escaped quote's second half + i++ + continue + } + inLiteral = !inLiteral + continue + } + if inLiteral { + out[i] = ' ' + } + } + return string(out) +} + +// unqualifiedAssociationsInConstraint returns the bare association names used in +// an XPath constraint. attrs holds the constrained entity's attribute names; +// assocs maps an unqualified association name to its qualified spelling. +// +// Attributes are checked first, so an association whose name collides with an +// attribute of THIS entity is read as the attribute — which is what Mendix does. +func unqualifiedAssociationsInConstraint(constraint string, attrs map[string]bool, assocs map[string]string) []xpathAssocHit { + if constraint == "" || len(assocs) == 0 { + return nil + } + scanned := blankXPathLiterals(constraint) + + var hits []xpathAssocHit + seen := map[string]bool{} + for _, loc := range xpathIdentRe.FindAllStringIndex(scanned, -1) { + start, end := loc[0], loc[1] + // A qualified tail (`Module.Name`) or a variable (`$var`) is already + // unambiguous; neither is a bare member reference. + if start > 0 && (scanned[start-1] == '.' || scanned[start-1] == '$') { + continue + } + // Followed by a dot, this is the MODULE half of a qualified name, not a + // member. It is the qualified spelling this rule asks for. + if end < len(scanned) && scanned[end] == '.' { + continue + } + name := scanned[start:end] + if attrs[name] || seen[name] { + continue + } + qualified, ok := assocs[name] + if !ok { + continue + } + seen[name] = true + hits = append(hits, xpathAssocHit{Name: name, Qualified: qualified}) + } + return hits +} + +// buildAssociationIndex maps every association's unqualified name to its +// qualified spelling. An association is stored in the FROM entity's module +// (see MDL070), so the domain model holding it names it. +// +// A name defined in more than one module is left OUT rather than guessed at: +// the rule's value is naming the right spelling, and offering one of two would +// be wrong half the time. Such a constraint still fails the build, but it fails +// with mxbuild's message rather than with our wrong advice. +func buildAssociationIndex(ctx *ExecContext) map[string]string { + modules, err := getModulesFromCache(ctx) + if err != nil { + return nil + } + moduleNames := make(map[model.ID]string, len(modules)) + for _, m := range modules { + moduleNames[m.ID] = m.Name + } + dms, err := ctx.Backend.ListDomainModels() + if err != nil { + return nil + } + index := map[string]string{} + ambiguous := map[string]bool{} + add := func(modName, name string) { + if modName == "" || name == "" { + return + } + qualified := modName + "." + name + if prev, ok := index[name]; ok && prev != qualified { + ambiguous[name] = true + return + } + index[name] = qualified + } + for _, dm := range dms { + modName := moduleNames[dm.ContainerID] + for _, a := range dm.Associations { + add(modName, a.Name) + } + for _, a := range dm.CrossAssociations { + add(modName, a.Name) + } + } + for name := range ambiguous { + delete(index, name) + } + return index +} + +// validateXPathAssociations flags a bare association name in a retrieve's XPath +// constraint — accepted by mxcli, rejected by the build. +// +// It reads the script's own declarations as well as the project's, because the +// common shape is ONE script that creates the entity, the association and the +// microflow that constrains on it. Seeing only stored associations would leave +// the rule silent on exactly the scripts that reach a build half-written, and +// silent with no project at all — which is how `mxcli check script.mdl` is most +// often run. +func validateXPathAssociations(ctx *ExecContext, retrieves []retrieveConstraintRef, sc *scriptContext) []string { + assocs := buildAssociationIndex(ctx) + if assocs == nil { + assocs = map[string]string{} + } + entities := buildEntityIndex(ctx) + + // Script declarations win over stored ones: within this script, that is the + // definition about to be in force. + if sc != nil { + for name, qualified := range sc.associations { + if !sc.ambiguousAssc[name] { + assocs[name] = qualified + } + } + for name := range sc.ambiguousAssc { + delete(assocs, name) + } + } + if len(assocs) == 0 { + return nil + } + + var errors []string + for _, r := range retrieves { + attrs, ok := entityAttrNames(entities, sc, r.entity) + if !ok { + continue // entity-not-found is reported separately + } + for _, hit := range unqualifiedAssociationsInConstraint(r.constraint, attrs, assocs) { + errors = append(errors, fmt.Sprintf( + "constraint on %s names the association %s unqualified — Mendix requires the qualified form and the build fails with "+ + "\"Error(s) in XPath constraint\" (CE0161). Write %s instead.", + r.entity, hit.Name, hit.Qualified)) + } + } + return errors +} + +// XPathAssociationViolations is the linter-facing form of the same rule, for +// callers that report violations rather than error strings. +func XPathAssociationViolations(entity, constraint string, attrs map[string]bool, assocs map[string]string) []linter.Violation { + var out []linter.Violation + for _, hit := range unqualifiedAssociationsInConstraint(constraint, attrs, assocs) { + out = append(out, linter.Violation{ + RuleID: xpathAssociationRule, + Severity: linter.SeverityError, + Location: linter.Location{ + Module: qualifiedNameModule(entity), + DocumentType: "entity", + DocumentName: strings.TrimPrefix(entity, qualifiedNameModule(entity)+"."), + }, + Message: fmt.Sprintf( + "XPath constraint on %s names the association %s unqualified; Mendix rejects this with CE0161", + entity, hit.Name), + Suggestion: fmt.Sprintf("Write %s instead of %s.", hit.Qualified, hit.Name), + }) + } + sort.SliceStable(out, func(i, j int) bool { return out[i].Message < out[j].Message }) + return out +} + +const xpathAssociationRule = "MDL-XPATH01" + +// entityAttrNames returns the constrained entity's attribute names, from the +// script when it declares the entity and from the project otherwise. ok is +// false when neither knows it — then the entity reference itself is the error, +// and this rule stays out of the way. +// +// The script is consulted FIRST because a CREATE OR MODIFY in the script is the +// shape that will be in force, attributes included. +func entityAttrNames(entities map[string]*domainmodel.Entity, sc *scriptContext, qualified string) (map[string]bool, bool) { + if sc != nil { + if attrs, ok := sc.entityAttrs[qualified]; ok { + return attrs, true + } + } + if entities == nil { + return nil, false + } + ent := entities[qualified] + if ent == nil { + return nil, false + } + attrs := make(map[string]bool, len(ent.Attributes)) + for _, a := range ent.Attributes { + attrs[a.Name] = true + } + return attrs, true +} diff --git a/mdl/executor/validate_xpath_association_test.go b/mdl/executor/validate_xpath_association_test.go new file mode 100644 index 0000000000..d186fde218 --- /dev/null +++ b/mdl/executor/validate_xpath_association_test.go @@ -0,0 +1,252 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// The constraints below are the real ones, from a reproduction on a blank +// Mendix 11.14.0 app. `Ticket_Reporter` is an association from +// MyFirstModule.Ticket to System.User; `Title` is one of Ticket's attributes. +// +// [Ticket_Reporter = $currentUser] check passed → build exit 3 +// [MyFirstModule.Ticket_Reporter = $currentUser] check passed → BUILD SUCCEEDED + +var ( + probeAttrs = map[string]bool{"Title": true, "Status": true} + probeAssocs = map[string]string{ + "Ticket_Reporter": "MyFirstModule.Ticket_Reporter", + "Ticket_Equipment": "MyFirstModule.Ticket_Equipment", + } +) + +func TestUnqualifiedAssociationsInConstraint(t *testing.T) { + tests := []struct { + name string + constraint string + want []string // the bare names expected, in order + }{ + { + name: "the reproduced failure", + constraint: "[Ticket_Reporter = $currentUser]", + want: []string{"Ticket_Reporter"}, + }, + { + name: "the qualified form is the fix, and must not fire", + constraint: "[MyFirstModule.Ticket_Reporter = $currentUser]", + want: nil, + }, + { + name: "bare association at the head of a traversal", + constraint: "[Ticket_Reporter/System.User/Name = 'ada']", + want: []string{"Ticket_Reporter"}, + }, + { + name: "a qualified traversal is clean", + constraint: "[MyFirstModule.Ticket_Reporter/System.User/Name = 'ada']", + want: nil, + }, + { + name: "two bare associations are both reported", + constraint: "[Ticket_Reporter = $currentUser and Ticket_Equipment != empty]", + want: []string{"Ticket_Reporter", "Ticket_Equipment"}, + }, + { + name: "the same one twice is reported once", + constraint: "[Ticket_Reporter = $a or Ticket_Reporter = $b]", + want: []string{"Ticket_Reporter"}, + }, + { + name: "an attribute of this entity is never an association", + constraint: "[Title = 'x' and Status = 'open']", + want: nil, + }, + { + name: "XPath keywords and functions cannot match an association name", + constraint: "[not(contains(Title, 'x')) and Status != empty]", + want: nil, + }, + { + name: "a Mendix token is left alone", + constraint: "[MyFirstModule.Ticket_Reporter = '[%CurrentUser%]']", + want: nil, + }, + { + name: "System members are another rule's business", + constraint: "[System.owner = '[%CurrentUser%]']", + want: nil, + }, + { + name: "empty constraint", + constraint: "", + want: nil, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + hits := unqualifiedAssociationsInConstraint(tc.constraint, probeAttrs, probeAssocs) + if len(hits) != len(tc.want) { + t.Fatalf("got %d hits %v, want %d %v", len(hits), assocHitNames(hits), len(tc.want), tc.want) + } + for i, w := range tc.want { + if hits[i].Name != w { + t.Errorf("hit %d = %q, want %q", i, hits[i].Name, w) + } + if !strings.HasSuffix(hits[i].Qualified, "."+w) { + t.Errorf("hit %d qualified = %q, must end in .%s", i, hits[i].Qualified, w) + } + } + }) + } +} + +// A name inside a string literal is not a member reference, and offering to +// qualify it would corrupt the literal. Mendix escapes a quote by doubling it, +// so the scanner has to track that or it loses its place and mis-reads the rest +// of the constraint — which is how a name OUTSIDE a literal gets missed. +func TestUnqualifiedAssociationsIgnoresStringLiterals(t *testing.T) { + tests := []struct { + name string + constraint string + want []string + }{ + { + name: "a name inside a literal is not a reference", + constraint: "[Title = 'Ticket_Reporter']", + want: nil, + }, + { + name: "a doubled quote is an escape, not the end of the literal", + constraint: "[Title = 'it''s Ticket_Reporter' and Ticket_Equipment != empty]", + want: []string{"Ticket_Equipment"}, + }, + { + name: "a real reference after a literal is still found", + constraint: "[Title = 'x' and Ticket_Reporter = $currentUser]", + want: []string{"Ticket_Reporter"}, + }, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + hits := unqualifiedAssociationsInConstraint(tc.constraint, probeAttrs, probeAssocs) + if len(hits) != len(tc.want) { + t.Fatalf("got %v, want %v", assocHitNames(hits), tc.want) + } + for i, w := range tc.want { + if hits[i].Name != w { + t.Errorf("hit %d = %q, want %q", i, hits[i].Name, w) + } + } + }) + } +} + +// blankXPathLiterals must preserve length, or every match index after the first +// literal points at the wrong character and the rule reports the wrong name. +func TestBlankXPathLiteralsPreservesLength(t *testing.T) { + for _, in := range []string{ + "[Title = 'abc']", + "[Title = 'it''s here' and X = 'y']", + "[Title = 'unterminated", + "", + } { + if got := blankXPathLiterals(in); len(got) != len(in) { + t.Errorf("blankXPathLiterals(%q) length %d, want %d", in, len(got), len(in)) + } + } +} + +// An ambiguous name is left out of the index on purpose, so the rule stays +// silent rather than offering one of two spellings and being wrong half the +// time. This asserts the silence, since a lookup miss is indistinguishable from +// a correct constraint at the call site. +func TestUnqualifiedAssociationsSilentWhenNameIsNotAKnownAssociation(t *testing.T) { + hits := unqualifiedAssociationsInConstraint( + "[Some_Ambiguous_Assoc = $currentUser]", probeAttrs, probeAssocs) + if len(hits) != 0 { + t.Errorf("must not flag a name that is not a known association, got %v", assocHitNames(hits)) + } +} + +func assocHitNames(hits []xpathAssocHit) []string { + out := make([]string, len(hits)) + for i, h := range hits { + out[i] = h.Name + } + return out +} + +// scriptContext has TWO parallel collectors over the same statement types — +// collectDefinitions (whole program, up front) and collectSingle (incremental). +// MDL-XPATH01 first shipped with its cases added to collectSingle only, so it +// fired against stored associations and stayed silent on script-declared ones, +// which is the shape the rule exists for. Neither the unit tests nor a +// project-backed run caught that: the rule was correct, it was simply never +// given the data. +// +// This asserts the two agree. It fails if a case is added to one and not the +// other, which is the only way the gap can reappear. +func TestBothCollectorsRecordAssociationsAndAttrs(t *testing.T) { + prog := &ast.Program{Statements: []ast.Statement{ + &ast.CreateEntityStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Ticket"}, + Attributes: []ast.Attribute{{Name: "Title"}, {Name: "Status"}}, + }, + &ast.CreateAssociationStmt{ + Name: ast.QualifiedName{Module: "M", Name: "Ticket_Equipment"}, + Parent: ast.QualifiedName{Module: "M", Name: "Ticket"}, + Child: ast.QualifiedName{Module: "M", Name: "Equipment"}, + }, + }} + + whole := newScriptContext() + whole.collectDefinitions(prog) + + incremental := newScriptContext() + for _, stmt := range prog.Statements { + incremental.collectSingle(stmt) + } + + for _, sc := range []struct { + label string + ctx *scriptContext + }{{"collectDefinitions", whole}, {"collectSingle", incremental}} { + if got := sc.ctx.associations["Ticket_Equipment"]; got != "M.Ticket_Equipment" { + t.Errorf("%s: associations[Ticket_Equipment] = %q, want M.Ticket_Equipment", sc.label, got) + } + attrs := sc.ctx.entityAttrs["M.Ticket"] + if !attrs["Title"] || !attrs["Status"] { + t.Errorf("%s: entityAttrs[M.Ticket] = %v, want Title and Status", sc.label, attrs) + } + } +} + +// The same association name in two modules is dropped rather than guessed at, +// on both collectors. +func TestAmbiguousAssociationNameIsDropped(t *testing.T) { + prog := &ast.Program{Statements: []ast.Statement{ + &ast.CreateAssociationStmt{Name: ast.QualifiedName{Module: "A", Name: "Shared_Link"}}, + &ast.CreateAssociationStmt{Name: ast.QualifiedName{Module: "B", Name: "Shared_Link"}}, + }} + sc := newScriptContext() + sc.collectDefinitions(prog) + if !sc.ambiguousAssc["Shared_Link"] { + t.Fatal("a name declared in two modules must be marked ambiguous") + } + // And the rule must then stay silent on it — offering one of two spellings + // would be wrong half the time. + assocs := map[string]string{} + for name, q := range sc.associations { + if !sc.ambiguousAssc[name] { + assocs[name] = q + } + } + if hits := unqualifiedAssociationsInConstraint("[Shared_Link = $x]", nil, assocs); len(hits) != 0 { + t.Errorf("must stay silent on an ambiguous name, got %v", assocHitNames(hits)) + } +} diff --git a/mdl/executor/webservice_names.go b/mdl/executor/webservice_names.go new file mode 100644 index 0000000000..b97a9a4250 --- /dev/null +++ b/mdl/executor/webservice_names.go @@ -0,0 +1,259 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/mdl/backend" +) + +// A SOAP call stores TWO names for the service it calls, and they are not the +// same string: +// +// ImportedService "Clients.OrderSoapClient" the DOCUMENT, qualified +// ServiceName "OrdersWS" the WSDL +// +// Both engines derived the second from the first by taking the part after the +// last dot, which is right only when someone happened to name the document after +// the service. Mendix resolves the operation WITHIN the named service, so when +// they differ the call does not merely look odd — it fails to validate. Measured +// on Mendix 11.14.0 against ako/TestApp, whose baseline is 0 errors: +// +// [CE0386] "Operation 'GetOrder' does not exist in consumed web service +// 'Clients.OrderSoapClient'." +// +// GetOrder does exist. Mendix looked for it inside a service called +// "OrderSoapClient", which the WSDL does not define. +// +// The real name is in the imported service document, in structured form — no +// WSDL parsing needed. `Description.Services[]` holds WebServices$ServiceInfoImpl +// entries, each with a Name and an Operations list of +// WebServices$OperationInfoImpl. + +// importedServiceType is the stored $Type of an imported (consumed) SOAP +// service, in FULL — including the `Impl` suffix. +// +// Two things measured on ako/TestApp, both of which cost a debugging round: +// +// - ListRawUnitsByType matches the type EXACTLY, despite its parameter being +// called typePrefix. "WebServices$ImportedServiceImpl" returns the document; +// "WebServices$ImportedService" and "WebServices" both return nothing. +// - The type is `ImportedServiceImpl`, not `ImportedWebService`. The name +// modelsdk/gen and generated/metamodel both use — WebServices$ImportedWebService +// — is the SDK name; nothing is stored under it. A DESCRIBE-side resolver +// asked for that one and therefore matched nothing, which went unnoticed +// because its fallback was already the right answer (see +// TestFormatAction_WebServiceCallRendersStoredQualifiedNames); it has since +// been removed. +const importedServiceType = "WebServices$ImportedServiceImpl" + +// resolveWebServiceName returns the WSDL service name for the imported service +// document named by qualifiedName, and the operation names it declares. +// +// It returns "" when the answer cannot be established — an unresolvable +// document, a backend that cannot list raw units, a document whose shape does +// not match, or a name that matches more than one document. The caller then +// falls back to the old derivation: a wrong ServiceName is no worse than the one +// shipping today, and an invented one would be. +// +// The match is on the document's BARE name. A raw unit carries its container id +// rather than a module name, and resolving that needs the ExecContext the flow +// builder does not hold; ambiguity is refused instead of resolved, which costs a +// fallback in the rare two-modules-same-name case and never picks the wrong +// service. +func resolveWebServiceName(b backend.FullBackend, qualifiedName, operationName string) string { + if b == nil || qualifiedName == "" { + return "" + } + units, err := b.ListRawUnitsByType(importedServiceType) + if err != nil || len(units) == 0 { + return "" + } + _, bare, ok := strings.Cut(qualifiedName, ".") + if !ok || bare == "" { + bare = qualifiedName + } + + var matched []byte + for _, unit := range units { + if unit == nil || len(unit.Contents) == 0 { + continue + } + if !strings.EqualFold(rawUnitName(unit.Contents), bare) { + continue + } + if matched != nil { + return "" // ambiguous — two documents of this name + } + matched = unit.Contents + } + if matched == nil { + return "" + } + return serviceNameFromImportedService(matched, operationName) +} + +// serviceNameFromImportedService reads Description.Services[] and returns the +// name of the service to call. +// +// A WSDL may define more than one service, so the operation decides: the service +// DECLARING it is the one Mendix resolves against. With one service the operation +// is not consulted, and with none — or an operation no service declares — the +// answer is "" rather than a guess, since guessing reproduces CE0386 with a +// different name in it. +func serviceNameFromImportedService(contents []byte, operationName string) string { + var doc map[string]any + if err := bson.Unmarshal(contents, &doc); err != nil { + return "" + } + services := typedArrayElements(docLookup(doc["Description"], "Services")) + if len(services) == 0 { + return "" + } + + names := make([]string, 0, len(services)) + for _, svc := range services { + name, _ := docLookup(svc, "Name").(string) + if name == "" { + continue + } + names = append(names, name) + if operationName != "" && serviceDeclaresOperation(svc, operationName) { + return name + } + } + if len(names) == 1 { + return names[0] + } + return "" +} + +// serviceDeclaresOperation reports whether a WebServices$ServiceInfoImpl lists an +// operation of this name. +func serviceDeclaresOperation(svc any, operationName string) bool { + for _, op := range typedArrayElements(docLookup(svc, "Operations")) { + if name, _ := docLookup(op, "Name").(string); strings.EqualFold(name, operationName) { + return true + } + } + return false +} + +// typedArrayElements drops the leading version marker from a Mendix typed array +// and returns the elements. A value that is not an array yields none. +func typedArrayElements(v any) []any { + var arr []any + switch a := v.(type) { + case bson.A: + arr = a + case []any: + arr = a + default: + return nil + } + if len(arr) == 0 { + return nil + } + switch arr[0].(type) { + case int32, int64, int: + return arr[1:] + } + return arr +} + +// docLookup reads a key from a BSON sub-document whichever way the driver +// decoded it. +// +// This is not defensive dressing: a unit unmarshalled into map[string]any nests +// its sub-documents as map[string]any, while the same bytes decoded into a +// bson.D nest as bson.D. Asserting only the second silently found nothing, and +// "found nothing" here is indistinguishable from "no such service" — it fell +// back to the wrong name instead of failing. +func docLookup(v any, key string) any { + switch d := v.(type) { + case map[string]any: + return d[key] + case bson.D: + for _, e := range d { + if e.Key == key { + return e.Value + } + } + } + return nil +} + +// importMappingType is the stored $Type of an import mapping document. As with +// importedServiceType, this is matched EXACTLY — and note it is +// `ImportMappings$ImportMapping`, not the `Mappings$…` prefix its child elements +// use (Mappings$ObjectMappingElement also appears in these documents). +const importMappingType = "ImportMappings$ImportMapping" + +// resolveImportMappingEntity returns the qualified entity an import mapping +// produces — the Entity of its root ObjectMappingElement. +// +// It is what Mendix stores as the SOAP call's result VariableType. Writing +// DataTypes$VoidType instead (which both engines did) tells Mendix the call +// returns nothing, and assigning that to a variable is two errors at once, +// measured on 11.14.0 against ako/TestApp: +// +// [CE0243] "The mapping used to return a value of type 'Nothing', but now +// returns a value of type 'Clients.Order'." +// [CE0366] "Cannot store in variable when there is no return value." +// +// "" when it cannot be established, and the writers then keep VoidType — which +// is wrong, but is what ships today, so an unresolvable mapping is no worse off. +func resolveImportMappingEntity(b backend.FullBackend, qualifiedName string) string { + if b == nil || qualifiedName == "" { + return "" + } + units, err := b.ListRawUnitsByType(importMappingType) + if err != nil || len(units) == 0 { + return "" + } + _, bare, ok := strings.Cut(qualifiedName, ".") + if !ok || bare == "" { + bare = qualifiedName + } + + var matched []byte + for _, unit := range units { + if unit == nil || len(unit.Contents) == 0 { + continue + } + if !strings.EqualFold(rawUnitName(unit.Contents), bare) { + continue + } + if matched != nil { + return "" // ambiguous — refuse, as with the service lookup + } + matched = unit.Contents + } + if matched == nil { + return "" + } + return rootMappingEntity(matched) +} + +// rootMappingEntity reads Elements[] and returns the first object mapping +// element's Entity. +// +// The root is the only element whose entity the CALL is typed on; the children +// are value mappings and nested objects, which belong to the mapping's own +// structure rather than to the result. +func rootMappingEntity(contents []byte) string { + var doc map[string]any + if err := bson.Unmarshal(contents, &doc); err != nil { + return "" + } + for _, el := range typedArrayElements(doc["Elements"]) { + entity, _ := docLookup(el, "Entity").(string) + if entity != "" { + return entity + } + } + return "" +} diff --git a/mdl/executor/webservice_names_test.go b/mdl/executor/webservice_names_test.go new file mode 100644 index 0000000000..67dea5de82 --- /dev/null +++ b/mdl/executor/webservice_names_test.go @@ -0,0 +1,242 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/mock" + mdltypes "github.com/mendixlabs/mxcli/mdl/types" + "go.mongodb.org/mongo-driver/bson" +) + +// importedServiceDoc builds a WebServices$ImportedServiceImpl the way the driver +// hands it back: sub-documents nested as maps once the unit is unmarshalled into +// map[string]any, and each collection prefixed with its typed-array marker. +func importedServiceDoc(t *testing.T, docName string, services ...bson.M) []byte { + t.Helper() + arr := bson.A{int32(2)} + for _, s := range services { + arr = append(arr, s) + } + out, err := bson.Marshal(bson.M{ + "$Type": importedServiceType, + "Name": docName, + "Description": bson.M{ + "$Type": "WebServices$WsdlDescriptionImpl", + "Services": arr, + }, + }) + if err != nil { + t.Fatalf("marshal: %v", err) + } + return out +} + +func serviceInfo(name string, operations ...string) bson.M { + ops := bson.A{int32(2)} + for _, op := range operations { + ops = append(ops, bson.M{"$Type": "WebServices$OperationInfoImpl", "Name": op}) + } + return bson.M{"$Type": "WebServices$ServiceInfoImpl", "Name": name, "Operations": ops} +} + +func backendWithUnits(units ...[]byte) *mock.MockBackend { + return &mock.MockBackend{ + ListRawUnitsByTypeFunc: func(typeName string) ([]*mdltypes.RawUnit, error) { + out := make([]*mdltypes.RawUnit, 0, len(units)) + for _, u := range units { + out = append(out, &mdltypes.RawUnit{Type: typeName, Contents: u}) + } + return out, nil + }, + } +} + +// TestResolveWebServiceName is the regression test for CE0386. +// +// A SOAP call stores the WSDL service name, which is not the local part of the +// imported service's qualified name. Deriving it — what both engines did — made +// Mendix look for the operation inside a service that does not exist: +// +// [CE0386] "Operation 'GetOrder' does not exist in consumed web service +// 'Clients.OrderSoapClient'." +// +// measured on 11.14.0 against ako/TestApp, whose document is named +// OrderSoapClient and whose WSDL service is OrdersWS. +func TestResolveWebServiceName(t *testing.T) { + b := backendWithUnits(importedServiceDoc(t, "OrderSoapClient", + serviceInfo("OrdersWS", "GetOrder", "SaveOrder"))) + + if got := resolveWebServiceName(b, "Clients.OrderSoapClient", "GetOrder"); got != "OrdersWS" { + t.Errorf("resolveWebServiceName = %q, want OrdersWS (the WSDL service, not the document)", got) + } + // A single service answers even when the operation is not one of its own — + // the operation only has to break ties. + if got := resolveWebServiceName(b, "Clients.OrderSoapClient", "Unknown"); got != "OrdersWS" { + t.Errorf("single-service lookup = %q, want OrdersWS", got) + } +} + +// TestResolveWebServiceName_PicksTheServiceDeclaringTheOperation — a WSDL may +// define several services, and Mendix resolves the operation within one of them. +func TestResolveWebServiceName_PicksTheServiceDeclaringTheOperation(t *testing.T) { + b := backendWithUnits(importedServiceDoc(t, "MultiClient", + serviceInfo("OrdersWS", "GetOrder"), + serviceInfo("CustomersWS", "GetCustomer"))) + + if got := resolveWebServiceName(b, "M.MultiClient", "GetCustomer"); got != "CustomersWS" { + t.Errorf("= %q, want CustomersWS", got) + } + if got := resolveWebServiceName(b, "M.MultiClient", "GetOrder"); got != "OrdersWS" { + t.Errorf("= %q, want OrdersWS", got) + } + // Two services and an operation neither declares: refuse rather than pick + // one. Guessing here just reproduces CE0386 with a different name in it. + if got := resolveWebServiceName(b, "M.MultiClient", "Nope"); got != "" { + t.Errorf("= %q, want \"\" (ambiguous, must fall back)", got) + } +} + +// TestResolveWebServiceName_UnresolvableIsEmpty — every way the answer cannot be +// established returns "", so the writers fall back to the derivation that ships +// today. A wrong name is no worse than the current one; an invented one is. +func TestResolveWebServiceName_UnresolvableIsEmpty(t *testing.T) { + doc := importedServiceDoc(t, "OrderSoapClient", serviceInfo("OrdersWS", "GetOrder")) + + for _, tc := range []struct { + name string + b *mock.MockBackend + qn string + }{ + {"no such document", backendWithUnits(doc), "Clients.SomethingElse"}, + {"no documents at all", backendWithUnits(), "Clients.OrderSoapClient"}, + {"empty qualified name", backendWithUnits(doc), ""}, + {"document declares no services", backendWithUnits(importedServiceDoc(t, "Bare")), "M.Bare"}, + // Two documents of the same bare name: the raw unit carries a container + // id, not a module, so the module cannot be checked here. Refuse. + {"ambiguous document name", backendWithUnits(doc, doc), "Clients.OrderSoapClient"}, + } { + t.Run(tc.name, func(t *testing.T) { + if got := resolveWebServiceName(tc.b, tc.qn, "GetOrder"); got != "" { + t.Errorf("= %q, want \"\"", got) + } + }) + } + + if got := resolveWebServiceName(nil, "Clients.OrderSoapClient", "GetOrder"); got != "" { + t.Errorf("nil backend = %q, want \"\"", got) + } +} + +// TestResolveWebServiceName_ReadsMapDecodedDocuments pins the shape trap that +// cost a debugging round: a unit unmarshalled into map[string]any nests its +// sub-documents as maps, not bson.D. A lookup asserting only bson.D found +// nothing — and "found nothing" is indistinguishable here from "no such +// service", so it fell back to the wrong name instead of failing. +func TestResolveWebServiceName_ReadsMapDecodedDocuments(t *testing.T) { + raw := importedServiceDoc(t, "OrderSoapClient", serviceInfo("OrdersWS", "GetOrder")) + + var asMap map[string]any + if err := bson.Unmarshal(raw, &asMap); err != nil { + t.Fatalf("unmarshal: %v", err) + } + if _, isMap := asMap["Description"].(map[string]any); !isMap { + t.Fatalf("Description decoded as %T; this test no longer pins the trap it was written for", asMap["Description"]) + } + if got := serviceNameFromImportedService(raw, "GetOrder"); got != "OrdersWS" { + t.Errorf("serviceNameFromImportedService = %q, want OrdersWS", got) + } +} + +// TestTypedArrayElements — a Mendix typed array leads with an int32 version +// marker, which is not an element. +func TestTypedArrayElements(t *testing.T) { + if got := typedArrayElements(bson.A{int32(2), "a", "b"}); len(got) != 2 { + t.Errorf("marker not dropped: %v", got) + } + if got := typedArrayElements(bson.A{int32(2)}); len(got) != 0 { + t.Errorf("empty typed array = %v, want none", got) + } + if got := typedArrayElements([]any{int32(3), "a"}); len(got) != 1 { + t.Errorf("[]any form not handled: %v", got) + } + if got := typedArrayElements("not an array"); got != nil { + t.Errorf("non-array = %v, want nil", got) + } +} + +// importMappingDoc builds an ImportMappings$ImportMapping whose root object +// mapping element names an entity. +func importMappingDoc(t *testing.T, name, rootEntity string) []byte { + t.Helper() + elements := bson.A{int32(2)} + if rootEntity != "" { + elements = append(elements, bson.M{ + "$Type": "ImportMappings$ObjectMappingElement", + "Entity": rootEntity, + // A value child, to prove the root is what is read rather than the + // first element carrying any key at all. + "Children": bson.A{int32(2), bson.M{ + "$Type": "ImportMappings$ValueMappingElement", + "Attribute": rootEntity + ".SomeAttr", + }}, + }) + } + out, err := bson.Marshal(bson.M{ + "$Type": importMappingType, "Name": name, "Elements": elements, + }) + if err != nil { + t.Fatalf("marshal: %v", err) + } + return out +} + +// TestResolveImportMappingEntity is the regression test for CE0243/CE0366. +// +// The call's result VariableType is the entity the receive mapping produces. +// Both engines wrote DataTypes$VoidType — "returns nothing" — which contradicts +// the mapping and makes assigning the result an error of its own. Measured on +// 11.14.0 against ako/TestApp: writing Void gave +// +// [CE0243] "The mapping used to return a value of type 'Nothing', but now +// returns a value of type 'Clients.Order'." +// [CE0366] "Cannot store in variable when there is no return value." +// +// and both cleared once the entity was read off the mapping. +func TestResolveImportMappingEntity(t *testing.T) { + b := backendWithUnits(importMappingDoc(t, "SoapOrdersImportMapping", "Clients.Order")) + + if got := resolveImportMappingEntity(b, "Clients.SoapOrdersImportMapping"); got != "Clients.Order" { + t.Errorf("resolveImportMappingEntity = %q, want Clients.Order", got) + } +} + +// TestResolveImportMappingEntity_UnresolvableIsEmpty — every way the entity +// cannot be established returns "", and the writers then keep VoidType. That is +// still wrong, but it is what ships today: an unresolvable mapping must not be +// made worse, and must not be guessed at. +func TestResolveImportMappingEntity_UnresolvableIsEmpty(t *testing.T) { + doc := importMappingDoc(t, "SoapOrdersImportMapping", "Clients.Order") + + for _, tc := range []struct { + name string + b *mock.MockBackend + qn string + }{ + {"no such mapping", backendWithUnits(doc), "Clients.Missing"}, + {"no mappings at all", backendWithUnits(), "Clients.SoapOrdersImportMapping"}, + {"empty name", backendWithUnits(doc), ""}, + {"mapping has no root entity", backendWithUnits(importMappingDoc(t, "Bare", "")), "M.Bare"}, + {"ambiguous name", backendWithUnits(doc, doc), "Clients.SoapOrdersImportMapping"}, + } { + t.Run(tc.name, func(t *testing.T) { + if got := resolveImportMappingEntity(tc.b, tc.qn); got != "" { + t.Errorf("= %q, want \"\"", got) + } + }) + } + if got := resolveImportMappingEntity(nil, "Clients.SoapOrdersImportMapping"); got != "" { + t.Errorf("nil backend = %q, want \"\"", got) + } +} diff --git a/sdk/microflows/microflows_actions.go b/sdk/microflows/microflows_actions.go index bd53b6d8f1..b2a1275845 100644 --- a/sdk/microflows/microflows_actions.go +++ b/sdk/microflows/microflows_actions.go @@ -781,12 +781,23 @@ type WebServiceCallAction struct { ErrorHandlingType ErrorHandlingType `json:"errorHandlingType,omitempty"` RawBSON []byte `json:"-"` ServiceID model.ID `json:"serviceId,omitempty"` - OperationName string `json:"operationName,omitempty"` - SendMappingID model.ID `json:"sendMappingId,omitempty"` - ReceiveMappingID model.ID `json:"receiveMappingId,omitempty"` - OutputVariable string `json:"outputVariable,omitempty"` - UseReturnVariable bool `json:"useReturnVariable"` - TimeoutExpression string `json:"timeoutExpression,omitempty"` + // ServiceName is the WSDL , which is NOT the local + // part of ServiceID's qualified name — Mendix stores both and resolves the + // operation within this one. Resolved from the imported service document by + // the executor (see resolveWebServiceName); empty means it could not be + // established, and the writers fall back to deriving it from ServiceID. + ServiceName string `json:"serviceName,omitempty"` + OperationName string `json:"operationName,omitempty"` + SendMappingID model.ID `json:"sendMappingId,omitempty"` + ReceiveMappingID model.ID `json:"receiveMappingId,omitempty"` + OutputVariable string `json:"outputVariable,omitempty"` + UseReturnVariable bool `json:"useReturnVariable"` + TimeoutExpression string `json:"timeoutExpression,omitempty"` + // ResultEntity is the qualified entity the RECEIVE mapping produces, which + // Mendix stores as the call's result VariableType. Resolved from the mapping + // document by the executor; empty means it could not be established and the + // writers keep DataTypes$VoidType. + ResultEntity string `json:"resultEntity,omitempty"` } func (WebServiceCallAction) isMicroflowAction() {} diff --git a/sdk/mpr/writer_microflow_actions.go b/sdk/mpr/writer_microflow_actions.go index b05edcd03b..c46223142a 100644 --- a/sdk/mpr/writer_microflow_actions.go +++ b/sdk/mpr/writer_microflow_actions.go @@ -776,10 +776,18 @@ func serializeWebServiceCallAction(a *microflows.WebServiceCallAction) bson.D { } } - // ServiceName: use the local part of the qualified name (after the last dot). - serviceName := string(a.ServiceID) - if idx := strings.LastIndex(serviceName, "."); idx >= 0 { - serviceName = serviceName[idx+1:] + // ServiceName is the WSDL , which Mendix resolves the + // operation within — NOT the local part of the qualified document name. The + // executor reads the real one off the imported service document. Deriving it + // (the fallback here, and what this writer always did) is right only when the + // document happens to be named after the service; otherwise the call fails + // with CE0386 "Operation … does not exist in consumed web service …". + serviceName := a.ServiceName + if serviceName == "" { + serviceName = string(a.ServiceID) + if idx := strings.LastIndex(serviceName, "."); idx >= 0 { + serviceName = serviceName[idx+1:] + } } doc := bson.D{ @@ -817,7 +825,12 @@ func serializeWebServiceCallAction(a *microflows.WebServiceCallAction) bson.D { {Key: "$ID", Value: idToBsonBinary(GenerateID())}, {Key: "$Type", Value: "Microflows$ImportMappingCall"}, {Key: "Commit", Value: "YesWithoutEvents"}, - {Key: "ContentType", Value: "Json"}, + // Xml, not Json: a SOAP response IS XML. Studio Pro writes "Xml" + // here in both reference calls that carry an import mapping + // (ako/TestApp, Clients.GetOrders and GetCustomerOrders, 11.14.0). + // This is the receive side only — the REST and import-from-mapping + // ImportMappingCalls elsewhere in this file are unrelated. + {Key: "ContentType", Value: "Xml"}, {Key: "ForceSingleOccurrence", Value: false}, {Key: "ObjectHandlingBackup", Value: "Create"}, {Key: "ParameterVariableName", Value: ""}, @@ -831,12 +844,24 @@ func serializeWebServiceCallAction(a *microflows.WebServiceCallAction) bson.D { } else { resultHandling = append(resultHandling, bson.E{Key: "ImportMappingCall", Value: nil}) } + // VariableType is the type the call RETURNS — the entity the receive mapping + // produces. VoidType says it returns nothing, which contradicts the mapping + // (CE0243) and makes assigning the result an error too (CE0366). It stays the + // fallback for a mapping mxcli could not resolve. + variableType := bson.D{ + {Key: "$ID", Value: idToBsonBinary(GenerateID())}, + {Key: "$Type", Value: "DataTypes$VoidType"}, + } + if a.ResultEntity != "" { + variableType = bson.D{ + {Key: "$ID", Value: idToBsonBinary(GenerateID())}, + {Key: "$Type", Value: "DataTypes$ObjectType"}, + {Key: "Entity", Value: a.ResultEntity}, + } + } resultHandling = append(resultHandling, bson.E{Key: "ResultVariableName", Value: a.OutputVariable}, - bson.E{Key: "VariableType", Value: bson.D{ - {Key: "$ID", Value: idToBsonBinary(GenerateID())}, - {Key: "$Type", Value: "DataTypes$VoidType"}, - }}, + bson.E{Key: "VariableType", Value: variableType}, ) doc = append(doc, bson.E{Key: "NewResultHandling", Value: resultHandling})