Skip to content

refactor: implement mutation backends and migrate handlers#237

Merged
ako merged 5 commits intomendixlabs:mainfrom
retran:feature/mutation-backends
Apr 20, 2026
Merged

refactor: implement mutation backends and migrate handlers#237
ako merged 5 commits intomendixlabs:mainfrom
retran:feature/mutation-backends

Conversation

@retran
Copy link
Copy Markdown
Contributor

@retran retran commented Apr 19, 2026

Why

PR #236 defined the mutation interfaces — this PR provides the MPR-backed implementations that actually perform the BSON manipulation. The goal is to move all BSON mutation logic out of the executor and behind the backend abstraction, so the executor becomes a thin orchestrator.

What changed (incremental from PR #236)

28 files changed, +4,511/-3,523

New backend implementations:

  • mdl/backend/mpr/page_mutator.go (1,554 lines) — full PageMutator implementation: widget tree traversal, property updates, widget add/drop/move, layout changes, pluggable widget operations, all operating on BSON documents
  • mdl/backend/mpr/workflow_mutator.go (771 lines) — full WorkflowMutator implementation: activity insertion/removal, outcome management, path operations, boundary events
  • mdl/backend/mpr/widget_builder.go (1,007 lines) — widget construction and serialization: WidgetObjectBuilder, property type resolution, default widget scaffolding

Executor simplification:

  • cmd_alter_page.go: 1,721 → 256 lines — rewritten as thin orchestrator calling PageMutator methods
  • cmd_alter_workflow.go: 887 → 178 lines — same pattern with WorkflowMutator
  • widget_engine.go refactored to use WidgetObjectBuilder interface instead of direct BSON construction
  • Deleted from executor: widget_defaults.go, widget_operations.go, widget_templates.go (logic moved to backend)

Interface integration from #236:

Tests relocated: ALTER PAGE and ALTER WORKFLOW tests moved from executor to mdl/backend/mpr/ where the implementation now lives.

Review-driven improvements (post-PR creation):

  • NewWidgetRegistryWithOps(extraOps) — reintroduces extensibility injection point for user-defined widget operations that was lost when OperationRegistry was replaced by a fixed knownOperations set. NewWidgetRegistry() remains the zero-config default.
  • Round-trip and wrapper tests for convert/unconvert layer in mdl/backend/mpr/convert_roundtrip_test.go — 33 tests covering all forward conversion, reverse unconvert, nil/error passthrough, and Slice/Ptr wrapper functions.

Stack

PR 3/5 in the shared-types refactoring stack. This is the largest PR in the stack.
Merge order: #232#235#236this#238#239

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR continues the shared-types refactor by moving Mendix project value types and mutation/serialization responsibilities into backend interfaces, reducing direct sdk/mpr / BSON coupling in the executor and improving testability via mock backends.

Changes:

  • Introduces mdl/types (WASM-safe shared types) and migrates multiple handlers/readers from sdk/mpr types to mdl/types.
  • Adds backend interfaces for page/workflow mutation and widget serialization/building, and wires the executor to use them.
  • Adds extensive mock-based tests for executor commands and updates widget engine/registry validation accordingly.

Reviewed changes

Copilot reviewed 117 out of 137 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
mdl/types/doc.go Adds package docs clarifying purpose and conversion layer location.
mdl/types/asyncapi.go Adds AsyncAPI 2.x parsing into WASM-safe shared types.
mdl/linter/rules/page_navigation_security.go Switches navigation menu traversal to mdl/types menu item types.
mdl/executor/widget_templates.go Removes executor-side BSON widget template helpers (moved behind backend).
mdl/executor/widget_registry_test.go Updates tests to match new operation validation approach.
mdl/executor/widget_registry.go Replaces OperationRegistry with a fixed knownOperations set for validation.
mdl/executor/widget_operations.go Removes executor-side operation registry and operation implementations.
mdl/executor/widget_engine_test.go Updates tests to validate knownOperations and removes registry tests.
mdl/executor/widget_defaults.go Removes executor-side object-list default population (moved behind backend).
mdl/executor/mock_test_helpers_test.go Adds shared mock context/model factories + assertion helpers for executor tests.
mdl/executor/helpers.go Migrates folder info + ID generation to mdl/types.
mdl/executor/executor.go Migrates cached unit/folder info to mdl/types.
mdl/executor/cmd_workflows_write.go Migrates workflow ID generation to mdl/types.
mdl/executor/cmd_workflows_mock_test.go Adds mock-based tests for workflow show/describe.
mdl/executor/cmd_settings_mock_test.go Adds mock-based tests for settings show/describe.
mdl/executor/cmd_security_write.go Migrates security member-access and revocation types to mdl/types.
mdl/executor/cmd_security_mock_test.go Adds mock-based tests for multiple security commands.
mdl/executor/cmd_rest_clients_mock_test.go Adds mock-based tests for REST client show/describe.
mdl/executor/cmd_rename.go Migrates rename hit types to mdl/types.
mdl/executor/cmd_published_rest_mock_test.go Adds mock-based tests for published REST services.
mdl/executor/cmd_pages_mock_test.go Adds mock-based tests for pages/snippets/layouts listing.
mdl/executor/cmd_pages_create_v3.go Wires widgetBackend into the page builder for V3 creation.
mdl/executor/cmd_pages_builder_v3_pluggable.go Uses bsonutil IDs and backend widget serialization hook.
mdl/executor/cmd_pages_builder_v3_layout.go Migrates layout/widget ID generation to mdl/types.
mdl/executor/cmd_pages_builder_input_filters.go Uses bsonutil and mdl/types IDs for filter widget BSON creation.
mdl/executor/cmd_pages_builder_input_cloning_test.go Updates cloning tests to use bsonutil IDs.
mdl/executor/cmd_pages_builder_input_cloning.go Replaces sdk/mpr ID helpers with bsonutil/mdl/types.
mdl/executor/cmd_pages_builder_input.go Removes executor-side datasource serialization and uses bsonutil IDs for refs.
mdl/executor/cmd_pages_builder.go Adds widget backend dependency and migrates folder info caching to mdl/types.
mdl/executor/cmd_odata_mock_test.go Adds mock-based tests for OData commands.
mdl/executor/cmd_odata.go Switches EDMX parsing to mdl/types and ID generation to mdl/types.
mdl/executor/cmd_notconnected_mock_test.go Adds mock-based “not connected” tests for many commands.
mdl/executor/cmd_navigation_mock_test.go Adds mock-based tests for navigation show/describe.
mdl/executor/cmd_navigation.go Migrates navigation document/spec types to mdl/types.
mdl/executor/cmd_modules_mock_test.go Adds mock-based tests for modules listing.
mdl/executor/cmd_misc_mock_test.go Adds mock-based tests for version output.
mdl/executor/cmd_microflows_mock_test.go Adds mock-based tests for microflows/nanoflows show/describe.
mdl/executor/cmd_microflows_create.go Migrates microflow IDs to mdl/types.
mdl/executor/cmd_microflows_builder_workflow.go Migrates workflow-related microflow builder IDs to mdl/types.
mdl/executor/cmd_microflows_builder_graph.go Migrates graph builder IDs to mdl/types.
mdl/executor/cmd_microflows_builder_flows.go Migrates sequence flow IDs to mdl/types.
mdl/executor/cmd_microflows_builder_control.go Migrates split/merge/loop IDs to mdl/types.
mdl/executor/cmd_microflows_builder_annotations.go Migrates annotation and event IDs to mdl/types.
mdl/executor/cmd_mermaid_mock_test.go Adds mock-based test for Mermaid domain model rendering.
mdl/executor/cmd_jsonstructures_mock_test.go Adds mock-based tests for JSON structures listing.
mdl/executor/cmd_jsonstructures.go Migrates JSON structure types/helpers to mdl/types.
mdl/executor/cmd_javascript_actions_mock_test.go Adds mock-based tests for JavaScript actions.
mdl/executor/cmd_javaactions_mock_test.go Adds mock-based tests for Java actions.
mdl/executor/cmd_javaactions.go Migrates Java action ID generation to mdl/types.
mdl/executor/cmd_import_mappings_mock_test.go Adds mock-based tests for import mappings listing.
mdl/executor/cmd_import_mappings.go Migrates JSON element types (schema alignment) to mdl/types.
mdl/executor/cmd_imagecollections_mock_test.go Adds mock-based tests for image collections show/describe.
mdl/executor/cmd_imagecollections.go Migrates image collection types to mdl/types.
mdl/executor/cmd_fragments_mock_test.go Adds mock-based tests for fragments listing.
mdl/executor/cmd_folders.go Migrates folder info usage to mdl/types.
mdl/executor/cmd_export_mappings_mock_test.go Adds mock-based tests for export mappings listing.
mdl/executor/cmd_export_mappings.go Migrates JSON element types (schema alignment) to mdl/types.
mdl/executor/cmd_enumerations_mock_test.go Refactors enumeration tests to common mock helpers and adds describe tests.
mdl/executor/cmd_entities_mock_test.go Adds mock-based tests for entities listing.
mdl/executor/cmd_entities.go Migrates entity-related ID generation to mdl/types.
mdl/executor/cmd_diff_local.go Moves microflow parsing from sdk/mpr to backend method.
mdl/executor/cmd_dbconnection_mock_test.go Adds mock-based tests for database connections show/describe.
mdl/executor/cmd_datatransformer_mock_test.go Adds mock-based tests for data transformers list/describe.
mdl/executor/cmd_constants_mock_test.go Adds mock-based tests for constants show/describe.
mdl/executor/cmd_businessevents_mock_test.go Adds mock-based tests for business event service commands.
mdl/executor/cmd_businessevents.go Migrates ID generation to mdl/types.
mdl/executor/cmd_associations_mock_test.go Adds mock-based tests for association listing.
mdl/executor/cmd_agenteditor_mock_test.go Adds mock-based tests for agent editor commands.
mdl/catalog/builder_references.go Migrates navigation menu item reference extraction to mdl/types.
mdl/catalog/builder_navigation.go Migrates navigation indexing functions to mdl/types.
mdl/catalog/builder_contract.go Migrates EDMX/AsyncAPI parsing to mdl/types.
mdl/catalog/builder.go Updates CatalogReader interface to use mdl/types shared structs.
mdl/bsonutil/bsonutil.go Adds BSON-safe UUID helpers without sdk/mpr dependency.
mdl/backend/workflow.go Migrates image collection types to mdl/types.
mdl/backend/security.go Migrates member access + revocation types to mdl/types.
mdl/backend/navigation.go Migrates navigation types/specs to mdl/types.
mdl/backend/mutation.go Adds mutation + widget builder/serialization backend interfaces.
mdl/backend/mock/mock_workflow.go Updates mock backend to use mdl/types image collection types.
mdl/backend/mock/mock_security.go Updates mock backend to use mdl/types revocation types.
mdl/backend/mock/mock_navigation.go Updates mock backend to use mdl/types navigation types/specs.
mdl/backend/mock/mock_mutation.go Adds mock methods for new mutation/serialization/builder interfaces.
mdl/backend/mock/mock_module.go Updates folder listing mocks to mdl/types.
mdl/backend/mock/mock_microflow.go Adds mock hook for backend microflow parsing from raw BSON maps.
mdl/backend/mock/mock_mapping.go Updates JSON structure types to mdl/types.
mdl/backend/mock/mock_java.go Updates Java/JavaScript action types to mdl/types.
mdl/backend/mock/mock_infrastructure.go Updates raw unit + rename + widget type mocks to mdl/types; makes agent editor create/delete overrideable.
mdl/backend/mock/mock_connection.go Migrates version/project version types to mdl/types.
mdl/backend/mock/backend.go Extends mock backend surface to cover new interfaces and mdl/types types.
mdl/backend/microflow.go Adds ParseMicroflowFromRaw to backend API.
mdl/backend/mapping.go Migrates mapping backend JSON structure types to mdl/types.
mdl/backend/java.go Migrates Java backend list/read API types to mdl/types.
mdl/backend/infrastructure.go Migrates rename/raw unit/metadata/widget type APIs to mdl/types.
mdl/backend/doc.go Updates docs to reflect mdl/types extraction.
mdl/backend/connection.go Migrates connection + folder backend types to mdl/types.
mdl/backend/backend.go Extends FullBackend to include mutation + serialization + widget builder backends.
cmd/mxcli/project_tree.go Migrates menu tree building to mdl/types.NavMenuItem.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread mdl/executor/cmd_pages_builder.go
Comment thread mdl/executor/cmd_pages_builder_v3_pluggable.go
Comment thread mdl/types/asyncapi.go Outdated
Comment thread mdl/types/asyncapi.go Outdated
Comment thread mdl/types/asyncapi.go Outdated
Comment thread mdl/executor/widget_registry.go Outdated
Comment thread mdl/types/asyncapi.go
@retran retran marked this pull request as draft April 19, 2026 18:09
@retran retran force-pushed the feature/mutation-backends branch 2 times, most recently from 0adddbc to 55b21eb Compare April 20, 2026 08:01
@ako
Copy link
Copy Markdown
Collaborator

ako commented Apr 20, 2026

Code Review

Overview

This is the largest PR in the stack (+12,710/-5,703 lines, 28 files) and delivers the actual implementation behind the interfaces defined in PR #236. The executor's cmd_alter_page.go shrinks from 1,721 → 256 lines and cmd_alter_workflow.go from 887 → 178 lines. Both become thin orchestrators. The BSON mutation logic moves to page_mutator.go (1,554 lines), workflow_mutator.go (771 lines), and widget_builder.go (1,007 lines) in mdl/backend/mpr/. This is a well-executed refactoring overall.

Note: This PR is currently in DRAFT state.


Positive Aspects

  • Compile-time interface satisfaction checks are present. Both page_mutator.go and workflow_mutator.go have var _ backend.PageMutator = (*mprPageMutator)(nil) guards — exactly what was flagged as missing in the PR refactor: define mutation backend interfaces #236 review.
  • BSON tree walking is thorough. findBsonWidget and findBsonWidgetInSnippet correctly handle all container types: FormCall, LayoutGrid (rows/columns), TabContainer, DataView (Widgets + FooterWidgets), ControlBar, CustomWidget (pluggable properties). No container type is obviously missing.
  • Widget builder uses templates, not hand-constructed BSON. widget_builder.go loads pre-built templates via widgets.GetTemplateFullBSON() and modifies them — which is the safe approach established in the codebase.
  • Thin orchestrators are genuinely thin. cmd_alter_page.go is a clean dispatcher with no BSON logic. cmd_alter_workflow.go maps all 10+ operation types through a single switch and delegates every case to a mutator call.
  • page_mutator_test.go (725 lines) has solid coverage — widget tree walking, drop/set operations, scope extraction, BSON array marker preservation.

Issues & Concerns

1. Potential method visibility issue in workflow_mutator.go — verify before merge

The analysis flagged that the MprBackend factory method may be defined with a lowercase name (openWorkflowForMutation vs the interface's OpenWorkflowForMutation). If real, this would be a compile error. Since this is a DRAFT it may be intentional/WIP — but worth explicitly verifying the method name matches the interface before promoting to ready-for-review.

2. Workflow mutation operations are undertested

workflow_mutator_test.go (484 lines) covers property setting, activity finding, and drop operations. It does not test:

  • InsertAfterActivity
  • ReplaceActivity
  • InsertOutcome / DropOutcome
  • InsertPath / DropPath
  • InsertBranch / DropBranch
  • InsertBoundaryEvent
  • SetActivityProperty (only one test for DUE_DATE)

These are the most complex mutation operations and the most likely to have edge case bugs (e.g., position handling, sub-flow construction, deduplication). The corresponding page_mutator_test.go has better coverage of its operations.

3. BSON helpers are duplicated across three files

bson_helpers.go (executor-private), page_mutator.go, and workflow_mutator.go all define the same low-level helpers (dGet, dGetDoc, dSet, dGetArrayElements, etc.). This is pragmatic but creates a maintenance risk — a fix to one copy won't propagate. These belong in mdl/bsonutil or a shared internal package.

4. No MockPageMutator / MockWorkflowMutator types

mock_mutation.go mocks the backend factory methods but doesn't provide types that implement the mutator interfaces themselves. The test files construct mprPageMutator/mprWorkflowMutator directly (testing the real implementation), which is fine for backend tests. But executor-level tests that want to test cmd_alter_page.go logic in isolation (without BSON roundtrips) have no mock mutator to inject.

5. SetLayout (96 lines in page_mutator.go) warrants close review

The parameter remapping logic when changing a page's layout is the most complex method in the mutator. It correctly handles old/new layout qualified names and argument list remapping, but the complexity makes it a candidate for dedicated tests — none exist currently.


Checklist Against Project Standards

Item Status
Interface satisfaction compile-time checks ✅ Present in both mutator files
Thin orchestrators ✅ Both executor files genuinely delegate all logic
BSON container type coverage ✅ All known container types handled
Widget builder uses templates ✅ No hand-constructed BSON
page_mutator_test.go coverage ✅ Tree walking, drops, properties, scope
workflow_mutator_test.go coverage ⚠️ Missing insert/replace/outcome/path/branch tests
BSON helper deduplication ⚠️ Three copies across executor and backend
Mock mutator types ⚠️ Factory mocked; mutator interface not mockable

Verdict

Strong work overall — the refactoring goal is achieved. The executor is genuinely decoupled from BSON, the implementations are thorough, and the page-side tests are solid. The main gaps before this moves out of DRAFT are: (1) verify the OpenWorkflowForMutation method name; (2) add tests for the write-path workflow operations (insert/replace/outcomes/paths/branches) — these are the riskiest operations and currently have zero test coverage.

Implement PageMutator, WorkflowMutator, and WidgetBuilderBackend in
mdl/backend/mpr/. Rewrite ALTER PAGE (1721→256 lines) and ALTER
WORKFLOW (887→178 lines) as thin orchestrators using mutator sessions.
Implement PluggableWidgetEngine with WidgetObjectBuilder interface,
eliminating all BSON from widget_engine.go.

- Create mdl/backend/mpr/page_mutator.go (1554 lines)
- Create mdl/backend/mpr/workflow_mutator.go (771 lines)
- Create mdl/backend/mpr/widget_builder.go (1007 lines)
- Migrate SerializeWidget/ClientAction/DataSource to backend interface
- Add ParseMicroflowFromRaw to MicroflowBackend interface
- Delete widget_operations.go, widget_templates.go, widget_defaults.go
- Move ALTER PAGE/WORKFLOW tests to backend/mpr/ package
@retran retran force-pushed the feature/mutation-backends branch 2 times, most recently from 96b12d4 to a2d0752 Compare April 20, 2026 09:51
@github-actions
Copy link
Copy Markdown

AI Code Review

Critical Issues

  • None found

Moderate Issues

  • None found

Minor Issues

  • None found

What Looks Good

  • The refactor successfully moves BSON mutation logic out of the executor and into the backend abstraction layer as intended
  • Executor simplification is significant (cmd_alter_page.go: 1,721 → 256 lines, cmd_alter_workflow.go: 887 → 178 lines) showing effective thin orchestrator pattern
  • Test coverage is strong with new convert_roundtrip_test.go containing 33 tests covering forward/reverse conversion, error passthrough, and wrapper functions
  • Proper error handling with contextual wrapping using fmt.Errorf("%w", err) throughout
  • Uses interfaces and types defined in PR refactor: define mutation backend interfaces #236 (ContainerKind, InsertPosition, PluggablePropertyOp)
  • Test relocation is appropriate - ALTER PAGE/WORKFLOW tests moved to backend where implementation now lives
  • WidgetBuilderBackend interface integration is complete with mock implementation
  • No duplication of existing functionality - this is a clean refactor moving existing logic to new location
  • Code follows established patterns for BSON manipulation and widget tree traversal
  • Maintains full-stack consistency - executor still orchestrates but delegates to backend implementations

Recommendation

Approve. This PR is a well-executed refactor that successfully implements the mutation backends as described in PR #236, moves BSON manipulation logic behind the backend abstraction, simplifies executors to thin orchestrators, relocates tests appropriately, and adds comprehensive test coverage for the conversion layer. The changes are focused, consistent with project patterns, and maintain all existing functionality while improving architecture.


Automated review via OpenRouter (Nemotron Super 120B) — workflow source

@retran
Copy link
Copy Markdown
Contributor Author

retran commented Apr 20, 2026

Thanks for the thorough review! Addressing each point:

#1 (Method visibility): Verified — no issue. OpenWorkflowForMutation (exported) is defined on MprBackend in backend.go:741 and delegates to openWorkflowForMutation (unexported) in workflow_mutator.go:36. Both exist, compiles and tests pass.

#2 (Workflow mutation tests): Fixed in 3b67e04. Added 48+ tests to workflow_mutator_test.go covering all previously untested operations: InsertAfterActivity, ReplaceActivity, InsertOutcome/DropOutcome, InsertPath/DropPath, InsertBranch/DropBranch (true/false/default/enum conditions), InsertBoundaryEvent (InterruptingTimer, NonInterruptingTimer, Timer with activities), and expanded SetActivityProperty (PAGE, DESCRIPTION, TARGETING_MICROFLOW, TARGETING_XPATH) plus SetPropertyWithEntity (OVERVIEW_PAGE set/clear, PARAMETER new/update/clear).

Also discovered and documented a dSet bug: it only updates existing BSON keys, never creates new ones. SetActivityProperty for PAGE/TARGETING_MICROFLOW/TARGETING_XPATH silently fails when the target key does not pre-exist. See TestWorkflowMutator_SetActivityProperty_Page_MissingKey. Tracked for a follow-up fix.

#3 (BSON helpers dedup): Already addressed in PR #238mdl/executor/bson_helpers.go is removed in feature/decouple-executor. The remaining copy in page_mutator.go is shared within mprbackend package by workflow_mutator.go. No action needed in #237.

#4 (MockPageMutator/MockWorkflowMutator): Deferred. These ~30-method FuncField mock types would enable executor-level isolation tests for cmd_alter_page.go/cmd_alter_workflow.go, but the current PR tests exercise the real mpr implementation directly, which gives stronger guarantees. Tracked as a future item.

#5 (SetLayout tests): Fixed in 3b67e04. Added 6 dedicated tests to page_mutator_test.go: basic layout change with parameter remapping, explicit param mappings, same layout no-op, snippet rejection, missing FormCall, and empty Form field.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread mdl/executor/cmd_alter_workflow.go
Comment thread mdl/executor/widget_registry.go
Comment thread mdl/executor/cmd_pages_builder_input_datagrid.go
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread mdl/executor/widget_engine.go
Comment thread mdl/backend/mpr/workflow_mutator.go
Comment thread mdl/executor/bson_helpers.go Outdated
@github-actions
Copy link
Copy Markdown

AI Code Review

Let me review this pull request against the checklist.

First, I need to understand what this PR does. From the PR description:

  • This PR provides MPR-backed implementations that actually perform the BSON manipulation
  • It moves all BSON mutation logic out of the executor and behind the backend abstraction
  • The executor becomes a thin orchestrator
  • It provides implementations for PageMutator and WorkflowMutator
  • It adds WidgetObjectBuilder for widget construction and serialization
  • It refactors the executor to be thin orchestrators calling these backend methods
  • It moves tests from executor to backend

Now let me go through the checklist:

Overlap & duplication

  • Check proposals list: There's no proposals directory mentioned, so nothing to check there

  • Check if the diff introduces code that duplicates existing implementations: Looking at the diff, I see:

    • New files: page_mutator.go, workflow_mutator.go, widget_builder.go, convert_roundtrip_test.go
    • Renamed/moved files: alter_page_test.go -> page_mutator_test.go, alter_workflow_test.go -> workflow_mutator_test.go
    • Modified files: backend.go, microflow.go, mock/backend.go, mock/mock_microflow.go, mock/mock_mutation.go, mpr/backend.go
    • The PR moves logic from executor to backend, so it's not duplicating but refactoring
    • I don't see obvious duplication of existing functionality
  • Check doctype test files: The PR mentions tests were relocated from executor to backend. The doctype test files listed (01-domain-model-examples.mdl, etc.) don't seem to cover PAGE or WORKFLOW alterations specifically, so this is likely new test coverage.

  • Flag if the PR re-documents already-shipped features as new: This PR is refactoring existing functionality (ALTER PAGE and ALTER WORKFLOW were already implemented in PR refactor: define mutation backend interfaces #236 which defined the mutation interfaces), so it's not documenting new features but moving existing implementation.

MDL syntax design

This PR doesn't appear to add or modify MDL syntax. It's refactoring the implementation behind existing MDL syntax (ALTER PAGE, ALTER WORKFLOW). The PR description says it's implementing the mutation interfaces defined in PR #236. So no new MDL syntax is being added here.

Full-stack consistency

Since this PR doesn't add new MDL syntax or features (it's implementing existing interfaces), the full-stack consistency check doesn't apply. The grammar, AST, visitor, etc. would have been handled in PR #236 which defined the interfaces.

Test coverage

  • New packages: The PR creates new backend/mpr/ package implementations, and there are test files:
    • convert_roundtrip_test.go (new)
    • page_mutator_test.go (moved from executor)
    • workflow_mutator_test.go (moved from executor, though not shown in the diff snippet)
  • The PR mentions: "Tests relocated: ALTER PAGE and ALTER WORKFLOW tests moved from executor to mdl/backend/mpr/"
  • New executor commands: Not applicable since this is backend refactoring
  • Integration paths: The tests moved from executor to backend should cover integration paths
  • No time.sleep: I don't see any in the visible diff

Security & robustness

  • Command injection, XSS, SQL injection, temp file issues: Not applicable as this is BSON manipulation
  • Unix sockets: Not applicable
  • File I/O in hot paths: The PR is moving BSON manipulation logic, not changing I/O patterns significantly
  • Silent side effects on typos: The code looks to have proper error handling (returns errors when widgets not found, etc.)
  • Method receivers: I don't see obvious issues with receivers in the visible code

Scope & atomicity

  • Each commit should do one thing: The PR description indicates it's a single refactor - moving BSON mutation logic to backend
  • PR scoped to single feature: Yes, it's about implementing the mutation backends
  • Independent features: Not applicable
  • Refactors touching many files: This is a refactor touching many files, but it's presented as a single coherent refactor (moving logic behind backend abstraction)

Code quality

  • Refactors applied consistently: The PR shows a consistent pattern of moving logic from executor to backend
  • Manually maintained lists: I don't see obvious keyword lists or type mappings that would be maintenance risks
  • Design docs matching implementation: The PR is implementing interfaces defined in PR refactor: define mutation backend interfaces #236, so it should match

Bugs & correctness

  • Logic errors: Hard to spot without running, but the code follows patterns from the previous executor implementation
  • Error handling: Appears to have proper error handling (returns errors when widgets not found, etc.)

Now let me look at specific concerns from the visible diff:

  1. In mdl/backend/mpr/page_mutator.go:

    • The SetLayout function has logic to detect old layout name by parsing FormCall
    • There's a comment: "// Detect old layout name"
    • The logic seems complex but necessary for the functionality
    • I notice in the parameter mapping section:
      if paramMappings != nil {
          if mapped, ok := paramMappings[placeholder]; ok {
              placeholder = mapped
          }
      }
      
      This looks correct for handling parameter mappings
  2. In the same file, SetPluggableProperty function:

    • Has a long switch statement for different operation types
    • Seems to handle various property types correctly
    • Uses context appropriately
  3. Error handling:

    • Functions return errors when widgets not found, etc.
    • Uses fmt.Errorf for wrapping errors
  4. The convert_roundtrip_test.go file has many tests for conversion functions, which is good for ensuring correctness.

One potential issue I notice: In mdl/backend/mpr/page_mutator.go, the SetLayout function has this logic:

if m.containerType == "snippet" {
    return fmt.Errorf("SET Layout is not supported for snippets")
}

But looking at the ContainerKind type (from PR #236 which we don't see here), I wonder if it's comparing string to enum correctly. However, in the OpenPageForMutation function, I see:

containerType := backend.ContainerPage
switch {
case strings.Contains(typeName, "Snippet"):
    containerType = backend.ContainerSnippet
case strings.Contains(typeName, "Layout"):
    containerType = backend.ContainerLayout
}

And then later:

func (m *mprPageMutator) ContainerType() backend.ContainerKind { return m.containerType }

So the comparison in SetLayout should be:

if m.containerType == backend.ContainerSnippet {

Not:

if m.containerType == "snippet" {

This looks like a bug! It's comparing a backend.ContainerKind enum value to a string literal.

Let me double-check this. In the SetLayout function:

func (m *mprPageMutator) SetLayout(newLayout string, paramMappings map[string]string) error {
    if m.containerType == "snippet" {
        return fmt.Errorf("SET Layout is not supported for snippets")
    }
    ...
}

But m.containerType is of type backend.ContainerKind, which is likely an enum/constant type defined in backend.go.

Looking at the diff for backend.go:

diff --git a/mdl/backend/backend.go b/mdl/backend/backend.go
index 18fa681e..f9fe9ad0 100644
--- a/mdl/backend/backend.go
+++ b/mdl/backend/backend.go
@@ -34,4 +34,5 @@ type FullBackend interface {
 	PageMutationBackend
 	WorkflowMutationBackend
 	WidgetSerializationBackend
+	WidgetBuilderBackend
 }

This doesn't show the ContainerKind definition. But from the OpenPageForMutation function in mpr/backend.go, we can see it's being set to backend.ContainerPage, backend.ContainerSnippet, etc.

So yes, this is a bug. It should be comparing to the enum constant, not a string.

Let me see if I can find where ContainerKind is defined. It's not in the visible diff, but it's referenced.

Actually, looking more carefully at the OpenPageForMutation function in mpr/backend.go:

containerType := backend.ContainerPage
switch {
case strings.Contains(typeName, "Snippet"):
    containerType = backend.ContainerSnippet
case strings.Contains(typeName, "Layout"):
    containerType = backend.ContainerLayout
}

And then:

func (m *mprPageMutator) ContainerType() backend.ContainerKind { return m.containerType }

So ContainerKind is definitely a type in the backend package, and ContainerSnippet is a value of that type.

Therefore, in SetLayout, it should be:

if m.containerType == backend.ContainerSnippet {

Not:

if m.containerType == "snippet" {

This is a critical bug that would cause the SET Layout check to always fail (since an enum value will never equal a string), allowing SET Layout on snippets when it should be prohibited.

Let me check if there are other similar issues.

In the same file, I see:

// Detect old layout name
oldLayoutQN := ""
for _, elem := range formCall {
    if elem.Key == "Form" {
        if s, ok := elem.Value.(string); ok && s != "" {
            oldLayoutQN = s
        }
    }
    if elem.Key == "Arguments" {
        if arr, ok := elem.Value.(bson.A); ok {
            for _, item := range arr {
                if doc, ok := item.(bson.D); ok {
                    for _, field := range doc {
                        if field.Key == "Parameter" {
                            if s, ok := field.Value.(string); ok && oldLayoutQN == "" {
                                if lastDot := strings.LastIndex(s, "."); lastDot > 0 {
                                    oldLayoutQN = s[:lastDot]
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

This logic seems okay for extracting the module name from a qualified name like "Module.LayoutName" -> "Module".

But then later:

if oldLayoutQN == newLayout {
    return nil
}

This comparison might be problematic if newLayout is expected to be a qualified name but oldLayoutQN is just the module name. Let me see how it's used.

Actually, looking at the usage:

if paramMappings != nil {
    if mapped, ok := paramMappings[placeholder]; ok {
        placeholder = mapped
    }
}
doc[j].Value = newLayout + "." + placeholder

So newLayout is expected to be a module name (since it's concatenated with "." and placeholder), and oldLayoutQN was extracted as the module name from the parameter (e.g., from "Module.LayoutName" we get "Module").

So the comparison if oldLayoutQN == newLayout makes sense - both are module names.

But the initial check for snippet should definitely be fixed.

Another thing: In the SetLayout function, there's:

// Update Form field
for i, elem := range formCall {
    if elem.Key == "Form" {
        formCall[i].Value = newLayout
    }
}

But newLayout here should be the qualified name (e.g., "MyModule.MyLayout"), not just the module name. Let me check.

Actually, looking at the parameter: SetLayout(newLayout string, ...)
And from the usage in the Form field update: formCall[i].Value = newLayout
And later when updating parameters: doc[j].Value = newLayout + "." + placeholder

This suggests that newLayout is expected to be just the module name, and the layout name (placeholder) is appended to it.

But then the early return check if oldLayoutQN == newLayout compares module names, which is correct for detecting if the layout (module) is changing.

However, the Form field should store the qualified name, not just the module name. Let me check how it's stored originally.

In the detection loop:

if elem.Key == "Form" {
    if s, ok := elem.Value.(string); ok && s != "" {
        oldLayoutQN = s
    }
}

Here, s is the full qualified name (e.g., "MyModule.MyLayout"), but then:

if lastDot := strings.LastIndex(s, "."); lastDot > 0 {
    oldLayoutQN = s[:lastDot]
}

So oldLayoutQN becomes just the module name ("MyModule").

Then when updating:

formCall[i].Value = newLayout

If newLayout is just the module name (as used later in newLayout + "." + placeholder), then this would store just the module name in the Form field, which is incorrect - it should be the qualified name.

This is another bug.

Let me trace through:

  1. Original Form field value: "MyModule.MyLayout" (qualified name)
  2. Detection: oldLayoutQN = "MyModule" (module name only)
  3. User wants to change to "MyModule.NewLayout"
  4. Check: if oldLayoutQN == newLayout -> if "MyModule" == "MyModule" -> true, so returns early (no change) - but this is wrong because the layout name changed from "MyLayout" to "NewLayout"!

Actually, wait. The check is:

if oldLayoutQN == newLayout {
    return nil
}

But newLayout parameter is supposed to be the new layout reference. If the user says "SET Layout TO MyModule.NewLayout", then newLayout should be "MyModule.NewLayout".

But in the code, when updating the Form field, it does:

formCall[i].Value = newLayout

And when updating parameters:

doc[j].Value = newLayout + "." + placeholder

This suggests that newLayout is expected to be just the module name, and the layout-specific part comes from the parameter mapping.

But that doesn't make sense for the Form field, which should store the qualified layout name.

Let me look at how this is called. I don't see the caller in the diff, but presumably from the executor.

Actually, re-reading the SetLayout function signature and usage:

The function is: SetLayout(newLayout string, paramMappings map[string]string)

And in the Form field update: formCall[i].Value = newLayout

In the parameter update: doc[j].Value = newLayout + "." + placeholder

This implies that newLayout is the module name only.

But then the detection:

if elem.Key == "Form" {
    if s, ok := elem.Value.(string); ok && s != "" {
        oldLayoutQN = s
    }
}

Here s is the full qualified name from the Form field.

Then:

if lastDot := strings.LastIndex(s, "."); lastDot > 0 {
    oldLayoutQN = s[:lastDot]
}

So oldLayoutQN becomes the module name.

Then the check: if oldLayoutQN == newLayout compares module name to module name - this is correct for checking if the module (and thus potentially the layout) is changing.

But if the user changes from "MyModule.MyLayout" to "MyModule.NewLayout", the module name doesn't change, so oldLayoutQN == newLayout would be true, and it would return early without making any changes - which is wrong because the layout name did change.

Ah, I see the issue. The check should be comparing the full qualified names, not just the module names.

Actually, no. Let's think about what SET Layout does.

In Mendix, when you SET Layout, you're changing which layout the page uses. The layout is identified by a qualified name like "Module.LayoutName".

The Form field in the page's FormCall stores the qualified name of the layout.

When you change layouts, you might also need to update parameter mappings if the new layout has different parameters.

The function receives:

  • newLayout: the new layout qualified name? Or just the module name?

Looking at the parameter update logic:

doc[j].Value = newLayout + "." + placeholder

This strongly suggests that newLayout is expected to be just the module name, and placeholder is the layout name (e.g., "MyLayout").

But then what should go in the Form field? It should be the qualified layout name, which would be newLayout + "." + (some layout name).

But which layout name? It should be determined from the parameter mappings or something.

Actually, looking more carefully at the detection of oldLayoutQN:

For the Form field:

if elem.Key == "Form" {
    if s, ok := elem.Value.(string); ok && s != "" {
        oldLayoutQN = s
    }
}

So oldLayoutQN gets the full qualified name from the Form field (e.g., "MyModule.MyLayout").

Then for parameters:

if elem.Key == "Arguments" {
    if arr, ok := elem.Value.(bson.A); ok {
        for _, item := range arr {
            if doc, ok := item.(bson.D); ok {
                for _, field := range doc {
                    if field.Key == "Parameter" {
                        if s, ok := field.Value.(string); ok && oldLayoutQN == "" {
                            if lastDot := strings.LastIndex(s, "."); lastDot > 0 {
                                oldLayoutQN = s[:lastDot]
                            }
                        }
                    }
                }
            }
        }
    }
}

This is trying to extract the module name from a parameter value that might be a qualified name like "Module.Something".

But there's a bug here: oldLayoutQN == "" - it's checking if oldLayoutQN is empty, but oldLayoutQN was already set from the Form field above! So this condition will almost never be true unless the Form field was empty.

This looks wrong.

Let me re-read:

oldLayoutQN := ""
// ...
if elem.Key == "Form" {
    if s, ok := elem.Value.(string); ok && s != "" {
        oldLayoutQN = s
    }
}
if elem.Key == "Arguments" {
    // ...
    for _, field := range doc {
        if field.Key == "Parameter" {
            if s, ok := field.Value.(string); ok && oldLayoutQN == "" {  // BUG: should not check oldLayoutQN == "" here
                if lastDot := strings.LastIndex(s, "."); lastDot > 0 {
                    oldLayoutQN = s[:lastDot]
                }
            }
        }
    }
}

The condition && oldLayoutQN == "" means it only tries to extract the module name from parameters


Automated review via OpenRouter (Nemotron Super 120B) — workflow source

@retran
Copy link
Copy Markdown
Contributor Author

retran commented Apr 20, 2026

Thanks for the thorough review. Addressing the two findings:

1. ContainerKind string comparison (line 257): Not a bug. ContainerKind is type ContainerKind string (defined in mdl/backend/mutation.go:12), and ContainerSnippet = "snippet". Go allows comparing a named string type with a string literal — this compiles and works correctly. That said, using the constant backend.ContainerSnippet would be more explicit; will update.

2. SetLayout oldLayoutQN detection logic: Not a bug. The detection works in two stages: (a) the Form field stores the full qualified name (e.g. "MyModule.MyLayout"), so oldLayoutQN gets the full QN; (b) the Arguments fallback (&& oldLayoutQN == "") only runs if the Form field was empty, extracting a module name from parameter values as a last resort. The newLayout parameter is also a full QN (confirmed by tests: SetLayout("MyModule.NewLayout", nil)), so the comparison oldLayoutQN == newLayout correctly compares full qualified names. The parameter remapping strips the old QN prefix and prepends the new one. All 6 SetLayout tests pass and verify this behavior.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread mdl/executor/widget_engine.go Outdated
Comment thread mdl/executor/bson_helpers.go Outdated
@github-actions
Copy link
Copy Markdown

AI Code Review

We are given a diff of a pull request that refactors the mutation backends and migrates handlers.
We are to review against the provided checklist.

Since the diff is very long (truncated to 80k characters, total 387634 bytes), we must focus on the key aspects.

However, note that the PR description says:
"PR #236 defined the mutation interfaces — this PR provides the MPR-backed implementations that actually perform the BSON manipulation."

We are to check:

  1. Overlap & duplication
  2. MDL syntax design (if new or modified MDL syntax is present)
  3. Full-stack consistency (for MDL features)
  4. Test coverage
  5. Security & robustness
  6. Scope & atomicity
  7. Code quality
  8. Bugs & correctness

Let's analyze the provided diff:

The diff shows changes in:

  • mdl/backend/backend.go: Added WidgetBuilderBackend to FullBackend interface.
  • mdl/backend/microflow.go: Added ParseMicroflowFromRaw method to MicroflowBackend.
  • mdl/backend/mock/backend.go: Added methods for WidgetBuilderBackend and ParseMicroflowFromRaw to MockBackend.
  • mdl/backend/mock/mock_microflow.go: Implemented ParseMicroflowFromRawFunc.
  • mdl/backend/mock/mock_mutation.go: Implemented WidgetBuilderBackend methods.
  • mdl/backend/mpr/backend.go: Implemented ParseMicroflowFromRaw and changed SerializeDataSource to use mpr.SerializeCustomWidgetDataSource.
  • mdl/backend/mpr/convert_roundtrip_test.go: New file with many tests for conversion functions.
  • mdl/backend/mpr/page_mutator.go: New file with 1554 lines of PageMutator implementation.
  • mdl/executor/alter_page_test.go: Renamed and moved to mdl/backend/mpr/page_mutator_test.go (with some changes in test function names and structure).

We note that the PR also mentions:

  • New backend implementations: page_mutator.go, workflow_mutator.go, widget_builder.go.
  • Executor simplification: cmd_alter_page.go and cmd_alter_workflow.go were rewritten as thin orchestrators (but these files are not in the diff?).
  • Tests relocated: ALTER PAGE and ALTER WORKFLOW tests moved from executor to mdl/backend/mpr/.

However, the diff provided does not show the executor files (cmd_alter_page.go, cmd_alter_workflow.go) or the widget_builder.go.

Since the diff is truncated, we must rely on what we see and the description.

Let's break down by checklist:

  1. Overlap & duplication:

    • The PR moves BSON mutation logic from the executor to the backend (mpage_mutator.go, etc.).
    • We see that the executor files (cmd_alter_page.go, cmd_alter_workflow.go) are not in the diff, but the PR says they were simplified.
    • The PR also says that widget_engine.go was refactored to use WidgetObjectBuilder and that some files were deleted (widget_defaults.go, widget_operations.go, widget_templates.go).
    • We must check if there is duplication. Since the logic is moved, we should not see the same logic in two places.
    • However, without seeing the executor files after the change, we cannot be sure. But the PR says they are now thin orchestrators.

    Given the description, it seems the duplication has been removed by moving the logic to the backend.

  2. MDL syntax design:

    • The PR does not appear to add or modify any MDL syntax. It is about refactoring the implementation of existing MDL commands (ALTER PAGE, ALTER WORKFLOW) to use a backend.
    • Therefore, we do not expect changes to the grammar, AST, visitor, etc. for new MDL syntax.
    • However, note that the PR mentions: "New MDL syntax must follow design guidelines", but this PR is not adding new MDL syntax.

    So we skip this section.

  3. Full-stack consistency (for MDL features):

    • Since no new MDL syntax is added, we don't need to check the full pipeline for a new feature.
    • However, note that the PR says: "New MDL features MUST be wired through the full pipeline". This PR is not adding a new MDL feature, but refactoring the implementation of existing ones.
    • Therefore, we don't expect changes to the grammar, AST, visitor, LSP, etc. for this PR.

    But wait: the PR does mention that tests were relocated and that the executor was simplified. We don't see the executor changes in the diff, but we do see that the test file for alter_page was moved and renamed.

    We see in the diff:
    mdl/executor/alter_page_test.go -> mdl/backend/mpr/page_mutator_test.go

    This is a test relocation, not a change to the MDL feature itself.

  4. Test coverage:

    • We see a new test file: mdl/backend/mpr/convert_roundtrip_test.go with 600 lines of tests for conversion functions.
    • We also see that the alter_page_test.go was moved and renamed to page_mutator_test.go (and the test function names were changed from TestApply* to Test* without the "Apply" prefix, but the logic seems similar).
    • The PR says: "Tests relocated: ALTER PAGE and ALTER WORKFLOW tests moved from executor to mdl/backend/mpr/ where the implementation now lives."
    • We should check if the tests are adequate and if they cover the new implementation.

    However, without seeing the full test files, we can only note that the relocation is done and a new round-trip test file was added.

  5. Security & robustness:

    • We must check for command injection, XSS, SQL injection, temp file issues, etc.

    • The changes are in the backend and mutation logic, which are internal and not exposed directly to the user via MDL (the MDL commands are still the same, just implemented differently).

    • We see in the page_mutator.go:
      - Uses fmt.Errorf for error messages, which is safe.
      - Uses BSON manipulation via the mongo-driver, which is safe if used correctly.
      - There are checks for nil and empty strings.

    • We note that the SetWidgetProperty function has a switch for known properties and then falls back to pluggable widget properties. This is safe.

    • However, we must check for potential issues like:
      - In SetPluggablePropertyMut, we see:
      switch v := value.(type) {
      case string:
      dSet(valDoc, "PrimitiveValue", v)
      case bool:
      if v {
      dSet(valDoc, "PrimitiveValue", "yes")
      } else {
      dSet(valDoc, "PrimitiveValue", "no")
      }
      ...
      This is safe because we are setting a string value in the BSON.

      - In `setWidgetAttributeRefMut`, we see:
            attrPath, ok := value.(string)
            if !ok {
                return fmt.Errorf("Attribute value must be a string")
            }
            ... 
            This is safe.
      
    • We also see in InsertWidget:
      insertIdx := result.index
      if strings.EqualFold(string(position), "after") {
      insertIdx = result.index + 1
      }
      This is case-insensitive for "after", which is acceptable.

    • We don't see any obvious security issues.

  6. Scope & atomicity:

    • The PR description says: "PR refactor: define mutation backend interfaces #236 defined the mutation interfaces — this PR provides the MPR-backed implementations that actually perform the BSON manipulation."
    • The goal is to move all BSON mutation logic out of the executor and behind the backend abstraction.
    • This is a single concern: refactoring the mutation logic to the backend.
    • However, the PR is large (28 files changed, +4,511/-3,523) and touches multiple areas (page, workflow, widget builder).
    • The PR description says it is the largest in a stack of 5 PRs (shared-types refactoring stack).
    • We must check if the PR is doing one thing. It is: moving the mutation logic to the backend.

    But note: the PR also moved tests and simplified the executor. These are part of the same refactoring.

    However, the checklist says: "Each PR should be scoped to a single feature or concern". This PR is about refactoring the mutation backends, which is a single concern.

  7. Code quality:

    • We must look for old patterns still present, manually maintained lists, and design docs matching implementation.

    In the page_mutator.go, we see:
    - A lot of BSON manipulation code that was previously in the executor.
    - The code is organized into sections (with comments) and helper functions.

    We see some potential improvements:
    - In SetPluggableProperty, there is a long switch statement for opName. This is acceptable but could be refactored if it grows.
    - There are several helper functions that are internal to the file (starting with lowercase) which is good.

    We also see in the convert_roundtrip_test.go:
    - Many test functions that follow a pattern.

    However, without seeing the entire codebase, we cannot say if there are old patterns still present.

    We note that the PR says: "Review-driven improvements (post-PR creation):"
    - NewWidgetRegistryWithOps(extraOps) was reintroduced for extensibility.
    - Round-trip and wrapper tests for convert/unconvert layer.

    This indicates that the code quality was considered.

  8. Bugs & correctness:

    • We must look for logic errors, race conditions, resource leaks, error handling gaps.

    We see in the page_mutator.go:
    - In OpenPageForMutation:
    rawBytes, err := b.reader.GetRawUnitBytes(unitID)
    if err != nil {
    return nil, fmt.Errorf("load raw unit bytes: %w", err)
    }
    var rawData bson.D
    if err := bson.Unmarshal(rawBytes, &rawData); err != nil {
    return nil, fmt.Errorf("unmarshal unit BSON: %w", err)
    }
    This is correct error handling.

    - In `SetWidgetProperty`:
          if widgetRef == "" {
              // Page-level property
              return applyPageLevelSetMut(m.rawData, prop, value)
          }
          result := m.widgetFinder(m.rawData, widgetRef)
          if result == nil {
              return fmt.Errorf("widget %q not found", widgetRef)
          }
          return setRawWidgetPropertyMut(result.widget, prop, value)
      This seems correct.
    
    - In `InsertWidget`:
          // Serialize widgets
          newBsonWidgets, err := serializeWidgets(widgets)
          if err != nil {
              return fmt.Errorf("serialize widgets: %w", err)
          }
      Then it builds a new array and sets it.
    
    - We note that the `serializeWidgets` function returns an error if any widget fails to serialize? Actually, it skips widgets that return nil from `mpr.SerializeWidget(w)`. 
      This might be a bug if we expect all widgets to be serialized. However, the comment says: "if bsonDoc == nil { continue }", meaning we skip nil.
    
      But note: the `mpr.SerializeWidget` function might return nil for unsupported widgets? We don't see the implementation of `mpr.SerializeWidget` in the diff.
    
      However, the PR says that the widget_builder.go was created for widget construction and serialization. We don't see that file in the diff.
    
      We must trust that the widget_builder.go handles serialization correctly.
    
    • We also see in DropWidget:
      for _, ref := range refs {
      var result *bsonWidgetResult
      if ref.IsColumn() {
      result = findBsonColumn(m.rawData, ref.Widget, ref.Column, m.widgetFinder)
      } else {
      result = m.widgetFinder(m.rawData, ref.Widget)
      }
      if result == nil {
      return fmt.Errorf("widget %q not found", ref.Name())
      }
      ... // remove the widget from the array
      }
      This returns an error on the first missing widget. It does not continue to try to drop the others. This is acceptable because if one widget is missing, the operation fails.

    • In SetPluggableProperty:
      We iterate over the properties to find the one matching propKey. If not found, we return an error.
      This is correct.

    • We see in setColumnPropertyMut:
      internalKey := columnPropertyAliases[propName]
      if internalKey == "" {
      internalKey = propName
      }
      Then we look for a property with that internalKey in the column's properties.
      If not found, we return an error.
      This is correct.

    • We see in applyPageLevelSetMut:
      switch prop {
      case "Title":
      if formCall := dGetDoc(rawData, "FormCall"); formCall != nil {
      setTranslatableText(formCall, "Title", value)
      } else {
      setTranslatableText(rawData, "Title", value)
      }
      case "Url":
      strVal, _ := value.(string)
      dSet(rawData, "Url", strVal)
      default:
      return fmt.Errorf("unsupported page-level property: %s", prop)
      }
      This is correct for the two supported properties.

    • We note that the setTranslatableText function:
      It sets the Text field in the first translation. If there are no translations, it sets the Text field directly.
      This is the Mendix way.

    • We see in setWidgetAttributeRefMut:
      It tries to set the AttributeRef property of the widget. If the widget doesn't have an AttributeRef property, it returns an error.
      This is correct.

    • We see in setPluggableWidgetPropertyMut:
      It builds a map from TypePointer ID to PropertyKey for the widget's type.
      Then it iterates over the properties to find the one matching propName.
      If found, it sets the value appropriately (handling string, bool, int, float64, else default).
      If not found, it returns an error.
      This is correct.

    • We note that the serializeDataSourceBson function returns nil for unsupported DataSource types, and then the caller (SetWidgetDataSource) returns an error.
      This is correct.

    • We see in serializeWidgets:
      It returns an error only if there is an error in serialization? Actually, it returns the first error encountered?
      Let's look:
      func serializeWidgets(widgets []pages.Widget) ([]any, error) {
      var result []any
      for _, w := range widgets {
      bsonDoc := mpr.SerializeWidget(w)
      if bsonDoc == nil {
      continue
      }
      result = append(result, bsonDoc)
      }
      return result, nil
      }
      It never returns an error! It skips widgets that return nil from mpr.SerializeWidget and returns nil error.

      This might be a problem if we expect all widgets to be serialized. However, the `mpr.SerializeWidget` function might return nil for unsupported widgets, and we want to skip them? 
      But the caller (InsertWidget and ReplaceWidget) expects to insert the given widgets. If we skip some, then the number of widgets inserted is less than expected.
      
      We should check the `mpr.SerializeWidget` function. Since we don't have it in the diff, we must assume it is correct.
      
      However, note that the PR says that the widget_builder.go was created for widget construction and serialization. It is possible that `mpr.SerializeWidget` is defined there and returns a proper BSON document for all supported widgets, and nil only for unsupported ones (which should not happen in normal use).
      
      Given that this is a refactoring of existing code, we assume the behavior is preserved.
      
    • We see in FindWidget:
      return m.widgetFinder(m.rawData, name) != nil
      This is correct.

    • We see in Save:
      outBytes, err := bson.Marshal(m.rawData)
      if err != nil {
      return fmt.Errorf("marshal modified %s: %w", m.containerType, err)
      }
      return m.backend.writer.UpdateRawUnit(string(m.unitID), outBytes)
      This is correct.

    • We note that the mprPageMutator holds a pointer to the backend and the rawData. The Save method writes the rawData back. This is correct.

    • We see in the convert_roundtrip_test.go:
      Many tests for conversion functions, including nil and error passthrough.

    • We don't see any obvious bugs.

Given the above, we can summarize:

Critical Issues: None found.
Moderate Issues:

  • The serializeWidgets function in page_mutator.go skips widgets that return nil from mpr.SerializeWidget without returning an error. This might lead to inserting fewer widgets than expected. However, without knowing the behavior of mpr.SerializeWidget, we cannot be sure. This is a potential issue.

Minor Issues:

  • The PR is large and touches multiple files. While it is scoped to a single concern (moving mutation logic to the backend), the size might make it harder to review. However, the description says it is part of a stack and the largest in the stack, so it may be acceptable.
  • In SetPluggableProperty, the long switch statement might be harder to maintain if more operations are added. But it is acceptable for now.

What Looks Good:

  • The refactoring moves the BSON mutation logic out of the executor and into the backend, making the executor a thin orchestrator (as intended).
  • The code is well-organized with clear sections and helper functions.
  • Error handling is proper.
  • Tests were relocated and a new round-trip test file was added for the conversion layer.
  • The PR reintroduces extensibility for widget operations (NewWidgetRegistryWithOps) as per review-driven improvements.

Recommendation:
We would recommend approval, but we note the minor issue about serializeWidgets potentially skipping widgets without error. However, since we don't have the full context of `


Automated review via OpenRouter (Nemotron Super 120B) — workflow source

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread mdl/backend/mpr/workflow_mutator.go
Comment thread mdl/backend/mpr/widget_builder.go
retran added 3 commits April 20, 2026 14:05
- 33 round-trip and wrapper tests for convert/unconvert layer covering
  all forward conversions, reverse (unconvert) paths, nil handling,
  and error passthrough
- 48+ workflow mutator tests: InsertAfterActivity, ReplaceActivity,
  InsertOutcome, DropOutcome, InsertPath, DropPath, InsertBranch,
  DropBranch, InsertBoundaryEvent, SetActivityProperty, SetPropertyWithEntity
- 6 SetLayout tests: basic layout change, explicit param mappings,
  same-layout no-op, snippet rejection, missing FormCall, empty Form
- Widget engine test updates for applyChildSlots operation validation
- Add NewWidgetRegistryWithOps(extraOps) for extending known operations
- Move knownOperations from package-level var to WidgetRegistry field
  to eliminate global mutable state race
- Convert validateDefinitionOperations and validateMappings to
  WidgetRegistry methods
- Use backend.ContainerSnippet constant in SetLayout check
- Fix SetPropertyWithEntity for OVERVIEW_PAGE: pass entity value instead
  of empty string (was clearing AdminPage instead of setting it)
- Split OVERVIEW_PAGE and PARAMETER handling in cmd_alter_workflow.go
- Reorder Build(): auto-datasource before applyChildSlots so entityContext
  is available for child widgets
- Validate slot.Operation in applyChildSlots (must be "widgets")
- setWidgetCaption returns validation error when Caption missing
  (matching setWidgetContent behavior)
- extractBinaryIDFromDoc handles []byte in addition to primitive.Binary
@retran
Copy link
Copy Markdown
Contributor Author

retran commented Apr 20, 2026

Thanks for the review. Addressing each point:

Moderate — serializeWidgets skips nil widgets: By design. mpr.SerializeWidget returns nil for widget types that have no BSON representation (e.g., placeholder widgets, layout-only containers). These are structural AST nodes that don't produce output in the serialized model. Skipping them is correct — returning an error would break valid page structures. The caller (InsertWidget, SetChildWidgets) expects that the resulting slice may be shorter than the input.

Minor — PR size: Acknowledged. This is the largest PR in a 5-PR stack (#235#236#237#238#239). Each PR is scoped to a single concern, and this one is the implementation that backs the interfaces defined in #236. The size is a consequence of moving ~50 handler functions from executor to backend — splitting further would create partial migrations that don't compile independently.

Minor — SetPluggableProperty switch statement: Agreed it could grow. The operations are currently finite and stable (defined by the .def.json spec). If new operation types are added, we can refactor to a registry pattern (similar to WidgetRegistry.knownOperations). Acceptable for now.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

mdl/executor/cmd_diff_local.go:503

  • microflowBsonToMDL claims it "Falls back to a header-only stub if parsing fails", but the current implementation unconditionally calls renderMicroflowMDL on the result of ParseMicroflowFromRaw. If the backend returns nil (or in future if parsing can fail), this will panic. Add a nil check (and/or recover) and return a minimal stub MDL when parsing fails to match the comment and prevent crashes in diff-local.
// microflowBsonToMDL converts a microflow BSON to MDL using the same
// renderer as DESCRIBE MICROFLOW, so diffs include activity bodies.
// Falls back to a header-only stub if parsing fails.
func microflowBsonToMDL(ctx *ExecContext, raw map[string]any, qualifiedName string) string {
	qn := splitQualifiedName(qualifiedName)
	mf := ctx.Backend.ParseMicroflowFromRaw(raw, model.ID(qn.Name), "")

	entityNames, microflowNames := buildNameLookups(ctx)
	return renderMicroflowMDL(ctx, mf, qn, entityNames, microflowNames, nil)
}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread mdl/backend/mpr/widget_builder.go
Comment thread mdl/backend/mock/mock_microflow.go Outdated
Comment thread mdl/backend/mutation.go
Comment on lines 82 to +92
// InsertWidget inserts serialized widgets at the given position
// relative to the target widget.
InsertWidget(targetWidget string, position InsertPosition, widgets []pages.Widget) error
// relative to the target widget or column. Position is "before" or "after".
// columnRef is "" for widget targeting; non-empty for column targeting.
InsertWidget(widgetRef string, columnRef string, position InsertPosition, widgets []pages.Widget) error

// DropWidget removes widgets by name from the tree.
DropWidget(widgetRefs []string) error
// DropWidget removes widgets by ref from the tree.
DropWidget(refs []WidgetRef) error

// ReplaceWidget replaces the target widget with the given widgets.
ReplaceWidget(targetWidget string, widgets []pages.Widget) error
// ReplaceWidget replaces the target widget or column with the given widgets.
// columnRef is "" for widget targeting.
ReplaceWidget(widgetRef string, columnRef string, widgets []pages.Widget) error
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PageMutator API mixes WidgetRef (used by DropWidget) with separate widgetRef/columnRef string parameters (used by InsertWidget/ReplaceWidget). This makes call sites error-prone (easy to swap/forget columnRef) and undermines the purpose of introducing WidgetRef. Consider updating InsertWidget/ReplaceWidget to accept a single WidgetRef (or consistently use separate strings everywhere) to keep the API coherent and harder to misuse.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid API design concern. The current split exists because InsertWidget/ReplaceWidget need to target both widget-level and column-level positions (columnRef disambiguates), while DropWidget only needs widget identification. Unifying to WidgetRef would require WidgetRef to carry an optional column field — tracked for a follow-up API cleanup.

Comment thread mdl/backend/mpr/widget_builder.go Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

mdl/executor/widget_engine.go:155

  • Auto-datasource selection is based on ranging over propertyTypeIDs (a Go map). Since map iteration order is randomized, the chosen "first" DataSource property can vary between runs, leading to non-deterministic widget output and potentially different entityContext resolution. Consider selecting deterministically (e.g., collect matching keys, sort, then pick the first) or using template-defined property order if available.
		if ds := w.GetDataSource(); ds != nil {
			for propKey, entry := range propertyTypeIDs {
				if entry.ValueType == "DataSource" {
					dataSource, entityName, err := e.pageBuilder.buildDataSourceV3(ds)
					if err != nil {
						return nil, mdlerrors.NewBackend("auto datasource for "+propKey, err)
					}
					builder.SetDataSource(propKey, dataSource)
					if entityName != "" {
						e.pageBuilder.entityContext = entityName
					}
					break
				}

mdl/backend/mpr/backend.go:762

  • MprBackend still implements several WidgetSerializationBackend methods via panic("not yet implemented"). Since backend.FullBackend includes this interface, any call into these methods will hard-crash at runtime. Consider either implementing these using the existing mpr.Serialize* helpers (as done for SerializeDataSource) or removing them from FullBackend if they are no longer part of the supported backend contract.
func (b *MprBackend) SerializeWidget(w pages.Widget) (any, error) {
	panic("MprBackend.SerializeWidget not yet implemented") // TODO: implement in PR #237
}

func (b *MprBackend) SerializeClientAction(a pages.ClientAction) (any, error) {
	panic("MprBackend.SerializeClientAction not yet implemented") // TODO: implement in PR #237
}

func (b *MprBackend) SerializeDataSource(ds pages.DataSource) (any, error) {
	return mpr.SerializeCustomWidgetDataSource(ds), nil
}

func (b *MprBackend) SerializeWorkflowActivity(a workflows.WorkflowActivity) (any, error) {
	panic("MprBackend.SerializeWorkflowActivity not yet implemented") // TODO: implement in PR #237
}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread mdl/backend/mpr/workflow_mutator.go Outdated
Comment thread mdl/executor/widget_engine.go
@github-actions
Copy link
Copy Markdown

AI Code Review

Critical Issues

None found.

Moderate Issues

None found.

Minor Issues

None found.

What Looks Good

  • Clean architectural separation: The PR successfully moves BSON mutation logic from the executor to backend implementations, making the executor a thin orchestrator as intended. This improves maintainability and follows the backend abstraction pattern.

  • Significant executor simplification:

    • cmd_alter_page.go reduced from 1,721 → 256 lines
    • cmd_alter_workflow.go reduced from 887 → 178 lines
    • This demonstrates successful extraction of complex logic into focused backend components.
  • Proper interface integration: Correctly uses the typed interfaces (ContainerKind, InsertPosition, PluggablePropertyOp) defined in PR refactor: define mutation backend interfaces #236, maintaining type safety.

  • Test coverage maintained:

    • ALTER PAGE and ALTER WORKFLOW tests were relocated to mdl/backend/mpr/ alongside their implementations
    • Added comprehensive convert_roundtrip_test.go with 33 tests covering forward/reverse conversion, nil/error handling, and wrapper functions
    • No loss of test coverage observed.
  • Extensibility preserved: Reintroduced NewWidgetRegistryWithOps(extraOps) to restore the extensibility injection point for user-defined widget operations that was lost during refactoring.

  • Consistent with project patterns:

    • Uses standard Go error handling patterns
    • Follows existing code style and naming conventions
    • Properly implements all required backend interfaces
  • Focused scope: The PR addresses a single concern (moving mutation logic to backend) without mixing unrelated changes, making it easy to review and verify.

Recommendation

Approve - This PR represents a successful refactor that improves code organization while maintaining all existing functionality. The changes are well-scoped, properly tested, and align with the architectural goals outlined in the PR description and previous PRs in the stack. No blocking issues were identified during review.


Automated review via OpenRouter (Nemotron Super 120B) — workflow source

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@retran retran marked this pull request as ready for review April 20, 2026 13:01
@ako ako merged commit 41cde57 into mendixlabs:main Apr 20, 2026
4 of 6 checks passed
@retran retran deleted the feature/mutation-backends branch April 20, 2026 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants