Integration tests: download marketplace modules on demand instead of bundling .mpk (unblocks agent-editor demo)

[ako]

Goal

Replace the marketplace .mpk files bundled in mx-modules/ with on-demand marketplace downloads during integration testing, now that mxcli marketplace download/install exists. This removes binary blobs from the repo and — more importantly — unblocks integration-testing the agent-editor demo (27/28), whose 7-module dependency is too large to bundle.

Current mechanism

mdl/executor/roundtrip_doctype_test.go runs each doctype script against a fresh project and, for scripts in scriptModuleDeps, imports a bundled .mpk via mx module-import:

var scriptModuleDeps = map[string][]string{
    "05-database-connection-examples.mdl": {"ExternalDatabaseConnector-v6.2.3.mpk"},
    "13-business-events-examples.mdl":     {"BusinessEvents_3.12.0.mpk"},
}

27/28 (agent editor) are not in the map — their 7 modules aren't bundled, so they can't be integration-tested with full references today.

Proposed approach — hybrid resolver (not a hard replacement)

A module resolver backed by a pinned manifest (contentId + exact version):

resolve(module) -> local .mpk:
  1. cache hit (gitignored cache / CI cache)              -> use it
  2. else MENDIX_PAT set -> marketplace download --version <pinned> -> cache
  3. else bundled .mpk exists                              -> offline fallback
  4. else skip test with a clear reason

• PAT-gated so contributors without a token still run the suite (fallback/skip).
• Manifest replaces hardcoded .mpk filenames; version bump = one-line edit.
• Wire 27/28 into the manifest with the 7 agent-stack content-ids.

Resolved content-ids (verified live 2026-06)

• External Database Connector = content 219862 (NOT 2888 — 2888 is the separate "Database Connector"). Version 6.2.3 is available (min Mendix 11.4.0), so the exact bundled version is reproducible: marketplace download 219862 --version 6.2.3.
• BusinessEvents (BusinessEvents_3.12.0.mpk) — content-id TBD (not surfaced by client-side search; needs raw-API lookup).
• Agent stack (7) — GenAICommons, MxGenAIConnector, AgentCommons, AgentEditorCommons, MCPClient, ConversationalUI, Encryption — content-ids TBD.

Open decisions

• CI auth: is a MENDIX_PAT CI secret acceptable? If not, download-backed tests stay opt-in (local, MENDIX_PAT-gated) and CI keeps bundled modules.
• Scope: hybrid resolver for all three, agent-demo only, or full removal of mx-modules/*.mpk.
• Cache location: gitignored dir vs CI cache key.

Caveats

• Network/availability dependency on marketplace.mendix.com + CDN (flake risk).
• mx module-import needs version-matched mxbuild; downloaded module minSupportedMendixVersion must be compatible with the test's Mendix version.
• Old versions can be unpublished over time — pin to available versions and keep the bundled fallback until proven.

References

• docs/11-proposals/PROPOSAL_marketplace_modules.md (Phase B / download shipped)
• mdl/executor/roundtrip_doctype_test.go (scriptModuleDeps)
• Agent 7-module dependency: .claude/skills/mendix/agents.md, mdl-examples/doctype-tests/27/28

[github-actions]

Issue Triage

Summary:
The issue proposes adding an on‑demand marketplace download resolver (with a pinned manifest and optional fallback) to replace bundled .mpk files in integration tests, which would unblock testing of the agent‑editor demo.

Checklist for feature request

Required info	Present?	Notes
Use case – why the feature is needed / what problem it solves	✅	Explains removal of binary blobs and enabling integration tests for the agent‑editor demo.
Proposed syntax or behavior (if applicable)	✅	Describes a hybrid resolver, manifest‑based pinning, cache logic, and fallback steps.
Mendix version relevance (if version‑specific)	✅	Mentions minimum Mendix version for the External Database Connector (11.4.0) and compatibility considerations.

Feedback:
All required information for a feature request is present. Thank you for the detailed proposal! If you have any further questions or need clarification on implementation details, feel free to ask.

---

Automated triage via OpenRouter — workflow source

[github-actions]

Enhancement Implementation Guide

We are given an enhancement request about replacing bundled marketplace .mpk files with on-demand downloads during integration testing.

The goal is to unblock integration testing for the agent-editor demo (doctypes 27 and 28) by avoiding the need to bundle large .mpk files in the repository.

The proposed approach is a hybrid resolver that:

1. Uses a cached .mpk (gitignored or CI cache) if available.
2. If MENDIX_PAT is set, downloads the module from the marketplace (using mxcli marketplace download/install) and caches it.
3. Falls back to the bundled .mpk (if exists) for offline use.
4. Skips the test with a clear reason if none of the above work.

We are to implement this in the context of the existing roundtrip_doctype_test.go which currently uses a map scriptModuleDeps to bundle specific .mpk files for certain doctype scripts.

Steps to implement:

1. Completeness Assessment: The issue is detailed. We have the current mechanism, the proposed approach, and even resolved content-ids for some modules. We need to verify if we have all the necessary information (like the exact content-ids for the agent stack modules) but the issue says they are TBD and will be looked up. However, for the implementation guide, we can proceed with the structure and note that the content-ids will be filled in later.

2. Overlap Check: We check if this can be done with existing syntax. The issue states that the mxcli marketplace download/install commands exist (as per references). So we are not adding new MDL syntax but changing how the test obtains the module packages. Therefore, this is not an MDL syntax change but a change in the test infrastructure.

3. Proposed MDL Syntax: Since this is not about MDL syntax but about the test setup, we don't need to propose new MDL syntax. However, note that the issue mentions that the resolver is hybrid and uses a pinned manifest. We will implement this in Go code.

4. Implementation Roadmap: We are going to modify the existing test code in mdl/executor/roundtrip_doctype_test.go. We will replace the scriptModuleDeps map with a function that resolves a module by its content-id and version (from a manifest) using the hybrid approach.

   Steps in the roadmap:

   • Define a manifest (probably a Go map or a struct) that maps module names (or content-ids) to their exact version and content-id.
   • Create a function that, given a module identifier (like content-id and version), tries to:
     a. Look in a local cache (a directory, gitignored) for the .mpk file.
     b. If not found and MENDIX_PAT is set, run mxcli marketplace download <content-id> --version <version> to download and cache it.
     c. If still not found, look for a bundled .mpk in mx-modules/ (the old way) as a fallback.
     d. If none, return an error (or skip the test with a message).
   • Modify the test setup to use this resolver for each module dependency instead of the current map.

   We will also need to handle the agent-editor demo (doctypes 27 and 28) by adding their 7 modules to the manifest.

5. BSON Investigation: This change does not involve BSON structures because we are not changing the MDL language or how we read/write MPR files. We are only changing how we obtain the module packages for the test. Therefore, no BSON investigation is needed.

6. Testing Strategy: We will need to test the new resolver function. We can do:

   • Unit tests for the resolver function (with mocking of the mxcli command and file system).
   • Integration tests by running the doctype tests for 27 and 28 (and optionally the existing ones) to ensure they work with the new resolver.
   • We should also test the fallback scenarios (no PAT, cache miss, bundled fallback).

7. Gotchas & Risks:

   • Network dependency: The marketplace download might fail. We have the fallback to bundled modules and then skipping.
   • Version compatibility: The downloaded module must be compatible with the Mendix version used in the test. We are pinning to exact versions that we know work (as per the issue).
   • Cache location: We must choose a cache directory that is gitignored and suitable for CI (e.g., ./.mxcli-cache/modules).
   • The mxcli marketplace download command must be available in the PATH during the test. We assume the test environment has mxcli built and available.

Given the above, we note that the enhancement request does not require changes to the MDL language itself, so we focus on the test code.

Let's break down the implementation:

Implementation Roadmap (files to change)

1. mdl/executor/roundtrip_doctype_test.go: This is the main file to change.

   • We will remove the scriptModuleDeps map.
   • We will define a manifest (as a Go variable) that maps a module identifier (we can use the content-id as key) to a struct containing:
     • ContentID: string
     • Version: string
     • (Optional) Name: for logging
   • We will create a function resolveModuleMPK(contentID, version string) (string, error) that returns the path to the .mpk file (or an error).
   • In the test setup, for each doctype script that has module dependencies, we will:
     • Look up the dependencies in the manifest (by content-id, which we will have to map from the old .mpk names to content-ids).
     • For each dependency, call resolveModuleMPK to get the .mpk path.
     • Then use mx module-import with that path.

   We will also need to update the manifest for the agent-editor demo (doctypes 27 and 28) with the 7 modules.

2. We might need to create a new file for the resolver function if it becomes complex, but for simplicity we can put it in roundtrip_doctype_test.go.

3. We will need to update the .gitignore to ignore the cache directory.

4. We will need to document the cache location and the use of MENDIX_PAT in the test.

Example of the manifest

We can have:

var moduleManifest = map[string]ModuleInfo{
    "ExternalDatabaseConnector": {
        ContentID: "219862",
        Version:   "6.2.3",
        Name:      "ExternalDatabaseConnector",
    },
    "BusinessEvents": {
        ContentID: "TBD", // to be filled
        Version:   "3.12.0",
        Name:      "BusinessEvents",
    },
    // Agent stack modules
    "GenAICommons": {
        ContentID: "TBD",
        Version:   "TBD",
        Name:      "GenAICommons",
    },
    // ... and so on for the 7 modules
}

But note: the issue says that the BusinessEvents content-id is TBD and the agent stack content-ids are TBD. We will have to look them up. However, for the implementation, we can leave placeholders and note that they must be filled in before the tests can run.

Alternatively, we can key the manifest by the old .mpk name (as we currently do in scriptModuleDeps) and then map to content-id and version. This way we don't have to change the test script that lists the dependencies by the old name.

We can do:

var moduleManifest = map[string]ModuleInfo{
    "ExternalDatabaseConnector-v6.2.3.mpk": {
        ContentID: "219862",
        Version:   "6.2.3",
    },
    "BusinessEvents_3.12.0.mpk": {
        ContentID: "TBD", // to be found
        Version:   "3.12.0",
    },
    // For agent stack, we don't have bundled mpk, so we won't have an entry here? 
    // But we want to use the manifest for all modules, including those without bundled fallback.
}

Then, in the test, we currently have:

var scriptModuleDeps = map[string][]string{
    "05-database-connection-examples.mdl": {"ExternalDatabaseConnector-v6.2.3.mpk"},
    "13-business-events-examples.mdl":     {"BusinessEvents_3.12.0.mpk"},
}

We can change the test to, for each script and each dependency (which is the old .mpk name), look up in the manifest to get the content-id and version, then resolve the .mpk.

But note: the agent-editor demo (27 and 28) are not in scriptModuleDeps because they don't have bundled mpk. We will add them to scriptModuleDeps (or a similar map) but now the value will be a list of module identifiers (we can use the content-id or a logical name) and then use the manifest to get the content-id and version.

Alternatively, we can change scriptModuleDeps to map to a list of module logical names (like "ExternalDatabaseConnector", "BusinessEvents", etc.) and then have the manifest map from logical name to {content-id, version}. Then we don't care about the .mpk name in the manifest.

Let's refactor:

type ModuleInfo struct {
    ContentID string
    Version   string
}

var moduleManifest = map[string]ModuleInfo{
    "ExternalDatabaseConnector": {ContentID: "219862", Version: "6.2.3"},
    "BusinessEvents":            {ContentID: "TBD", Version: "3.12.0"},
    // Agent stack
    "GenAICommons":              {ContentID: "TBD", Version: "TBD"},
    // ... etc for the 7 modules
}

var scriptModuleDeps = map[string][]string{
    "05-database-connection-examples.mdl": {"ExternalDatabaseConnector"},
    "13-business-events-examples.mdl":     {"BusinessEvents"},
    "27-agent-editor-examples.mdl":        {"GenAICommons", "MxGenAIConnector", /* ... all 7 */},
    "28-agent-editor-examples.mdl":        {"GenAICommons", "MxGenAIConnector", /* ... all 7 */},
}

Then, in the test, for each module name in the list, we look up in moduleManifest to get the content-id and version, and then resolve the .mpk.

This way, we can still use the bundled fallback for modules that have it (by checking if there is a bundled .mpk with the old naming convention) but note: the issue says the bundled fallback is only for offline use and we are moving away from bundling. However, we keep it as a fallback.

But note: the issue says the hybrid resolver uses:

1. cache hit
2. else if MENDIX_PAT set -> download and cache
3. else bundled .mpk exists -> use it
4. else skip

So we don't need to change the way we name the bundled .mpk? We can keep the existing bundled .mpk files in mx-modules/ and use them as fallback.

However, we are trying to remove the bundled .mpk from the repo. The issue says: "This removes binary blobs from the repo". So we are going to remove the bundled .mpk files from the repo and rely on the cache and download. But we keep the bundled fallback only for contributors without a PAT and when the cache is empty? And then we skip if not found.

But note: the issue says: "PAT-gated so contributors without a token still run the suite (fallback/skip)."

So we will keep the bundled .mpk in the repo for now as a fallback, but the goal is to remove them eventually. However, the issue says: "Scope: hybrid resolver for all three, agent-demo only, or full removal of mx-modules/*.mpk."

We are implementing the hybrid resolver, and we can leave the bundled .mpk in the repo for now as a fallback, and then in a future PR remove them once we are confident the resolver works.

But note: the issue says the agent-editor demo is unblocked because the 7 modules are too large to bundle. So we are not going to bundle those 7. We will rely on the resolver (cache/download) for them.

For the existing two (ExternalDatabaseConnector and BusinessEvents), we can keep the bundled .mpk as a fallback until we are sure the resolver works, and then remove them.

Given the above, we will implement the resolver to first try cache, then download (if PAT set), then fallback to bundled .mpk (if exists), then skip.

We will not change the existing scriptModuleDeps for the two existing tests, but we will change how we get the .mpk path for each dependency.

Steps in the test setup (in roundtrip_doctype_test.go)

We will replace the current loop that does:

for _, mpk := scriptModuleDeps[script] {
    // import the bundled mpk
    args := []string{"module-import", "--mpk", filepath.Join("mx-modules", mpk), "--module-path", "/"}
    if err := mxcli(args...); err != nil {
        t.Fatalf("failed to import module %s: %v", mpk, err)
    }
}

With:

for _, moduleName := range scriptModuleDeps[script] {
    info, ok := moduleManifest[moduleName]
    if !ok {
        t.Fatalf("module %s not found in manifest", moduleName)
    }
    mpkPath, err := resolveModuleMPK(info.ContentID, info.Version)
    if err != nil {
        t.Fatalf("failed to resolve module %s (content-id %s, version %s): %v", moduleName, info.ContentID, info.Version, err)
    }
    // import the mpk from mpkPath
    args := []string{"module-import", "--mpk", mpkPath, "--module-path", "/"}
    if err := mxcli(args...); err != nil {
        t.Fatalf("failed to import module %s from %s: %v", moduleName, mpkPath, err)
    }
}

Where resolveModuleMPK is:

func resolveModuleMPK(contentID, version string) (string, error) {
    // 1. Check cache
    cacheDir := os.Getenv("MXCLI_MODULE_CACHE")
    if cacheDir == "" {
        cacheDir = filepath.Join(os.TempDir(), "mxcli-module-cache")
    }
    mpkPath := filepath.Join(cacheDir, fmt.Sprintf("%s-%s.mpk", contentID, version))
    if _, err := os.Stat(mpkPath); err == nil {
        return mpkPath, nil
    }

    // 2. If MENDIX_PAT is set, download
    if pat := os.Getenv("MENDIX_PAT"); pat != "" {
        // We assume mxcli is in PATH
        // We will run: mxcli marketplace download <contentID> --version <version> --output-dir <cacheDir>
        // But note: the `mxcli marketplace download` command might not support --output-dir? We have to check.
        // Alternatively, we can download to a temp file and then move to cache.
        // We'll design the command to output to the cacheDir with the expected name.

        // We'll create the cacheDir if it doesn't exist.
        if err := os.MkdirAll(cacheDir, 0755); err != nil {
            return "", fmt.Errorf("failed to create cache dir %s: %w", cacheDir, err)
        }

        // Run: mxcli marketplace download <contentID> --version <version> --output <mpkPath>
        // We need to check the actual command. From the issue: `marketplace download 219862 --version 6.2.3`
        // We don't see an output flag. We might have to change the command or use the default download location and then move.

        // Let's assume the command downloads to the current directory by default? We don't want that.
        // We can run the command in the cacheDir and then rename the file to the expected name.

        // Alternatively, we can use the `mxcli marketplace install` command? The issue says `mxcli marketplace download/install` exists.

        // We'll look at the existing code for marketplace commands.

        // Since we are in the test, we can run the command and capture the output to know where it was saved.

        // For simplicity, we'll assume we can specify an output directory.

        // If the command does not support output-dir, we can do:
        //   cd cacheDir
        //   mxcli marketplace download <contentID> --version <version>
        //   Then the file will be named something like <contentID>_<version>.mpk? We need to check.

        // We'll leave this as a TODO and note that we need to check the mxcli marketplace command.

        // For now, we'll assume we can run:
        //   mxcli marketplace download <contentID> --version <version> --dir <cacheDir>
        // and it will save as <contentID>-<version>.mpk

        // We'll use the ctx.Backend to run mxcli? But we are in a test, we can use exec.Command.

        // We'll use exec.Command to run mxcli.

        cmd := exec.Command("mxcli", "marketplace", "download", contentID, "--version", version, "--dir", cacheDir)
        // We might need to set the MENDIX_PAT in the environment for the command.
        cmd.Env = append(os.Environ(), "MENDIX_PAT="+pat)
        var out bytes.Buffer
        var stderr bytes.Buffer
        cmd.Stdout = &out
        cmd.Stderr = &stderr
        err := cmd.Run()
        if err != nil {
            return "", fmt.Errorf("

---
*Automated guide via OpenRouter — [workflow source](https://github.com/mendixlabs/mxcli/blob/main/.github/workflows/ai-enhancement-guide.yml)*

[ako]

Update — resolved content-ids + search now paginates

The manifest's "BusinessEvents content-id TBD" is resolved, and the reason it was hard to find is now fixed:

• Business Events = content 202649 ("Mendix Business Events", type Service, latest 3.12.1).
• External Database Connector = content 219862 (confirmed earlier; bundled v6.2.3 is available).

Both have high content-ids that sit near the end of the ~2300-item catalog. marketplace search previously missed them because it only fetched the first page (the API caps limit at 100 server-side). Search now paginates via offset (commit on misc), so marketplace search "business events" / "external database" find them. page is ignored by the API; offset is honoured.

So for the test manifest, the resolver can use:

Script	Module	contentId	pinned version
05-database-connection-examples.mdl	External Database Connector	219862	6.2.3 (bundled fixture version)
13-business-events-examples.mdl	Mendix Business Events	202649	(pick a compatible version via marketplace versions 202649)
27/28 agent-editor	7-module stack	TBD (GenAICommons, MxGenAIConnector, AgentCommons, AgentEditorCommons, MCPClient, ConversationalUI, Encryption)	TBD

Note: pagination makes a deep/sparse search a near-full sequential scan (~23 pages) and therefore slow; concurrent page-fetching is a possible follow-up if search latency matters for the test harness.