Skip to content

refactor: replace BunProc with Npm module using @npmcli/arborist#18308

Merged
thdxr merged 33 commits intodevfrom
refactor/npm-over-bunproc
Apr 1, 2026
Merged

refactor: replace BunProc with Npm module using @npmcli/arborist#18308
thdxr merged 33 commits intodevfrom
refactor/npm-over-bunproc

Conversation

@thdxr
Copy link
Copy Markdown
Member

@thdxr thdxr commented Mar 19, 2026

Summary

  • Remove src/bun/ (BunProc + PackageRegistry) — replaces all bun add/bun install/bun info calls
  • Add src/npm/index.ts (Npm namespace) backed by @npmcli/arborist for deterministic installs
  • Npm.add(pkg) — install a single package into a per-package cache directory
  • Npm.install(dir) — reify dependencies in an existing project directory
  • Npm.which(pkg) — resolve the binary path for a cached package
  • Npm.outdated(pkg, version) — check npm registry for newer versions
  • Update all consumers: config.ts, formatter.ts (Info interface reworked — command removed, enabled() now returns string[] | false), lsp/server.ts, plugin/index.ts, provider/provider.ts
  • Remove BunProc-specific test; add @npmcli/arborist + @types/npmcli__arborist to package.json

Confidence Score: 1/5

  • Not safe to merge — two regressions (a brace-mismatch parse error and a TDZ ReferenceError) will break formatting and binary resolution at runtime.
  • The architectural direction is correct, but formatter.ts has a structural brace mismatch that likely prevents the entire formatter module from loading, and npm/index.ts has a const TDZ crash in which(). Both issues are in hot code paths exercised at startup.
  • packages/opencode/src/format/formatter.ts (brace mismatch) and packages/opencode/src/npm/index.ts (TDZ bug on line 163) require immediate attention before this is mergeable.

Important Files Changed

Filename Overview
packages/opencode/src/npm/index.ts New Npm namespace replacing BunProc; contains a TDZ ReferenceError on line 163 where unscoped is used before its const declaration, crashing which() for packages with a string bin field in multi-binary scenarios.
packages/opencode/src/format/formatter.ts Critical brace mismatch in prettier.enabled(): duplicated if condition consumes the original for-loop close brace, leaving the prettier object declaration unclosed and causing a parse/type error that likely prevents the entire formatter module from loading.
packages/opencode/src/format/index.ts Updated to consume `string[]
packages/opencode/src/lsp/server.ts Migrated from BunProc to Npm.which() for TypeScript-language-server, Vue, Pyright, and Biome LSPs; ESLint server now uses node directly (correct). Pyright now relies on Npm.which("pyright") which correctly resolves from package.json bin — if the TDZ bug in npm/index.ts is fixed.
packages/opencode/src/plugin/index.ts Uses Npm.add(plugin) for plugin installation; error handling logs and skips failed plugins. Acceptable.
packages/opencode/src/provider/provider.ts Replaces BunProc.install(pkg, "latest") with Npm.add(pkg); note that Npm.add no longer enforces the "latest" version constraint — any previously cached version is reused regardless of the requested version tag.
packages/opencode/src/mcp/index.ts Removes BUN_BE_BUN env var and adds as any cast to silence the TypeScript env type error. Minor but the as any suppresses potentially legitimate type checking.
packages/opencode/test/bun.test.ts BunProc tests removed; this is expected given the migration.

Comments Outside Diff (1)

  1. packages/opencode/src/format/formatter.ts, line 66-82 (link)

    P0 Brace mismatch in prettier.enabled() — prettier object never closed

    The refactor introduced a duplicated if block with mismatched indentation, which shifts all the original closing braces one level inward. Counting the braces:

    • Line 73 opens the outer if {
    • Line 74 opens the duplicate inner if {
    • Line 78 } closes the inner if
    • Line 79 } (the original for-loop close) now closes the outer if
    • Line 81 } from }, closes the for loop (leaving return false on line 80 inside it)
    • Line 82 } closes enabled() function body

    The export const prettier: Info = { opened on line 36 has no matching closing brace, so TypeScript/Bun treats all subsequent formatter exports (oxfmt, biome, etc.) as properties of the prettier object, producing a parse or type error.

    The fix is to remove the duplicated condition and restore the intended structure:

Last reviewed commit: "sync"

Greptile also left 2 inline comments on this PR.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 19, 2026

Greptile Summary

This PR replaces the BunProc + PackageRegistry implementation with a new Npm namespace backed by @npmcli/arborist, making package installation runtime-agnostic. The core install/cache logic is solid — per-package locks, lockfile-based change detection, and the Windows tar workaround are all well-considered — but several issues in the consumer integration and binary resolution need attention before this is production-safe.

Key issues found:

  • enabled() functions throw instead of returning falseprettier.enabled(), biome.enabled(), and oxfmt.enabled() in formatter.ts call await Npm.which(...) without a catch. If the npm install or binary resolution fails, the error propagates through isEnabled()getFormatter() in format/index.ts without being caught, silently breaking the entire file-formatting event handler for the rest of the session.
  • bin: string resolution returns the wrong namewhich() uses path.basename(bin) when the package.json bin field is a string, but npm names the .bin symlink after the package name, not the script filename. The correct fallback is the unscoped package name.
  • add() returns an unverified path — when the arborist virtual tree shows a package is installed, add() returns first.path without checking that the directory actually exists on disk. which() recovers via lockfile deletion and retry, but plugin/index.ts calls Npm.add() directly and passes the result straight to import() with no recovery path.
  • Multi-word variable nameslatestVersion / cachedVersion in outdated() violate the project's single-word naming convention.

Confidence Score: 2/5

  • Not safe to merge — uncaught Npm.which() throws can silently break the formatter service, and the bin: string resolution bug returns non-existent paths.
  • Two P1 logic bugs: the formatter enabled() propagation issue can knock out file formatting for an entire session on any install failure, and the bin: string basename resolution returns the wrong filename. The add() disk-validation gap is a third P1 that affects plugin loading under partial-cache conditions. These are not edge-case-only paths.
  • packages/opencode/src/npm/index.ts (binary resolution logic) and packages/opencode/src/format/formatter.ts (uncaught throws from Npm.which()) need the most attention before merge.

Important Files Changed

Filename Overview
packages/opencode/src/npm/index.ts New Npm module replacing BunProc; has bin: string path resolution bug in which(), add() returns unverified disk path for plugin consumers, and multi-word naming convention violations.
packages/opencode/src/format/formatter.ts enabled() functions for prettier, biome, and oxfmt can throw from Npm.which() instead of returning false; uncaught by the format/index.ts event handler, breaking the entire formatter service on install failure.
packages/opencode/src/lsp/server.ts ESLint server is launched via tsx (line 196); previously flagged and unresolved but noted to avoid repeat comment. Other LSP servers look correct with the new Npm API.
packages/opencode/src/config/config.ts Correctly replaces BunProc with Npm.install(dir); concurrency lock is now per-directory. No new issues found.
packages/opencode/src/plugin/index.ts Uses Npm.add(plugin) directly and passes result to import(); if add() returns a stale path (lockfile present but no on-disk install), import() will throw a module-not-found error with no recovery path.
packages/opencode/src/format/index.ts isEnabled() and getFormatter() do not catch errors from item.enabled(), leaving the File.Event.Edited handler vulnerable to unhandled rejections from formatter install failures.

Sequence Diagram

sequenceDiagram
    participant Caller as Caller (formatter/plugin/lsp)
    participant Npm as Npm module
    participant Lock as Lock
    participant Arborist as @npmcli/arborist
    participant Registry as npm registry
    participant FS as Filesystem

    Note over Caller,FS: Npm.which(pkg)
    Caller->>Npm: which(pkg)
    Npm->>FS: readdir(.bin/)
    FS-->>Npm: files[]
    alt files.length == 1
        Npm-->>Caller: binDir/files[0]
    else files.length > 1
        Npm->>FS: readJson(node_modules/pkg/package.json)
        FS-->>Npm: { bin: string | Record }
        Npm-->>Caller: binDir/resolvedName
    else no files
        Npm->>FS: rm(package-lock.json)
        Npm->>Npm: add(pkg)
        Note over Npm,Arborist: Npm.add(pkg)
        Npm->>Lock: write(npm-install:pkg)
        Npm->>Arborist: loadVirtual()
        alt lockfile exists & node in tree
            Arborist-->>Npm: tree (early return)
            Npm-->>Caller: first.path ⚠️ no disk check
        else fresh install
            Npm->>Arborist: reify({add:[pkg]})
            Arborist->>Registry: fetch package
            Registry-->>Arborist: tarball
            Arborist->>FS: extract to cache/packages/pkg/
            Arborist-->>Npm: result tree
            Npm-->>Caller: first.path
        end
    end

    Note over Caller,FS: Npm.install(dir)
    Caller->>Npm: install(dir)
    Npm->>Lock: write(npm-install:dir)
    Npm->>FS: exists(node_modules)
    alt node_modules missing
        Npm->>Arborist: reify() [errors silently swallowed]
    else check lock vs package.json
        Npm->>FS: readJson(package.json + package-lock.json)
        alt dependency not in lockfile
            Npm->>Arborist: reify() [errors silently swallowed]
        end
    end
Loading

Last reviewed commit: "fix: force reinstall..."

@thdxr thdxr force-pushed the refactor/npm-over-bunproc branch 3 times, most recently from 4dd891c to d5bb559 Compare March 20, 2026 00:15
@thdxr thdxr force-pushed the refactor/npm-over-bunproc branch from d5bb559 to ca2099e Compare March 20, 2026 00:17
@thdxr
Copy link
Copy Markdown
Member Author

thdxr commented Mar 20, 2026

@greptile try again

thdxr added 2 commits March 19, 2026 21:49
Use npm-install:${pkg} instead of a global npm-install lock so
concurrent installs of different packages can run in parallel.
When the lockfile exists but .bin is empty or absent, add() would
read the lockfile via loadVirtual() and return early without calling
reify(). Delete the lockfile before calling add() so it proceeds
with a full install.
@thdxr
Copy link
Copy Markdown
Member Author

thdxr commented Mar 20, 2026

@greptile

Replace tsx dependency with explicit 'node' invocation for the
compiled ESLint server. Remove all BUN_BE_BUN env var references
which are no longer needed after the Bun-to-Node migration.
opencode-agent bot added a commit that referenced this pull request Mar 20, 2026
thdxr and others added 3 commits March 19, 2026 23:19
…interface

The formatter Info interface uses enabled(): Promise<string[] | false> but
user configs provide a command property. Added explicit return type to the
arrow function that transforms command into the enabled() return value.

Also added test to verify custom formatters with command property work.
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@thdxr
Copy link
Copy Markdown
Member Author

thdxr commented Mar 20, 2026

@greptileai

Comment on lines +157 to +168
// Multiple binaries — resolve from package.json bin field like npx does
const pkgJson = await Filesystem.readJson<{ bin?: string | Record<string, string> }>(
path.join(dir, "node_modules", pkg, "package.json"),
).catch(() => undefined)
if (pkgJson?.bin) {
const unscoped = pkg.startsWith("@") ? pkg.split("/")[1] : pkg
const bin = pkgJson.bin
if (typeof bin === "string") return unscoped
const keys = Object.keys(bin)
if (keys.length === 1) return keys[0]
return bin[unscoped] ? unscoped : keys[0]
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P0 unscoped referenced before its const declaration (TDZ ReferenceError)

On line 163 return unscoped references a variable that is declared with const on line 166. In JavaScript/TypeScript, const bindings are in the Temporal Dead Zone until their declaration is reached, so accessing unscoped here throws a ReferenceError at runtime.

This means Npm.which() will always crash for any installed package whose package.json declares bin as a bare string (e.g., "bin": "./cli.js") when the .bin directory contains more than one file.

Suggested change
// Multiple binaries — resolve from package.json bin field like npx does
const pkgJson = await Filesystem.readJson<{ bin?: string | Record<string, string> }>(
path.join(dir, "node_modules", pkg, "package.json"),
).catch(() => undefined)
if (pkgJson?.bin) {
const unscoped = pkg.startsWith("@") ? pkg.split("/")[1] : pkg
const bin = pkgJson.bin
if (typeof bin === "string") return unscoped
const keys = Object.keys(bin)
if (keys.length === 1) return keys[0]
return bin[unscoped] ? unscoped : keys[0]
}
if (pkgJson?.bin) {
const bin = pkgJson.bin
const unscoped = pkg.startsWith("@") ? pkg.split("/")[1] : pkg
if (typeof bin === "string") return unscoped
const keys = Object.keys(bin)
if (keys.length === 1) return keys[0]
return bin[unscoped] ? unscoped : keys[0]
}

Comment on lines 95 to 97
return [await Npm.which("oxfmt"), "$FILE"]
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Npm.which("oxfmt") can throw uncaught inside enabled()

Unlike the prettier formatter (which wraps Npm.which in .catch(() => null)), oxfmt.enabled() calls await Npm.which("oxfmt") without any error handling. If the package fails to install (network error, InstallFailedError, etc.) the exception propagates up through isEnabled() in format/index.ts and breaks all subsequent formatting for the session.

Suggested change
return [await Npm.which("oxfmt"), "$FILE"]
}
}
return [await Npm.which("oxfmt").catch(() => null) ?? "", "$FILE"]

Or better, bail out early if which fails:

        const bin = await Npm.which("oxfmt").catch(() => null)
        if (!bin) return false
        return [bin, "$FILE"]

@opencode-agent
Copy link
Copy Markdown
Contributor

⚠️ Blocking Beta Release

This PR cannot be merged into the beta branch due to: Merge conflicts with dev branch

Please resolve this issue to include this PR in the next beta release.

thdxr added 3 commits March 27, 2026 11:03
- Replace BunProc.install with Npm.add for plugin resolution

- Remove needsInstall check from installDependencies

- Update test files to use Npm module instead of BunProc

- Delete bun/index.ts and bun/registry.ts (no longer needed)
// flag. See tar's get-write-flag.js.
// Must be set before @npmcli/arborist is imported since tar caches the flag
// at module evaluation time — so we use a dynamic import() below.
if (process.platform === "win32") {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can remove this now that we're on Bun 1.3.11 - my fix for this landed

@opencode-agent
Copy link
Copy Markdown
Contributor

⚠️ Blocking Beta Release

This PR cannot be merged into the beta branch due to: Merge conflicts with dev branch

Please resolve this issue to include this PR in the next beta release.

@opencode-agent
Copy link
Copy Markdown
Contributor

⚠️ Blocking Beta Release

This PR cannot be merged into the beta branch due to: Merge conflicts with dev branch

Please resolve this issue to include this PR in the next beta release.

2 similar comments
@opencode-agent
Copy link
Copy Markdown
Contributor

⚠️ Blocking Beta Release

This PR cannot be merged into the beta branch due to: Merge conflicts with dev branch

Please resolve this issue to include this PR in the next beta release.

@opencode-agent
Copy link
Copy Markdown
Contributor

⚠️ Blocking Beta Release

This PR cannot be merged into the beta branch due to: Merge conflicts with dev branch

Please resolve this issue to include this PR in the next beta release.

thdxr and others added 14 commits April 1, 2026 11:55
# Conflicts:
#	packages/opencode/src/bun/index.ts
#	packages/opencode/test/bun.test.ts
…st.ts - update resolvePluginTarget to use Npm.add instead of Npm.install - replace BunProc.install with Npm.add in provider.ts - update all tests to use Npm.add/Npm.install instead of BunProc
…ackage.json

Npm.add was returning the package directory path, but dynamic import()
requires the actual entry file path. Added resolveEntryPoint helper to
read package.json and resolve the correct ESM entry point (via exports.import,
module, or main fields) before returning the path.
Changed Npm.add() to return an object with both the package directory and resolved entrypoint. This allows callers to choose which they need - plugins use the directory for path resolution while providers use the entrypoint for loading.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants