refactor: replace remaining Bun APIs (zstd, mmap, CryptoHasher, file writer)#986
Conversation
|
fe61730 to
2ceb494
Compare
Codecov Results 📊✅ 7014 passed | Total: 7014 | Pass Rate: 100% | Execution Time: 0ms 📊 Comparison with Base Branch
All tests are passing successfully. ✅ Patch coverage is 80.65%. Project has 14204 uncovered lines. Files with missing lines (2)
Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
+ Coverage 77.10% 77.14% +0.04%
==========================================
Files 322 322 —
Lines 62112 62128 +16
Branches 0 0 —
==========================================
+ Hits 47884 47924 +40
- Misses 14228 14204 -24
- Partials 0 0 —Generated by Codecov Action |
2ceb494 to
f652595
Compare
| * Compress a chunk buffer with the chosen codec. Exported for testing. | ||
| * | ||
| * Both codecs run off-thread (Bun's zstd worker and libuv's zlib thread | ||
| * pool), so a chunk being compressed doesn't block the event loop -- |
There was a problem hiding this comment.
Zstd encoding selected without runtime support check
High Severity
pickUploadEncoding returns "zstd" whenever the server advertises it, without checking zstdCompressAsync. On Node < 22.15, encodeChunk silently returns raw bytes (falls through both if branches), but uploadChunk still sets Content-Encoding: zstd. The server then tries to zstd-decompress uncompressed data and fails. Additionally, zipCompression is set to "stored" (no deflate) because a wire codec was selected—compounding the issue.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit f652595. Configure here.
|
|
||
| return new BufferedStreamReader( | ||
| decompressed.getReader() as ReadableStreamDefaultReader<Uint8Array> | ||
| webStream.getReader() as ReadableStreamDefaultReader<Uint8Array> |
There was a problem hiding this comment.
Zstd bridge decompresses eagerly instead of on-demand
Medium Severity
createZstdStreamReader pipes the full compressed block through a flowing Node stream immediately upon construction, decompressing eagerly. Both diff and extra readers are created (lines 304–307) before loadOldBinary awaits, so both blocks can fully decompress and buffer in memory concurrently—contradicting the module's "minimal memory usage" design. The old DecompressionStream approach was pull-based and only decompressed on demand.
Reviewed by Cursor Bugbot for commit f652595. Configure here.
…writer)
Complete the Bun API removal from src/ by replacing:
- Bun.zstdCompress → node:zlib zstdCompress (sourcemaps.ts, zstd-transport.ts)
- Bun.zstdDecompressSync → node:zlib zstdDecompressSync (bspatch.ts)
- DecompressionStream('zstd') → node:zlib createZstdDecompress (bspatch.ts)
- Bun.mmap() → removed, using readFile fallback only (bspatch.ts)
- Bun.CryptoHasher → node:crypto createHash (bspatch.ts)
- Bun.file().writer() → fs.createWriteStream (bspatch.ts, upgrade.ts)
- globalThis.Bun.zstdCompress → node:zlib direct (zstd-transport.ts)
Updated tests:
- Removed tests for globalThis.Bun.zstdCompress fallback paths
(no longer applicable — node:zlib zstd is always available)
- Replaced Bun.zstdDecompress with zstdDecompressSync in test assertions
Zero non-comment Bun.* API calls remain in src/.
7014 tests pass, 0 failures.
f652595 to
054955a
Compare
| if (writeError) { | ||
| break; | ||
| } | ||
| writer.write(chunk); |
There was a problem hiding this comment.
Node.js WriteStream backpressure ignored, causing unbounded memory growth on large downloads
The writer.write(chunk) return value is never checked; when the internal buffer is full (write() returns false), the loop should await the drain event before writing more chunks, otherwise all decompressed data accumulates in the stream's internal buffer and can exhaust memory for large binaries.
Evidence
writeris afs.WriteStream(Node.jsWritable);write()returnsfalsewhen the highWaterMark is exceeded, signalling the caller to pause.- The
for awaitloop over the WebReadableStreamhas no backpressure coupling to theWriteStream; every chunk is handed towrite()immediately regardless of buffer state. - The Bun
FileSink.write()it replaces was synchronous/memory-mapped and did not have this buffering concern. - Decompressed upgrade binaries can be tens to hundreds of MB; if disk I/O stalls (e.g. slow/network-mounted FS), the full decompressed binary can be held in the write buffer.
Suggested fix: Await the drain event when write() returns false
| writer.write(chunk); | |
| const ok = writer.write(chunk); | |
| if (!ok) { | |
| await new Promise<void>((res) => writer.once("drain", res)); | |
| } |
Identified by Warden find-bugs · 86Y-GT9
| if (encoding === "zstd" && zstdCompressAsync) { | ||
| // L3 is libzstd's default; passed explicitly for self-documenting | ||
| // code. L9+ trades ~14% size for 4x compress time and forces the | ||
| // server's decoder to allocate 15-30 MiB of window state -- not | ||
| // worth it once decode cost is counted. | ||
| return await Bun.zstdCompress(buf, { level: 3 }); | ||
| return await zstdCompressAsync(buf, { | ||
| params: { [zlib.constants.ZSTD_c_compressionLevel]: 3 }, | ||
| }); |
There was a problem hiding this comment.
Bug: On Node.js < 22.15, uploadChunk sends uncompressed data with a Content-Encoding: zstd header because encodeChunk silently fails to compress but the header is set regardless.
Severity: HIGH
Suggested Fix
encodeChunk should return an object indicating both the resulting buffer and the encoding that was actually applied (e.g., { data: Buffer, encoding: 'zstd' | 'identity' }). The uploadChunk function should then use this returned encoding to correctly set the Content-Encoding header only when the data is actually compressed. This prevents a mismatch between the payload and its encoding header.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: src/lib/api/sourcemaps.ts#L222-L229
Potential issue: On Node.js versions older than 22.15, the `zstdCompressAsync` function
is unavailable. When a server advertises `zstd` support, the `encodeChunk` function is
called with `encoding: "zstd"` but falls through and returns an uncompressed buffer
because `zstdCompressAsync` is falsy. However, the `uploadChunk` function proceeds to
set the `Content-Encoding: zstd` header based on the initial `encoding` parameter,
without verifying if compression actually occurred. This results in sending uncompressed
data with a `zstd` header, causing the server to fail during decompression and reject
the sourcemap upload.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 3 total unresolved issues (including 2 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 054955a. Configure here.
| // zstdCompress is available in Node 22.15+. Feature-detect to avoid crashing | ||
| // the npm bundle on older Node versions (e.g., CI runners with Node 20). | ||
| // zstdCompress is available in Node 22.15+. Feature-detect to avoid crashing | ||
| // the npm bundle on older Node versions (e.g., CI runners with Node 20). |
There was a problem hiding this comment.
Duplicated feature-detection comments in two files
Low Severity
The two-line comment "zstdCompress is available in Node 22.15+. Feature-detect to avoid crashing…" is repeated verbatim twice in a row — a copy-paste error. The same duplication exists in zstd-transport.ts lines 82–85.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 054955a. Configure here.
Consolidates fixes for review comments across PRs #967, #970, #984, #986. Changes: - Bump engines.node from >=22.12 to >=22.15 (zstd APIs require 22.15+, Node 20 is EOL). Update dist/bin.cjs runtime version check to match. - Simplify zstd imports: replace feature-detection dance with direct imports from node:zlib now that >=22.15 is guaranteed. - Fix spawn error handling in upgrade.ts: propagate error object to catch block so ENOENT/EBUSY detection works (was silently resolving with 1). - Fix sqlite.ts transaction(): wrap ROLLBACK in try/catch so original error is preserved if ROLLBACK itself fails. - Guard semver.compare calls in delta-upgrade.ts with semver.valid() to avoid throwing on invalid version strings. - Narrow catch in shell.ts addToGitHubPath to ENOENT only, re-throw other errors (EACCES, EPERM) instead of silently swallowing. - Add WriteStream backpressure handling in upgrade.ts streamDecompressToFile: check write() return value, await drain. - Add setup-node@v6 with Node 22 to E2E CI job. - Fix whichSync Windows CRLF: split on /\r?\n/ instead of \n. 7014 tests pass, 0 failures.
Consolidates fixes for review comments across PRs #967, #970, #984, #986. Changes: - Bump engines.node from >=22.12 to >=22.15 (zstd APIs require 22.15+, Node 20 is EOL). Update dist/bin.cjs runtime version check to match. - Simplify zstd imports: replace feature-detection dance with direct imports from node:zlib now that >=22.15 is guaranteed. - Fix spawn error handling in upgrade.ts: propagate error object to catch block so ENOENT/EBUSY detection works (was silently resolving with 1). - Fix sqlite.ts transaction(): wrap ROLLBACK in try/catch so original error is preserved if ROLLBACK itself fails. - Guard semver.compare calls in delta-upgrade.ts with semver.valid() to avoid throwing on invalid version strings. - Narrow catch in shell.ts addToGitHubPath to ENOENT only, re-throw other errors (EACCES, EPERM) instead of silently swallowing. - Add WriteStream backpressure handling in upgrade.ts streamDecompressToFile: check write() return value, await drain. - Add setup-node@v6 with Node 22 to E2E CI job. - Fix whichSync Windows CRLF: split on /\r?\n/ instead of \n. 7014 tests pass, 0 failures.
Consolidates fixes for review comments across PRs #967, #970, #984, #986. Changes: - Bump engines.node from >=22.12 to >=22.15 (zstd APIs require 22.15+, Node 20 is EOL). Update dist/bin.cjs runtime version check to match. - Simplify zstd imports: replace feature-detection dance with direct imports from node:zlib now that >=22.15 is guaranteed. - Fix spawn error handling in upgrade.ts: propagate error object to catch block so ENOENT/EBUSY detection works (was silently resolving with 1). - Fix sqlite.ts transaction(): wrap ROLLBACK in try/catch so original error is preserved if ROLLBACK itself fails. - Guard semver.compare calls in delta-upgrade.ts with semver.valid() to avoid throwing on invalid version strings. - Narrow catch in shell.ts addToGitHubPath to ENOENT only, re-throw other errors (EACCES, EPERM) instead of silently swallowing. - Add WriteStream backpressure handling in upgrade.ts streamDecompressToFile: check write() return value, await drain. - Add setup-node@v6 with Node 22 to E2E CI job. - Fix whichSync Windows CRLF: split on /\r?\n/ instead of \n. 7014 tests pass, 0 failures.
## Summary Consolidates fixes for review comments across PRs #967, #970, #984, #986, superseding #988. ### Critical - **Bump `engines.node` to `>=22.15`** — `node:zlib` zstd APIs require 22.15+. Node 20 is EOL. Updated `dist/bin.cjs` runtime version check to match. - **Simplify zstd imports** — replaced feature-detection dance with direct `import { zstdCompress } from "node:zlib"` now that >=22.15 is guaranteed. ### High - **Fix spawn error handling in `upgrade.ts`** — `proc.on("error", () => resolve(1))` discarded the error object, making ENOENT/EBUSY detection dead code. Now properly rejects with the error. ### Medium - **Fix `sqlite.ts` ROLLBACK** — if ROLLBACK throws, the original transaction error was lost. Now wrapped in try/catch. - **Guard `semver.compare`** in `delta-upgrade.ts` with `semver.valid()` to avoid throwing on invalid version strings. - **Narrow catch in `shell.ts`** `addToGitHubPath` — only catch ENOENT, re-throw EACCES/EPERM. - **Add WriteStream backpressure** in `upgrade.ts` `streamDecompressToFile` — check `write()` return value, await `'drain'`. - **Add `setup-node@v6`** with Node 22 to E2E CI job. ### Low - **Fix `whichSync` Windows CRLF** — split on `/\r?\n/` instead of `\n` to strip trailing `\r` from `where.exe` output. ### Test results All 7014 tests pass, 0 failures. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>


Summary
Fourth step of the Bun → Node.js migration (follows #967, #970, #984). Replaces the remaining Bun-specific APIs in
src/— the "Group D" items that required more careful handling.Changes
src/lib/bspatch.ts(rewritten):Bun.zstdDecompressSync()→zstdDecompressSync()fromnode:zlibDecompressionStream('zstd')→createZstdDecompress()fromnode:zlibpiped through a Node Readable→Web ReadableStream bridgeBun.mmap()→ removed entirely; usesreadFile()for both paths (copy-then-read and direct-read fallback)Bun.CryptoHasher("sha256")→createHash("sha256")fromnode:cryptoBun.file(destPath).writer()→createWriteStream(destPath)fromnode:fssrc/lib/upgrade.ts:Bun.file(destPath).writer()→createWriteStream(destPath)fromnode:fssrc/lib/api/sourcemaps.ts:Bun.zstdCompress(buf, { level: 3 })→zstdCompress()fromnode:zlibsrc/lib/telemetry/zstd-transport.ts:globalThis.Bun.zstdCompressdynamic access andBunZstdHosttypezstdCompressfromnode:zlib(always available in Node 22.15+)node:zlib)hasZstdSupport()now checkstypeof zstdCompressCb === "function"directlyTests updated (
test/lib/telemetry/zstd-transport.test.ts):globalThis.Bun.zstdCompressfallback paths (no longer applicable)Bun.zstdDecompresswithzstdDecompressSyncin assertionsResult
Zero non-comment
Bun.*API calls remain insrc/. The onlybun:reference left is therequire("bun:sqlite")fallback insqlite.ts, which will be removed when the test runner migrates to Vitest.All 7014 tests pass, 0 failures.