Skip to content

feat: riscv64 prebuilt binaries#615

Open
gounthar wants to merge 8 commits into
withcatai:masterfrom
gounthar:feat/riscv64-prebuilt
Open

feat: riscv64 prebuilt binaries#615
gounthar wants to merge 8 commits into
withcatai:masterfrom
gounthar:feat/riscv64-prebuilt

Conversation

@gounthar

@gounthar gounthar commented Jun 6, 2026

Copy link
Copy Markdown

Description of change

Adds a prebuilt binary for linux riscv64, so npm install node-llama-cpp on a RISC-V host picks up a ready-to-run binding instead of compiling llama.cpp from source.

riscv64 already builds from source today, but that needs a full C/C++ toolchain and around 30 minutes of native compile time. This brings riscv64 in line with the platforms that already ship prebuilts.

What's in here:

  • New @node-llama-cpp/linux-riscv64 optional-dependency package (mirrors linux-armv7l).
  • compileLLamaCpp.ts: resolve the riscv64 prebuilt module when arch === "riscv64".
  • detectGlibc.ts: add the riscv64 multiarch lib path and the ld-linux-riscv64-lp64d.so* loader names.
  • movePrebuiltBinariesToStandaloneModules.ts: move the linux-riscv64 build output into the standalone module.
  • build.yml: a native riscv64 build leg.

A few CI notes, since the riscv64 leg is a bit different from the others:

  • The build runs on a native ubuntu-24.04-riscv runner from the RISE RISC-V GitHub App. That app has to be installed on the repo for the leg to get a runner. I'm happy to help coordinate that with RISE.
  • The binary is built from the x64-built dist (rolldown has no riscv64 binding yet), so the workflow splits into a dist-build job and a native binary-build job.
  • The riscv64 runner needs Node.js from the unofficial-builds index (setup-node has no riscv64 binary), a retry around npm ci, ELECTRON_SKIP_BINARY_DOWNLOAD=1, and gcc-14 (recent ggml uses RVV _Float16/zvfh intrinsics that only exist in GCC 14+).

I validated this end-to-end on a fork: dist built on x64, binary compiled natively on the RISE runner, producing an ELF RISC-V llama-addon.node that passes the smoke check. The split-job CI is the part I'm least sure about fitting your conventions, so let me know if you'd rather I structured it differently.

Fixes #614

Pull-Request Checklist

  • Code is up-to-date with the master branch
  • npm run format to apply eslint formatting
  • npm run test passes with this change
  • This pull request links relevant issues as Fixes #0000
  • There are new or updated unit tests validating the change (N/A: packaging and CI change, no runtime logic to unit test)
  • Documentation has been updated to reflect this change (N/A)
  • The new commits and pull request title follow the convention in the pull request guidelines

gounthar added 8 commits June 5, 2026 19:31
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>

@giladgd giladgd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

Could you please provide me with instructions on how I can access a riscv64 machine to manually test this on?
In case the llama.cpp build fails for some reason then I'll need a way to investigate and fix it in a faster manner than to trigger many CI runs.

Also, would you be willing to help me maintain riscv support in node-llama-cpp in the future?

# electron has no riscv64 prebuilt binary; the binary build does not
# use it (the electron example is a separate job), so skip its
# postinstall download to let npm ci complete on riscv64.
ELECTRON_SKIP_BINARY_DOWNLOAD: ${{ matrix.config.artifact == 'linux-riscv64' && '1' || '' }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only use the electron types in the build anyway, so we can set that to always skip the binary download

Comment on lines +246 to +248
# npm on the unofficial riscv64 node intermittently throws internal
# TypeErrors (e.g. "key.match is not a function"); retry a few times.
npm ci || npm ci || npm ci

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we update npm will those issues go away? What do they stem from?
I'd prefer solving the issues from the root, since if this workaround stops working I won't know what to do or how to even approach solving this

Comment on lines +89 to +102
- name: Install Node.js (riscv64 unofficial build)
if: matrix.config.artifact == 'linux-riscv64'
run: |
set -e
# node 24 (npm 11): node 20's npm 10.8.2 trips "set.delete is not a
# function" during npm ci on riscv64, and several devDeps require
# node >= 24.10. The built binary still targets node 20 N-API.
NODE_VER=v24.16.0
curl -fsSL -o /tmp/node.tar.xz "https://unofficial-builds.nodejs.org/download/release/${NODE_VER}/node-${NODE_VER}-linux-riscv64.tar.xz"
mkdir -p "$HOME/node"
tar -xJf /tmp/node.tar.xz -C "$HOME/node" --strip-components=1
echo "$HOME/node/bin" >> "$GITHUB_PATH"
"$HOME/node/bin/node" --version

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe updating to actions/setup-node@v6 would work out of the box?
Maybe some config option on it would solve the issue so we don't have to run a custom node installation step?

@giladgd giladgd changed the title feat: add riscv64 prebuilt binary feat: riscv64 prebuilt binaries Jun 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add linux riscv64 prebuilt binary

2 participants