feat: riscv64 prebuilt binaries#615
Conversation
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
giladgd
left a comment
There was a problem hiding this comment.
Thanks for the PR!
Could you please provide me with instructions on how I can access a riscv64 machine to manually test this on?
In case the llama.cpp build fails for some reason then I'll need a way to investigate and fix it in a faster manner than to trigger many CI runs.
Also, would you be willing to help me maintain riscv support in node-llama-cpp in the future?
| # electron has no riscv64 prebuilt binary; the binary build does not | ||
| # use it (the electron example is a separate job), so skip its | ||
| # postinstall download to let npm ci complete on riscv64. | ||
| ELECTRON_SKIP_BINARY_DOWNLOAD: ${{ matrix.config.artifact == 'linux-riscv64' && '1' || '' }} |
There was a problem hiding this comment.
We only use the electron types in the build anyway, so we can set that to always skip the binary download
| # npm on the unofficial riscv64 node intermittently throws internal | ||
| # TypeErrors (e.g. "key.match is not a function"); retry a few times. | ||
| npm ci || npm ci || npm ci |
There was a problem hiding this comment.
If we update npm will those issues go away? What do they stem from?
I'd prefer solving the issues from the root, since if this workaround stops working I won't know what to do or how to even approach solving this
| - name: Install Node.js (riscv64 unofficial build) | ||
| if: matrix.config.artifact == 'linux-riscv64' | ||
| run: | | ||
| set -e | ||
| # node 24 (npm 11): node 20's npm 10.8.2 trips "set.delete is not a | ||
| # function" during npm ci on riscv64, and several devDeps require | ||
| # node >= 24.10. The built binary still targets node 20 N-API. | ||
| NODE_VER=v24.16.0 | ||
| curl -fsSL -o /tmp/node.tar.xz "https://unofficial-builds.nodejs.org/download/release/${NODE_VER}/node-${NODE_VER}-linux-riscv64.tar.xz" | ||
| mkdir -p "$HOME/node" | ||
| tar -xJf /tmp/node.tar.xz -C "$HOME/node" --strip-components=1 | ||
| echo "$HOME/node/bin" >> "$GITHUB_PATH" | ||
| "$HOME/node/bin/node" --version | ||
|
|
There was a problem hiding this comment.
Maybe updating to actions/setup-node@v6 would work out of the box?
Maybe some config option on it would solve the issue so we don't have to run a custom node installation step?
Description of change
Adds a prebuilt binary for linux riscv64, so
npm install node-llama-cppon a RISC-V host picks up a ready-to-run binding instead of compiling llama.cpp from source.riscv64 already builds from source today, but that needs a full C/C++ toolchain and around 30 minutes of native compile time. This brings riscv64 in line with the platforms that already ship prebuilts.
What's in here:
@node-llama-cpp/linux-riscv64optional-dependency package (mirrorslinux-armv7l).compileLLamaCpp.ts: resolve the riscv64 prebuilt module whenarch === "riscv64".detectGlibc.ts: add the riscv64 multiarch lib path and theld-linux-riscv64-lp64d.so*loader names.movePrebuiltBinariesToStandaloneModules.ts: move thelinux-riscv64build output into the standalone module.build.yml: a native riscv64 build leg.A few CI notes, since the riscv64 leg is a bit different from the others:
ubuntu-24.04-riscvrunner from the RISE RISC-V GitHub App. That app has to be installed on the repo for the leg to get a runner. I'm happy to help coordinate that with RISE.dist(rolldown has no riscv64 binding yet), so the workflow splits into a dist-build job and a native binary-build job.npm ci,ELECTRON_SKIP_BINARY_DOWNLOAD=1, and gcc-14 (recent ggml uses RVV_Float16/zvfh intrinsics that only exist in GCC 14+).I validated this end-to-end on a fork: dist built on x64, binary compiled natively on the RISE runner, producing an ELF RISC-V
llama-addon.nodethat passes the smoke check. The split-job CI is the part I'm least sure about fitting your conventions, so let me know if you'd rather I structured it differently.Fixes #614
Pull-Request Checklist
masterbranchnpm run formatto apply eslint formattingnpm run testpasses with this changeFixes #0000