Make struct cast implementation pluggable by robert3005 · Pull Request #7684 · vortex-data/vortex

robert3005 · 2026-04-28T00:30:34Z

This pr moves Struct Cast implementation to be overridable by VortexSession.

The default implementation stays the same, i.e. it resolves the fields by name.

In follow up we will add back the positional cast and add both implementations
to the fuzzer

Signed-off-by: Robert Kruszewski github@robertk.io

codspeed-hq · 2026-04-28T00:34:31Z

Merging this PR will degrade performance by 10.6%

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

❌ 1 regressed benchmark
✅ 1168 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
❌	Simulation	`bitwise_not_vortex_buffer_mut[128]`	246.1 ns	275.3 ns	-10.6%

_{Comparing rk/movecasttoplugin (9761d99) with develop (922ea0a)}

connortsui20 · 2026-04-28T13:31:16Z

Is this related to #6900?

robert3005 · 2026-04-28T17:13:13Z

It is related. We need to acknowledge that different systems have different requirements and we need to be able to override the behaviour

robert3005 · 2026-04-29T00:31:28Z

as I was refactoring this logic I made #7710 to cleanup validity casting

Signed-off-by: Robert Kruszewski <github@robertk.io>

joseph-isaacs

Later I want to make cases composable (not just pluggable) so we could document that this is non-composable due to the vortex_ensure!

Signed-off-by: Robert Kruszewski <github@robertk.io>

* Remove more scalar_at from sparse arrays (#7439) | Benchmark case | Before (mean) | After (mean) | Speedup | |---|---:|---:|---:| | `extend_from_array_non_zctl_overlapping (1000, 8)` | `1.413 ms` | `80.71 us` | `17.5x` | | `extend_from_array_non_zctl_overlapping (1000, 32)` | `2.586 ms` | `83.67 us` | `30.9x` | | `extend_from_array_non_zctl_overlapping (10000, 8)` | `9.708 ms` | `560.3 us` | `17.3x` | | `extend_from_array_zctl (1000, 8)` | `1.116 ms` | `14.31 us` | `78.0x` | | `extend_from_array_zctl (1000, 64)` | `5.511 ms` | `38.68 us` | `142.5x` | | `extend_from_array_zctl (10000, 8)` | `11.01 ms` | `108.4 us` | `101.6x` | Before = commit `d72bf9b93` (pre-fix), After = commit `c4ad4e2ad` (the ListViewBuilder scalar-at removal fix). --------- Signed-off-by: Nicholas Gates <nick@nickgates.com> Signed-off-by: Robert Kruszewski <github@robertk.io> Co-authored-by: Robert Kruszewski <github@robertk.io> * chore: codspeed display names (#7752) As part of that we only run benchmarks on 100M element count inputs. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com> * Fix Claude workflow cancellation (#7775) Claude workflows cancel themselves at the moment.. Signed-off-by: Nicholas Gates <nick@nickgates.com> * Allow bot approvals (#7774) Since we use renovate-approval[bot] to approve renovate PRs. We still require 2 approvals for any non-whitelisted bot-authored PRs. But this will fix the queue of renovate PRs Signed-off-by: Nicholas Gates <nick@nickgates.com> * Update dependency lucide-react to v1.14.0 (#7769) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [lucide-react](https://lucide.dev) ([source](https://redirect.github.com/lucide-icons/lucide/tree/HEAD/packages/lucide-react)) | [`1.11.0` → `1.14.0`](https://renovatebot.com/diffs/npm/lucide-react/1.11.0/1.14.0) | ![age](https://developer.mend.io/api/mc/badges/age/npm/lucide-react/1.14.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/lucide-react/1.11.0/1.14.0?slim=true) | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>lucide-icons/lucide (lucide-react)</summary> ### [`v1.14.0`](https://redirect.github.com/lucide-icons/lucide/releases/tag/1.14.0): Version 1.14.0 [Compare Source](https://redirect.github.com/lucide-icons/lucide/compare/1.13.0...1.14.0) #### What's Changed - feat(icons): added `repeat-off` icon by [@jguddas](https://redirect.github.com/jguddas) in [#3102](https://redirect.github.com/lucide-icons/lucide/pull/3102) **Full Changelog**: <https://github.com/lucide-icons/lucide/compare/1.13.0...1.14.0> ### [`v1.13.0`](https://redirect.github.com/lucide-icons/lucide/releases/tag/1.13.0): Version 1.13.0 [Compare Source](https://redirect.github.com/lucide-icons/lucide/compare/1.12.0...1.13.0) #### What's Changed - fix(docs): sync URL params with UI state on categories page by [@taimar](https://redirect.github.com/taimar) in [#4111](https://redirect.github.com/lucide-icons/lucide/pull/4111) - feat(icons): add `waves-vertical` icon by [@jamiemlaw](https://redirect.github.com/jamiemlaw) in [#3867](https://redirect.github.com/lucide-icons/lucide/pull/3867) **Full Changelog**: <https://github.com/lucide-icons/lucide/compare/1.12.0...1.13.0> ### [`v1.12.0`](https://redirect.github.com/lucide-icons/lucide/releases/tag/1.12.0): Version 1.12.0 [Compare Source](https://redirect.github.com/lucide-icons/lucide/compare/1.11.0...1.12.0) #### What's Changed - feat(icon): add folder-bookmark icon by [@swastik7805](https://redirect.github.com/swastik7805) in [#4262](https://redirect.github.com/lucide-icons/lucide/pull/4262) - docs(readme): Update readme files by [@ericfennis](https://redirect.github.com/ericfennis) in [#4320](https://redirect.github.com/lucide-icons/lucide/pull/4320) - feat(icons): added `astroid` icon by [@whoisBugsbunny](https://redirect.github.com/whoisBugsbunny) in [#4217](https://redirect.github.com/lucide-icons/lucide/pull/4217) **Full Changelog**: <https://github.com/lucide-icons/lucide/compare/1.10.0...1.12.0> </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Nicholas Gates <gatesn@users.noreply.github.com> * fix: make ExtDType metadata deserialization total over byte input (#7782) Fixes #7773 Generated with Claude Code Signed-off-by: vortex-claude <vortex-claude@users.noreply.github.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> * Fix logo on benchmarks website (#7779) ## Summary They were inconsistent with our current branding material. Only the black version changes on the current [website](https://bench.vortex.dev/). ## Testing N/A Signed-off-by: Connor Tsui <connor.tsui20@gmail.com> * Update dependency typescript-eslint to v8.59.2 (#7764) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [typescript-eslint](https://typescript-eslint.io/packages/typescript-eslint) ([source](https://redirect.github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint)) | [`8.59.0` → `8.59.2`](https://renovatebot.com/diffs/npm/typescript-eslint/8.59.0/8.59.2) | ![age](https://developer.mend.io/api/mc/badges/age/npm/typescript-eslint/8.59.2?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/typescript-eslint/8.59.0/8.59.2?slim=true) | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>typescript-eslint/typescript-eslint (typescript-eslint)</summary> ### [`v8.59.2`](https://redirect.github.com/typescript-eslint/typescript-eslint/blob/HEAD/packages/typescript-eslint/CHANGELOG.md#8592-2026-05-04) [Compare Source](https://redirect.github.com/typescript-eslint/typescript-eslint/compare/v8.59.1...v8.59.2) This was a version bump only for typescript-eslint to align it with other projects, there were no code changes. See [GitHub Releases](https://redirect.github.com/typescript-eslint/typescript-eslint/releases/tag/v8.59.2) for more information. You can read about our [versioning strategy](https://typescript-eslint.io/users/versioning) and [releases](https://typescript-eslint.io/users/releases) on our website. ### [`v8.59.1`](https://redirect.github.com/typescript-eslint/typescript-eslint/blob/HEAD/packages/typescript-eslint/CHANGELOG.md#8591-2026-04-27) [Compare Source](https://redirect.github.com/typescript-eslint/typescript-eslint/compare/v8.59.0...v8.59.1) This was a version bump only for typescript-eslint to align it with other projects, there were no code changes. See [GitHub Releases](https://redirect.github.com/typescript-eslint/typescript-eslint/releases/tag/v8.59.1) for more information. You can read about our [versioning strategy](https://typescript-eslint.io/users/versioning) and [releases](https://typescript-eslint.io/users/releases) on our website. </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * [claude] feat(bench): emit v3 JSONL records and dual-write to bench server (#7780) ## Summary Prototype website: http://ec2-18-219-54-101.us-east-2.compute.amazonaws.com:3000/ This is the first step we should make before we cut over to the new benchmarks website on https://github.com/vortex-data/vortex/pull/7643 This PR allows the CI actions to additionally post data to a server (on my EC2 instance for now). We want to check that this actually works before we start using this for all of our CI. Note that this does NOT change how the current benchmarks website works, as this just does a few extra things on top of that. Also for reviewers, even though this looks like 1k LoC I think the logic here is not that hard to review, a lot of this is boilerplate you can skim over. Below is a bunch of AI-generated description: read at your own discretion. <details> Brings the v3 emitter and CI dual-write plumbing from `ct/benchmarks-v3` onto `develop` **without** the v3 server/website code. CI continues to write v2 results to S3 unchanged; v3 ingest is a side channel that no-ops until the deploy track sets `vars.V3_INGEST_URL`. This is item 2 ("CI ingestion wiring") of the v3 production-readiness checklist in [`benchmarks-website/planning/README.md`](https://github.com/vortex-data/vortex/blob/ct/benchmarks-v3/benchmarks-website/planning/README.md). The v3 website itself ships in a separate PR off `ct/benchmarks-v3` once dual-write is verified healthy in production. ### What's included **Rust emitter (`vortex-bench`)** - New `vortex-bench/src/v3.rs`: one record per `kind` (`query_measurement`, `compression_time`, `compression_size`, `random_access_time`, `vector_search_run`) plus a serde-tagged `V3Record` enum, JSONL writer, and `insta` snapshot tests. Field shapes match [`02-contracts.md`](https://github.com/vortex-data/vortex/blob/ct/benchmarks-v3/benchmarks-website/planning/02-contracts.md). - `Dataset::v3_dataset_dims()` (default `(name(), None)`) lets Public-BI map to `(public-bi, <subset>)`. - `compress` and `runner` capture per-iteration timings and provide `SqlBenchmarkRunner::v3_records()`. **Benchmark binaries** - `compress-bench`, `datafusion-bench`, `duckdb-bench`, `lance-bench`, `random-access-bench`, `vector-search-bench` all gain `--gh-json-v3 <path>`. Bare records, no envelope. The legacy `-d gh-json -o ...` flow is untouched. **`bench-orchestrator`** - `vx-bench run --gh-json-v3 <path>` plumbs the flag through to the underlying benchmark binary. **`scripts/post-ingest.py`** (Python 3, stdlib only) - Reads JSONL, fills the `commit` envelope from `git show`, wraps in `{run_meta, commit, records}`, POSTs to `/api/ingest` with `Authorization: Bearer ${INGEST_BEARER_TOKEN}`. Exits non-zero on 4xx/5xx. No retry/spool — deferred. **Workflows** - `.github/workflows/bench.yml` and `sql-benchmarks.yml` add `--gh-json-v3 results.v3.jsonl` to the bench runs and a follow-up "Ingest results to v3 server" step. - New `.github/workflows/v3-commit-metadata.yml` POSTs an empty envelope on every push to `develop` so the v3 `commits` dim stays populated even when no benchmark ran. ### What's NOT included (intentionally) - Anything under `benchmarks-website/` — the v2 React/Node app stays in production unchanged. - Workspace member additions for `benchmarks-website/server` and `benchmarks-website/migrate` — those crates don't exist on `develop` yet. - `.github/workflows/ci.yml` and `publish-bench-server.yml` changes — they reference `vortex-bench-server`, which is also v3-server-only. ## Risk **Zero.** The v3 ingest step is gated on `vars.V3_INGEST_URL != ''` and `continue-on-error: true`. If the V3 server is down, the variable is unset, or the bearer secret is missing, the workflow no-ops and the v2 path keeps writing to S3 unchanged. The Rust emitter writes JSONL to a local file only; no network egress from the binaries themselves. ## Verify A CI run on this branch should show the new "Ingest results to v3 server" step running and POSTing successfully to the EC2 host at `vars.V3_INGEST_URL`. ## Follow-up The v3 website itself (server, migrator, web UI) ships in a separate PR off `ct/benchmarks-v3` once dual-write is verified healthy in production. Outbox-style retry on failed POSTs is also a follow-up — not built until we observe a failure in the wild. ## Test plan - [x] `cargo build -p vortex-bench` — clean. - [x] `cargo nextest run -p vortex-bench` — 49/49 pass, including 7 new v3 snapshot tests. - [x] `cargo build -p compress-bench -p datafusion-bench -p duckdb-bench -p lance-bench -p random-access-bench -p vector-search-bench` — clean. - [x] All six benchmark binaries print `--gh-json-v3 <GH_JSON_V3>` in `--help`. - [x] `python3 scripts/post-ingest.py --help` — clean. - [x] `pytest bench-orchestrator/tests/test_executor.py` — 5/5 pass, including 2 new `gh_json_v3` tests. - [x] `cargo +nightly fmt --all` — no diff. - [x] `cargo clippy --all-targets --all-features -p vortex-bench` — clean. - [x] `cargo clippy --all-targets -p compress-bench -p datafusion-bench -p lance-bench -p random-access-bench -p vector-search-bench` — clean. `duckdb-bench` skipped (transitively triggers a pre-existing `cognitive_complexity` lint in `vortex-duckdb/src/convert/expr.rs:47`, present on `develop` and unrelated to these changes). - [x] `yamllint --strict -c .yamllint.yaml` on the three changed/new workflow files — clean. - [x] `./scripts/public-api.sh` — N/A. All touched Rust crates have `publish = false`. - [ ] Real round-trip against the EC2 host — verifies once this branch triggers a CI bench run with `V3_INGEST_URL` set. --- _Generated by [Claude Code](https://claude.ai/code/session_0154XbxhgQztmbrQfJ4ZSxVo)_ </details> --------- Signed-off-by: Claude <noreply@anthropic.com> Signed-off-by: Connor Tsui <connor.tsui20@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * Benchmarks Website Version 3 (#7643) ## Summary Rewrites the benchmarks website. Replaces the static `data.json.gz` model with a single Rust server binary that owns a DuckDB database and accepts `POST /api/ingest` from CI. ## Design - Single binary: axum + maud (SSR HTML) + DuckDB + Chart.js. All static assets `include_bytes!`'d. - 5 fact tables (compression time, query measurement, vector search, RAG, random access). Backup is a file copy. - Ingest: versioned JSON envelopes, bearer-token gated. - Migrator ports v2 history forward via a classifier that routes each record to a fact table or skips it with a typed reason. - Charts/groups slug-addressed, URL round-trip with no DB lookup. - Routes: `/`, `/chart/{slug}`, `/group/{slug}`, `GET /api/chart/{slug}`. - Deploy: one binary, one DuckDB file, one `INGEST_BEARER_TOKEN`. UI/UX is still TBD — the relational backend opens up options we didn't have before. Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> * Lock file maintenance (#7771) This PR contains the following updates: | Update | Change | |---|---| | lockFileMaintenance | All locks refreshed | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. 🔧 This Pull Request updates lock files to use the latest dependency versions. --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - "before 4am on monday" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Lock file maintenance (#7772) This PR contains the following updates: | Update | Change | |---|---| | lockFileMaintenance | All locks refreshed | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. 🔧 This Pull Request updates lock files to use the latest dependency versions. --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - "before 4am on monday" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update Gradle to v9.5.0 (#7770) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [gradle](https://gradle.org) ([source](https://redirect.github.com/gradle/gradle)) | minor | `9.4.1` → `9.5.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>gradle/gradle (gradle)</summary> ### [`v9.5.0`](https://redirect.github.com/gradle/gradle/compare/v9.4.1...v9.5.0) [Compare Source](https://redirect.github.com/gradle/gradle/compare/v9.4.1...v9.5.0) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency globals to v17.6.0 (#7768) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [globals](https://redirect.github.com/sindresorhus/globals) | [`17.5.0` → `17.6.0`](https://renovatebot.com/diffs/npm/globals/17.5.0/17.6.0) | ![age](https://developer.mend.io/api/mc/badges/age/npm/globals/17.6.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/globals/17.5.0/17.6.0?slim=true) | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>sindresorhus/globals (globals)</summary> ### [`v17.6.0`](https://redirect.github.com/sindresorhus/globals/compare/v17.5.0...6b15870f1c08b60b5b57afe45a703d9ed0be39bc) [Compare Source](https://redirect.github.com/sindresorhus/globals/compare/v17.5.0...v17.6.0) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update dependency eslint to v10.3.0 (#7767) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [eslint](https://eslint.org) ([source](https://redirect.github.com/eslint/eslint)) | [`10.2.1` → `10.3.0`](https://renovatebot.com/diffs/npm/eslint/10.2.1/10.3.0) | ![age](https://developer.mend.io/api/mc/badges/age/npm/eslint/10.3.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/eslint/10.2.1/10.3.0?slim=true) | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>eslint/eslint (eslint)</summary> ### [`v10.3.0`](https://redirect.github.com/eslint/eslint/compare/v10.2.1...78892043a36da4aa7640b59c99344b00c181048a) [Compare Source](https://redirect.github.com/eslint/eslint/compare/v10.2.1...v10.3.0) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update crate-ci/typos action to v1.46.0 (#7766) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [crate-ci/typos](https://redirect.github.com/crate-ci/typos) | action | minor | `v1.45.1` → `v1.46.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>crate-ci/typos (crate-ci/typos)</summary> ### [`v1.46.0`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.46.0) [Compare Source](https://redirect.github.com/crate-ci/typos/compare/v1.45.2...v1.46.0) #### \[1.46.0] - 2026-04-30 ##### Features - Updated the dictionary with the [April 2026](https://redirect.github.com/crate-ci/typos/issues/1531) changes ### [`v1.45.2`](https://redirect.github.com/crate-ci/typos/releases/tag/v1.45.2) [Compare Source](https://redirect.github.com/crate-ci/typos/compare/v1.45.1...v1.45.2) ##### \[1.45.2] - 2026-04-27 ##### Fixes - Ignore ssh [`ed25519`](https://redirect.github.com/crate-ci/typos/commit/ed25519) public keys </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update storybook monorepo to v10.3.6 (#7765) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [@storybook/addon-docs](https://redirect.github.com/storybookjs/storybook/tree/next/code/addons/docs) ([source](https://redirect.github.com/storybookjs/storybook/tree/HEAD/code/addons/docs)) | [`10.3.5` → `10.3.6`](https://renovatebot.com/diffs/npm/@storybook%2faddon-docs/10.3.5/10.3.6) | ![age](https://developer.mend.io/api/mc/badges/age/npm/@storybook%2faddon-docs/10.3.6?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/@storybook%2faddon-docs/10.3.5/10.3.6?slim=true) | | [@storybook/react-vite](https://redirect.github.com/storybookjs/storybook/tree/next/code/frameworks/react-vite) ([source](https://redirect.github.com/storybookjs/storybook/tree/HEAD/code/frameworks/react-vite)) | [`10.3.5` → `10.3.6`](https://renovatebot.com/diffs/npm/@storybook%2freact-vite/10.3.5/10.3.6) | ![age](https://developer.mend.io/api/mc/badges/age/npm/@storybook%2freact-vite/10.3.6?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/@storybook%2freact-vite/10.3.5/10.3.6?slim=true) | | [eslint-plugin-storybook](https://redirect.github.com/storybookjs/storybook/tree/next/code/lib/eslint-plugin#readme) ([source](https://redirect.github.com/storybookjs/storybook/tree/HEAD/code/lib/eslint-plugin)) | [`10.3.5` → `10.3.6`](https://renovatebot.com/diffs/npm/eslint-plugin-storybook/10.3.5/10.3.6) | ![age](https://developer.mend.io/api/mc/badges/age/npm/eslint-plugin-storybook/10.3.6?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/eslint-plugin-storybook/10.3.5/10.3.6?slim=true) | | [storybook](https://storybook.js.org) ([source](https://redirect.github.com/storybookjs/storybook/tree/HEAD/code/core)) | [`10.3.5` → `10.3.6`](https://renovatebot.com/diffs/npm/storybook/10.3.5/10.3.6) | ![age](https://developer.mend.io/api/mc/badges/age/npm/storybook/10.3.6?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/storybook/10.3.5/10.3.6?slim=true) | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>storybookjs/storybook (@storybook/addon-docs)</summary> ### [`v10.3.6`](https://redirect.github.com/storybookjs/storybook/blob/HEAD/CHANGELOG.md#1036) [Compare Source](https://redirect.github.com/storybookjs/storybook/compare/v10.3.5...v10.3.6) - ESLint: Update deprecated [@storybook/test](https://redirect.github.com/storybook/test) reference to storybook/test - [#34430](https://redirect.github.com/storybookjs/storybook/pull/34430), thanks [@venkat22022202](https://redirect.github.com/venkat22022202)! - React: Add optional React type peers to packages that publish React-based declarations - [#34641](https://redirect.github.com/storybookjs/storybook/pull/34641), thanks [@copilot-swe-agent](https://redirect.github.com/copilot-swe-agent)! - UI: Fix global background + color styles affecting stories - [#34071](https://redirect.github.com/storybookjs/storybook/pull/34071), thanks [@Axadali](https://redirect.github.com/Axadali)! </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update release-drafter/release-drafter action to v7.2.1 (#7763) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [release-drafter/release-drafter](https://redirect.github.com/release-drafter/release-drafter) | action | patch | `v7.2.0` → `v7.2.1` | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>release-drafter/release-drafter (release-drafter/release-drafter)</summary> ### [`v7.2.1`](https://redirect.github.com/release-drafter/release-drafter/releases/tag/v7.2.1) [Compare Source](https://redirect.github.com/release-drafter/release-drafter/compare/v7.2.0...v7.2.1) ### What's Changed #### Bug Fixes - fix: initial-commits-since in config not overwritten by input ([#1593](https://redirect.github.com/release-drafter/release-drafter/issues/1593)) [@sroebert](https://redirect.github.com/sroebert) - fix: clarify prerelease-identifier behavior and precedence in configuration ([#1594](https://redirect.github.com/release-drafter/release-drafter/issues/1594)) [@neilime](https://redirect.github.com/neilime) #### Maintenance - chore: disable "No version input..." warning ([#1595](https://redirect.github.com/release-drafter/release-drafter/issues/1595)) [@cchanche](https://redirect.github.com/cchanche) **Full Changelog**: <https://github.com/release-drafter/release-drafter/compare/v7.2.0...v7.2.1> </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update taiki-e/install-action digest to 711e1c3 (#7762) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [taiki-e/install-action](https://redirect.github.com/taiki-e/install-action) ([changelog](https://redirect.github.com/taiki-e/install-action/compare/cf525cb33f51aca27cd6fa02034117ab963ff9f1..711e1c3275189d76dcc4d34ddea63bf96ac49090)) | action | digest | `cf525cb` → `711e1c3` | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update anthropics/claude-code-action digest to 2cc1ac1 (#7761) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [anthropics/claude-code-action](https://redirect.github.com/anthropics/claude-code-action) ([changelog](https://redirect.github.com/anthropics/claude-code-action/compare/567fe954a4527e81f132d87d1bdbcc94f7737434..2cc1ac1331eac7a6a96d716dd204dd2888d0fcd2)) | action | digest | `567fe95` → `2cc1ac1` | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Remove bad action (#7786) ## Summary Accidentally merged this... ## Testing N/A Signed-off-by: Connor Tsui <connor.tsui20@gmail.com> * Fix weird signature of with_slots functions (#7758) with_slots doesn't need to pass the original ref to the arrayinner methods as unique ownership arc is handled earlier Signed-off-by: Robert Kruszewski <github@robertk.io> * skip[fuzz]: less jobs to run for fuzzer (#7788) stop oom Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> * Try to fix benchmakrs-website tests on windows (#7794) If we drop the db before we open we should avoid the double open problem Signed-off-by: Robert Kruszewski <github@robertk.io> * Make struct cast implementation pluggable (#7684) This pr moves Struct Cast implementation to be overridable by VortexSession. The default implementation stays the same, i.e. it resolves the fields by name. In follow up we will add back the positional cast and add both implementations to the fuzzer --------- Signed-off-by: Robert Kruszewski <github@robertk.io> * Fix benchmark group parsing for v3 (#7797) ## Summary Fixes some issues with identifying what benchmark data actually is. ## Testing N/A Signed-off-by: Connor Tsui <connor.tsui20@gmail.com> * Allow Claude to open PRs (#7798) I forgot to add this part! Signed-off-by: Nicholas Gates <nick@nickgates.com> * Add Patches lookup benchmarks (#7795) Isolating benchmarks before this optimization: https://github.com/vortex-data/vortex/pull/7753 --------- Signed-off-by: Baris Palaska <barispalaska@gmail.com> * bench: CUDA sync load-to-device benchmark (#7799) This benchmark provides the baseline for GPU data loading exercising our synchronous logic. In this naive case, the data is copied to the GPU from host memory. This does not include loading the buffer from a file. This change will be followed up by benchmarks for different copy strategies. Benchmarks for a GH200: ``` cuda/cuda/load_to_device/ensure_on_device_sync/16MiB time: [1.1733 ms 1.1966 ms 1.2282 ms] thrpt: [12.722 GiB/s 13.058 GiB/s 13.317 GiB/s] change: time: [-2.3933% +0.2367% +3.1250%] (p = 0.88 > 0.05) thrpt: [-3.0303% -0.2362% +2.4520%] cuda/cuda/load_to_device/ensure_on_device_sync/64MiB time: [3.0456 ms 3.0634 ms 3.0819 ms] thrpt: [20.280 GiB/s 20.402 GiB/s 20.522 GiB/s] change: time: [+0.0301% +1.0798% +2.0330%] (p = 0.06 > 0.05) thrpt: [-1.9925% -1.0682% -0.0301%] cuda/cuda/load_to_device/ensure_on_device_sync/256MiB time: [11.363 ms 11.399 ms 11.434 ms] thrpt: [21.864 GiB/s 21.932 GiB/s 22.000 GiB/s] change: time: [-0.1484% +0.1656% +0.4805%] (p = 0.37 > 0.05) thrpt: [-0.4782% -0.1653% +0.1486%] cuda/cuda/load_to_device/ensure_on_device_sync/1GiB time: [42.364 ms 42.832 ms 43.712 ms] thrpt: [22.877 GiB/s 23.347 GiB/s 23.605 GiB/s] change: time: [-6.7679% -0.1064% +4.4406%] (p = 0.98 > 0.05) thrpt: [-4.2518% +0.1065% +7.2592%] ``` Signed-off-by: Alexander Droste <alexander.droste@protonmail.com> * Add more benchmarks for intersect by rank (#7800) Add more benchmarks for intersect_by_rank Signed-off-by: Robert Kruszewski <github@robertk.io> * feat[vortex-cuda]: GPU FSST decompression kernel (#7776) ## Summary This commit implements on-GPU decompression of the existing FSST encoding. This kernel achieves ~42% max throughput utilization as compared to the `throughput_cuda` benchmark on a DGX spark. CPU work is required to compute the output offsets. The core performance win is buffering up to 24 bytes of decompressed data in three u64 registers and emitting the widest aligned stores possible up to u128 (st.global.v2.u64). The 256-entry symbol table (≤ 2 KB) is read directly from global memory. Staging it into shared memory measured ~3% slower at 10M rows and ~15% slower at 1M rows. The hypothesis is that L1 already holds the table after a few iterations and the explicit shared copy adds bank-conflict latency on the warp-divergent symbols[code] reads; the gap is wider at 1M because the kernel is less bandwidth-bound there. Further optimizations would require an encoding change. Splits-style intra-string parallelism (one GPU thread per ~32-compressed-byte chunk instead of per-string) was prototyped on top of this kernel and measured an additional +30% kernel throughput at 1M clickbench URLs, +26% at 5M, +12% at 10M. Four kernel variants are generated for the unsigned widths of codes_offsets (u8/u16/u32/u64); signed integer ptypes are reinterpreted as their unsigned equivalent on the Rust side, so the bit pattern is preserved without copying.   Addresses: #6538  ## Testing  Unit tests against the CPU implementation on small and larger dataset. Signed-off-by: Alfonso Subiotto Marques <alfonso.subiotto@polarsignals.com> * Preload data on benchmarks website (#7802) ## Summary Right now, the v3 benchmarks website only loads data when a user clicks on a benchmark group. This change preloads the data as soon as the user lands so that by the time the user loads everything, the charts are already rendered and it feels instantaneous. ## Testing N/A Signed-off-by: Connor Tsui <connor.tsui20@gmail.com> * Improve UI of v3 benchmarks website (#7801) ## Summary Adds filtering per benchmark group as well as per-group linear vs log scale. Also cleans up some other things for mobile. ## Testing N/A Signed-off-by: Connor Tsui <connor.tsui20@gmail.com> * Fix zoom on v3 benchmarks website (#7803) ## Summary Accidentally hangs when loading data for the first time. ## Testing N/A Signed-off-by: Connor Tsui <connor.tsui20@gmail.com> * Run vortex-mask benchmarks with codspeed (#7804) We didn't use to have benchmarks there, now we do Signed-off-by: Robert Kruszewski <github@robertk.io> * Update dependency pip to v26.1 [SECURITY] (#7805) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [pip](https://redirect.github.com/pypa/pip) ([changelog](https://pip.pypa.io/en/stable/news/)) | `26.0` → `26.1` | ![age](https://developer.mend.io/api/mc/badges/age/pypi/pip/26.1?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/pip/26.0/26.1?slim=true) | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### pip Vulnerable to Inclusion of Functionality from Untrusted Control Sphere [CVE-2026-6357](https://nvd.nist.gov/vuln/detail/CVE-2026-6357) / [GHSA-jp4c-xjxw-mgf9](https://redirect.github.com/advisories/GHSA-jp4c-xjxw-mgf9) <details> <summary>More information</summary> #### Details pip prior to version 26.1 would run self-update check functionality after installing wheel files which required importing well-known Python modules names. These module imports were intentionally deferred to increase startup time of the pip CLI. The patch changes self-update functionality to run before wheels are installed to prevent newly-installed modules from being imported shortly after the installation of a wheel package. Users should still review package contents prior to installation. #### Severity - CVSS Score: 5.3 / 10 (Medium) - Vector String: `CVSS:4.0/AV:L/AC:L/AT:P/PR:H/UI:A/VC:H/VI:H/VA:N/SC:N/SI:N/SA:N` #### References - [https://nvd.nist.gov/vuln/detail/CVE-2026-6357](https://nvd.nist.gov/vuln/detail/CVE-2026-6357) - [https://github.com/pypa/pip/pull/13923](https://redirect.github.com/pypa/pip/pull/13923) - [https://ichard26.github.io/blog/2026/04/whats-new-in-pip-26.1/#security-fixes](https://ichard26.github.io/blog/2026/04/whats-new-in-pip-26.1/#security-fixes) - [http://www.openwall.com/lists/oss-security/2026/04/27/7](http://www.openwall.com/lists/oss-security/2026/04/27/7) - [https://github.com/pypa/pip/commit/b369bfc96cc524e00c267e1693290e6599c36bad](https://redirect.github.com/pypa/pip/commit/b369bfc96cc524e00c267e1693290e6599c36bad) - [https://github.com/advisories/GHSA-jp4c-xjxw-mgf9](https://redirect.github.com/advisories/GHSA-jp4c-xjxw-mgf9) This data is provided by the [GitHub Advisory Database](https://redirect.github.com/advisories/GHSA-jp4c-xjxw-mgf9) ([CC-BY 4.0](https://redirect.github.com/github/advisory-database/blob/main/LICENSE.md)). </details> --- ### Release Notes <details> <summary>pypa/pip (pip)</summary> ### [`v26.1`](https://redirect.github.com/pypa/pip/compare/26.0.1...26.1) [Compare Source](https://redirect.github.com/pypa/pip/compare/26.0.1...26.1) ### [`v26.0.1`](https://redirect.github.com/pypa/pip/compare/26.0...26.0.1) [Compare Source](https://redirect.github.com/pypa/pip/compare/26.0...26.0.1) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - "" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * perf[array]: outline the array parts data from the DynArray (#7750) Change the internals of `ArrayRef` to allow access to common fields without dyn dispatch. ```rust + pub struct ArrayRef(Arc<ArrayInner<dyn DynArrayData>>); - pub struct ArrayRef(Arc<dyn DynArray>); ``` ```rust pub(crate) struct ArrayInner<D: ?Sized> { pub(crate) len: usize, pub(crate) encoding_id: ArrayId, pub(crate) dtype: DType, pub(crate) slots: Vec<Option<ArrayRef>>, pub(crate) stats: ArrayStats, pub(crate) data: D, // must be last for unsized coercion } ``` Applied the following renames: - `ArrayInner` -> `ArrayData` - `<V as VTable>::ArrayData` -> `<V as VTable>::TypedArrayData` - `DynArray` -> `DynArrayData` --------- Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Signed-off-by: Robert Kruszewski <github@robertk.io> Co-authored-by: Robert Kruszewski <github@robertk.io> * use rstest for some test cleanups (#7813) Just using rstest to bring the line count down a little. --------- Signed-off-by: Andrew Duffy <andrew@a10y.dev> * bench: CUDA host-to-device copy modes (#7815) Compare pageable host memory with `cuMemHostAlloc` pinned allocations using default flags and `WRITECOMBINED`. Benchmark results on a GH200: ``` cuda/load_to_device/memcpy_htod/pageable/1GiB time: [10.717 ms 10.754 ms 10.793 ms] thrpt: [92.649 GiB/s 92.989 GiB/s 93.306 GiB/s] cuda/load_to_device/memcpy_htod/pinned_default/1GiB time: [10.085 ms 10.265 ms 10.527 ms] thrpt: [94.992 GiB/s 97.423 GiB/s 99.159 GiB/s] cuda/load_to_device/memcpy_htod/pinned_write_combined/1GiB time: [21.043 ms 21.127 ms 21.204 ms] thrpt: [47.161 GiB/s 47.333 GiB/s 47.522 GiB/s] cuda/load_to_device/device_alloc_memcpy_htod/pageable/1GiB time: [42.625 ms 42.704 ms 42.781 ms] thrpt: [23.375 GiB/s 23.417 GiB/s 23.460 GiB/s] cuda/load_to_device/device_alloc_memcpy_htod/pinned_default/1GiB time: [41.864 ms 42.186 ms 42.592 ms] thrpt: [23.478 GiB/s 23.704 GiB/s 23.887 GiB/s] change: time: [+1.7580% +2.5859% +3.6570%] (p = 0.00 < 0.05) thrpt: [-3.5280% -2.5207% -1.7276%] cuda/load_to_device/device_alloc_memcpy_htod/pinned_write_combined/1GiB time: [51.986 ms 52.077 ms 52.166 ms] thrpt: [19.170 GiB/s 19.202 GiB/s 19.236 GiB/s] ``` The insights here being, `WRITECOMBINED` yields significantly slower copy performance, whilst pageable host memory is roughly on par with pinned host memory (without using `WRITECOMBINED`). Signed-off-by: Alexander Droste <alexander.droste@protonmail.com> * feat: check VortexReadAt::read_at results in the I/O driver (#7783) This ensures misbehaving VortexReadAt implementations cannot trigger later bad behavior by the I/O driver. Instead, the driver raises an error as close as possible to the badly behaving VortexReadAt implementation. --------- Signed-off-by: Daniel King <dan@spiraldb.com> * fix: partition in filter should err (not panic). (#7816) fixes p1: https://github.com/vortex-data/vortex/issues/7808 Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> * [claude] Move benchmarks-website planning docs into code comments (#7810) ## Summary This PR consolidates the benchmarks-website v3 planning documentation from the `planning/` directory into inline code comments and a top-level `README.md`, making the architecture and wire contracts discoverable alongside the implementation. **Changes:** - Removed `planning/` directory (00-overview.md, 01-schema.md, 02-contracts.md, AGENTS.md, README.md, decisions.md, deferred.md, benchmark-mapping.md, components/*.md) - Added `benchmarks-website/README.md` with v2/v3 architecture overview and deployment notes - Added `benchmarks-website/AGENTS.md` with contributor guidelines for the v3 codebase - Migrated schema design principles and column contracts from `01-schema.md` into `server/src/schema.rs` doc comments - Migrated wire-format contracts from `02-contracts.md` into `server/src/records.rs` and `vortex-bench/src/v3.rs` doc comments - Migrated ingest handler details into `server/src/ingest.rs` doc comments - Updated module-level docs in `server/src/lib.rs`, `server/src/db.rs`, `server/src/slug.rs`, `server/src/error.rs`, `server/src/app.rs`, `server/src/api/mod.rs`, `server/src/api/dto.rs`, `migrate/src/v2.rs`, `migrate/src/classifier.rs`, `migrate/src/commits.rs`, `migrate/src/verify.rs`, and `vortex-bench/src/datasets/mod.rs` to reference the new doc locations and clarify their roles - Made test helper visibility explicit in `server/tests/common/mod.rs` (pub → pub(crate)) The planning docs were comprehensive but lived outside the codebase, making them easy to miss during development and harder to keep in sync with implementation. This change embeds the essential contracts and design rationale where developers will encounter them: in the module docs and type definitions they're working with. ## Testing Existing tests pass. No functional changes to the server, migrator, or emitter — only documentation reorganization and visibility adjustments. https://claude.ai/code/session_012kmSMpZiKfQcHw3PedaFR4 --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com> * fix(ffi): double-free in partition scan on error (#7817) ## Summary Fix for the second part of: #7808 ``` (gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3 0x000076a38cc4527e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #4 0x000076a38cc288ff in __GI_abort () at ./stdlib/abort.c:79 #5 0x000076a38cc297b6 in __libc_message_impl (fmt=fmt@entry=0x76a38cdce8d7 "%s\n") at ../sysdeps/posix/libc_fatal.c:134 #6 0x000076a38cca8ff5 in malloc_printerr ( str=str@entry=0x76a38cdd1bf0 "free(): double free detected in tcache 2") at ./malloc/malloc.c:5775 #7 0x000076a38ccab55f in _int_free (av=0x76a38ce03ac0 <main_arena>, p=<optimized out>, have_lock=0) at ./malloc/malloc.c:4541 #8 0x000076a38ccaddce in __GI___libc_free (mem=0x5be5cd9632c0) at ./malloc/malloc.c:3398 #9 0x000076a38eb6807e in alloc::alloc::dealloc (ptr=0x5be5cd9632c0, layout=...) at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/alloc.rs:114 #10 alloc::alloc::{impl#1}::deallocate (self=0x5be5cd95f708, ptr=..., layout=...) at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/alloc.rs:271 #11 0x000076a38ead9a64 in alloc::boxed::{impl#8}::drop<dyn vortex_scan::Partition, alloc::alloc::Global> (self=0x5be5cd95f6f8) at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1666 #12 0x000076a38ead349e in core::ptr::drop_in_place<alloc::boxed::Box<dyn vortex_scan::Partition, alloc::alloc::Global>> () at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:804 #13 0x000076a38e8764de in core::ptr::drop_in_place<vortex_ffi::scan::VxPartitionScan> () at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:804 #14 0x000076a38e876fb8 in core::ptr::drop_in_place<alloc::boxed::Box<vortex_ffi::scan::VxPartitionScan, alloc::alloc::Global>> () at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:804 #15 0x000076a38e87f2f5 in core::mem::drop<alloc::boxed::Box<vortex_ffi::scan::VxPartitionScan, alloc::alloc::Global>> (_x=0x5be5cd95f6f0) at /home/ubuntu/.rustup/toolchains/1.91.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/mem/mod.rs:961 #16 0x000076a38e84efa7 in vortex_ffi::scan::vx_partition_free (ptr=0x5be5cd95f6f0) at vortex-ffi/src/macros.rs:295 #17 0x00005be5b0c81126 in operator() (__closure=0x7fff2208a8b0) at /home/ubuntu/vortex/vortex-ffi/test/scan.cpp:940 ``` ## Testing Verifying existing behavior is maintained. Signed-off-by: Dergousov Maksim <dergousovmaxim99@gmail.com> * fix[sparse]: ensure cached patches are updated and use `PatchesData` (#7811) Sparse array help on to a cached via over the slots that form a Patch, make this read only. Include `PatchesData` to hold the non children/slot part of Patches. --------- Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> * fix[py]: remove mimalloc in library (#7826) * bench[gpu]: overlap of compute and copying (#7828) More benchmarks to pin down the perf implications of core primitives. Signed-off-by: Alexander Droste <alexander.droste@protonmail.com> * Use smallvec for internal stats storage (#7823) Instead of always allocating fixed sized vec we can use a smallvec instead --------- Signed-off-by: Robert Kruszewski <github@robertk.io> * feat: iterative execution for SparseArray (#7711) ensures sparse array is executed iteratively. see https://github.com/vortex-data/vortex/issues/7674 --------- Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> * bench[gpu]: CUDA device memory pool benchmarks (#7831) Adds benchmarks that compared pooled vs unpooled memory allocation. Main insight being that the memory release threshold for a pool can be adjusted by the `CU_MEMPOOL_ATTR_RELEASE_THRESHOLD` attribute. ``` cuda/core_primitives/device_alloc_reuse/default_pool/1GiB time: [31.125 ms 31.208 ms 31.292 ms] change: [-5.1888% -2.7137% -0.3753%] (p = 0.04 < 0.05) cuda/core_primitives/device_alloc_reuse/default_pool_75pct_threshold/1GiB time: [1.2864 µs 1.3728 µs 1.4528 µs] change: [-7.2072% +0.4684% +8.9330%] (p = 0.91 > 0.05) ``` Signed-off-by: Alexander Droste <alexander.droste@protonmail.com> * Add a new AggregateFn for UncompressedSize (#7715) ## Summary This stat turns out to be [very useful](https://github.com/vortex-data/vortex/pull/7628) for helping execution engines plan joins better, but its currently requires a full-decompression. This PR includes the basic structure for a new AggregateFn, including implementation for all canonical types. --------- Signed-off-by: Adam Gutglick <adam@spiraldb.com> * Fix features of vortex-python (#7837) We were missing dev features, fixes #7737 Signed-off-by: Robert Kruszewski <github@robertk.io> * skip[array]: add a benchmark of the optimize loop (#7840) Include a benchmark which uses the array optimize loop checking for the performance or this and lots of small array being created. --------- Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> * perf[array]: small vec for slots (#7825) try out small vec for arrays (either 2 or 4), here we try 4 --------- Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> * feat: all_non_distinct aggregation make fuzzer faster (#7827) Add a fused all(non_distinct) aggregation (which should be a streaming `non_distinct` and all). Use this for array equality. --------- Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> * skip[fuzz]: run the fuzzer for longer less times (#7842) The overhead of corpus validation is around 30 mins Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> * Improve intersect_by_rank performance (#7744) Intersect by rank is an operation for performing filter on top of another filter. This pr improves: Dense on Dense masks: 10-18x Sparse on Dense masks: 8x Sparse on Sparse masks: 2x --------- Signed-off-by: Robert Kruszewski <github@robertk.io> * Stop ignoring tests (#7844) These were ignored for reasons unknown. I removed ignores, fixed one case of actually failing test and converted one of them to an expected panic fix #7835 Signed-off-by: Robert Kruszewski <github@robertk.io> * fix[bitpacked]: slice patches in execute method (not reduce). (#7839) Problem: BitPacked's SliceReduce implementation called patches.slice(), which reads buffers (search_index does binary search on buffer data, execute_scalar executes on chunk offsets). This violates the SliceReduce contract that requires metadata-only operations. --------- Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> * Collect slots directly into smallvec instead of using vec and into (#7847) These copies are unnecessary Signed-off-by: Robert Kruszewski <github@robertk.io> * TurboQuant again! (#7829) ## Summary Tracking issue: https://github.com/vortex-data/vortex/issues/7830 Moves TurboQuant out of `vortex-tensor` into a new `vortex-turboquant` crate. The first commit was mostly copying and pasting a bunch of code, as well as adding the `unpack` method to replace canonicalization. The second commit was cleaning up everything holistically. A lot of the code in `vortex-tensor` was reviewed pretty lightly because we knew that it was unstable, but now that we are more certain about the implementation (not necessarily about the exact design, but the actual implementation of the TQ algorithms), I think it is worth reviewing everything as a whole. ## Testing These tests were mostly there before, but now there are more! --------- Signed-off-by: Connor Tsui <connor.tsui20@gmail.com> * Faster Mask::from_slices (#7857) Do not iterate individual bits to construct a mask from slices Signed-off-by: Nicholas Gates <nick@nickgates.com> * Update react monorepo to v19.2.6 (#7866) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [react](https://react.dev/) ([source](https://redirect.github.com/facebook/react/tree/HEAD/packages/react)) | [`19.2.5` → `19.2.6`](https://renovatebot.com/diffs/npm/react/19.2.5/19.2.6) | ![age](https://developer.mend.io/api/mc/badges/age/npm/react/19.2.6?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/react/19.2.5/19.2.6?slim=true) | | [react-dom](https://react.dev/) ([source](https://redirect.github.com/facebook/react/tree/HEAD/packages/react-dom)) | [`19.2.5` → `19.2.6`](https://renovatebot.com/diffs/npm/react-dom/19.2.5/19.2.6) | ![age](https://developer.mend.io/api/mc/badges/age/npm/react-dom/19.2.6?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/react-dom/19.2.5/19.2.6?slim=true) | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>facebook/react (react)</summary> ### [`v19.2.6`](https://redirect.github.com/facebook/react/releases/tag/v19.2.6): 19.2.6 (May 6th, 2026) [Compare Source](https://redirect.github.com/facebook/react/compare/v19.2.5...v19.2.6) #### React Server Components - Type hardening and performance improvements (by [@eps1lon](https://redirect.github.com/eps1lon) and [@unstubbable](https://redirect.github.com/unstubbable)) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - Between 12:00 AM and 03:59 AM, only on Monday (`* 0-3 * * 1`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update release-drafter/release-drafter action to v7.3.0 (#7868) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [release-drafter/release-drafter](https://redirect.github.com/release-drafter/release-drafter) | action | minor | `v7.2.1` → `v7.3.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/357) for more information. --- ### Release Notes <details> <summary>release-drafter/release-drafter (release-drafter/release-drafter)</summary> ### [`v7.3.0`](https://redirect.github.com/release-drafter/release-drafter/releases/tag/v7.3.0) [Compare Source](https://redirect.github.com/release-drafter/release-drafter/compare/v7.2.1...v7.3.0) ### What's Changed #### New - feat: recover recently merged PRs missed by associated PRs lag ([#1604](https://redirect.github.com/release-drafter/release-drafter/issues/1604)) [@jetersen](https://redirect.github.com/jetersen) - feat: switch release discovery to ref comparison and explicit missing-baseline warnings ([#1570](https://redirect.github.com/release-drafter/release-drafter/issues/1570)) [@jetersen](https://redirect.github.com/jetersen) #### Bug Fixes - fix: restore prerelease-identifier on first run when no prior releases exist ([#1602](https://redirect.github.com/release-drafter/release-drafter/issues/1602)) [@jrbeilke](https://redirect.github.com/jrbeilke) - fix: prevent using commitish like refs/pull ([#1598](https://redirect.github.com/release-drafter/release-drafter/issues/1598)) [@&#82…

robert3005 added the changelog/feature A new feature label Apr 28, 2026