Skip to content

Remove sqllogictest fork swap from regenerate_sqlite_files.sh#21578

Open
RafaelHerrero wants to merge 1 commit intoapache:mainfrom
RafaelHerrero:fix/remove-sqllogictest-fork-from-regenerate-script
Open

Remove sqllogictest fork swap from regenerate_sqlite_files.sh#21578
RafaelHerrero wants to merge 1 commit intoapache:mainfrom
RafaelHerrero:fix/remove-sqllogictest-fork-from-regenerate-script

Conversation

@RafaelHerrero
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Related to #6543 and #21260

Rationale for this change

The regenerate_sqlite_files.sh script swaps in a forked version of sqllogictest-rs (Omega359/sqllogictest-rs v0.27.2) before running test completion. This fork added valuewise result mode, valuesort, and custom normalizer support.

These features were upstreamed to risinglightdb/sqllogictest-rs in v0.24.0 (PR #237). DataFusion already uses upstream v0.29.1 which includes everything, so the fork swap is no longer needed.

What changes are included in this PR?

  • Remove the Cargo.toml dependency swap to the Omega359 fork (line 167)
  • Remove the bin/sqllogictests.rs file swap with the custom regenerate version (line 172)
  • Remove the cleanup steps that reverted those swaps (lines 203-204)
  • Update the script description to reflect the simplified flow

Are these changes tested?

The script has valid bash syntax. The change is purely removing dead code — the fork features are already available in the upstream version DataFusion uses.

Are there any user-facing changes?

No.

The Omega359 fork features (valuewise result mode, valuesort, custom
normalizer) were upstreamed to risinglightdb/sqllogictest-rs in v0.24.0
(PR risinglightdb/sqllogictest-rs#237). DataFusion already uses upstream
v0.29.1 which includes everything, so the fork swap is no longer needed.
@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Apr 13, 2026
@alamb alamb added the development-process Related to development process of DataFusion label Apr 15, 2026
Copy link
Copy Markdown
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this @RafaelHerrero . I tested running this script locally like

PG_URI=postgresql://postgres@localhost:5432/postgres nice ./datafusion/sqllogictest/regenerate_sqlite_files.sh

And it made a bunch of errors like this:

External error: task 948 panicked with message "called `Result::unwrap()` on an `Err` value: ParseError { kind: InvalidLine(\"onlyif mysql # use DIV operator for integer division\"), loc: Location { file: \"../../datafusion-testing/data/sqlite/random/select/slt_good_99.slt\", line: 60, upper: None } }"
Error: Execution("428 failures")
error: test failed, to rerun pass `-p datafusion-sqllogictest --test sqllogictests`

Caused by:
  process didn't exit successfully: `/Users/andrewlamb/Software/datafusion2/target/release-nonlto/deps/sqllogictests-a999e2d90c5771c0 --include-sqlite --postgres-runner --complete` (exit status: 1)
Completion of sqlite test files failed!
Cleaning up source code changes and temporary files and directories

Do you have any idea what is going on?

maybe @Omega359 has some ideas as he was the original author of this script

@Omega359
Copy link
Copy Markdown
Contributor

Omega359 commented Apr 16, 2026

I thought I replied to this already but apparently hitting the 'comment' button was too much for me the other day :/

I think the fork does more than what was upstreamed though I'd have to diff the code at this point to know what exactly I changed. Ideally, yes I'd like to not have a fork for this but I haven't had the time nor desire to see what that would involve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants