[refactor](be) Remove redundant spill state#64083
Open
mrhhsg wants to merge 1 commit into
Open
Conversation
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Several spill helpers and profile counters no longer carry useful state after synchronous spill execution and current spill-file accounting. Remove the dead run_spill_task wrapper, unused writer-side cumulative counters, unused spill file current count and total size profile fields, and the stale SpillSortLocalState::_opened guard. Keep SpillFileReader::seek and SpillFile-owned byte accounting intact. Update test helpers to stop registering the removed current-count counter.
### Release note
None
### Check List (For Author)
- Test: Manual test
- ./build-support/clang-format.sh
- ./build-support/check-format.sh
- git diff --check
- DORIS_HOME=$PWD ninja -C be/ut_build_ASAN affected source/test object targets
- Behavior changed: No
- Does this need documentation: No
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Member
Author
|
/review |
Contributor
There was a problem hiding this comment.
I reviewed the full PR diff and did not find blocking issues.
Critical checkpoint conclusions:
- Goal and proof: The PR removes redundant spill state, obsolete counters, and a dead synchronous spill wrapper. The changed call sites remain equivalent to the old helper because it only invoked the callback and returned its Status. Existing targeted spill/operator tests are listed by the author; I did not rerun the full test suite in this review.
- Scope and clarity: The change is small and focused on unused spill/operator state.
- Concurrency: The direct spill calls preserve the existing synchronous execution and lock-release behavior. No new dependency, locking, or atomic lifecycle paths were introduced.
- Lifecycle: The removed SpillSortLocalState _opened guard was stale and was never set true in the changed code path; removing it does not alter lifecycle semantics.
- Configuration and compatibility: No configuration, storage format, thrift protocol, or mixed-version compatibility concern was introduced.
- Parallel paths: Remaining references to the removed helper/counters were checked; no stale uses were found.
- Error handling: Status/exception propagation remains equivalent at the touched call sites.
- Memory and spill accounting: SpillFile-owned byte accounting and profile counters still used by SpillFileWriter remain in place; the removed cumulative writer fields were not used by behavior.
- Tests: Test helpers were updated to stop registering the removed counter. No new test result correctness issue was found.
- Observability: Removing the unused current-count counter reduces obsolete profile state; required write byte/file-count counters remain.
- User focus: No additional user-provided review focus was specified.
Overall opinion: The PR quality looks acceptable for this cleanup, with no critical blocking issue identified.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
Remove redundant spill-related state that is no longer used by the execution path. This includes unused spill task helpers, obsolete spill file counters, and stale opened-state tracking. The remaining spill file size accounting that is still used by readers/writers is kept intact.
Release note
None
Check List (For Author)
Test
./run-regression-test.sh --conf /tmp/doris-spill-regression-oldport.groovy --run -d spill_p0 -s aggregate_spill(passed)./run-be-ut.sh --run --filter='*Spill*:*MultiCastDataStreamer*' -j 64(passed, 79 tests)./build.sh --be --fe(passed as part of local regression preparation)Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)