Skip to content

[refactor](be) Remove redundant spill state#64083

Open
mrhhsg wants to merge 1 commit into
apache:masterfrom
mrhhsg:cleanup/remove-redundant-spill-code
Open

[refactor](be) Remove redundant spill state#64083
mrhhsg wants to merge 1 commit into
apache:masterfrom
mrhhsg:cleanup/remove-redundant-spill-code

Conversation

@mrhhsg
Copy link
Copy Markdown
Member

@mrhhsg mrhhsg commented Jun 3, 2026

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary:

Remove redundant spill-related state that is no longer used by the execution path. This includes unused spill task helpers, obsolete spill file counters, and stale opened-state tracking. The remaining spill file size accounting that is still used by readers/writers is kept intact.

Release note

None

Check List (For Author)

  • Test

    • Regression test
      • ./run-regression-test.sh --conf /tmp/doris-spill-regression-oldport.groovy --run -d spill_p0 -s aggregate_spill (passed)
    • Unit Test
      • ./run-be-ut.sh --run --filter='*Spill*:*MultiCastDataStreamer*' -j 64 (passed, 79 tests)
    • Manual test (add detailed scripts or steps below)
      • ./build.sh --be --fe (passed as part of local regression preparation)
      • Targeted BE object compilation for touched spill/operator files (passed)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Several spill helpers and profile counters no longer carry useful state after synchronous spill execution and current spill-file accounting. Remove the dead run_spill_task wrapper, unused writer-side cumulative counters, unused spill file current count and total size profile fields, and the stale SpillSortLocalState::_opened guard. Keep SpillFileReader::seek and SpillFile-owned byte accounting intact. Update test helpers to stop registering the removed current-count counter.

### Release note

None

### Check List (For Author)

- Test: Manual test
    - ./build-support/clang-format.sh
    - ./build-support/check-format.sh
    - git diff --check
    - DORIS_HOME=$PWD ninja -C be/ut_build_ASAN affected source/test object targets
- Behavior changed: No
- Does this need documentation: No
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@mrhhsg
Copy link
Copy Markdown
Member Author

mrhhsg commented Jun 3, 2026

/review

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the full PR diff and did not find blocking issues.

Critical checkpoint conclusions:

  • Goal and proof: The PR removes redundant spill state, obsolete counters, and a dead synchronous spill wrapper. The changed call sites remain equivalent to the old helper because it only invoked the callback and returned its Status. Existing targeted spill/operator tests are listed by the author; I did not rerun the full test suite in this review.
  • Scope and clarity: The change is small and focused on unused spill/operator state.
  • Concurrency: The direct spill calls preserve the existing synchronous execution and lock-release behavior. No new dependency, locking, or atomic lifecycle paths were introduced.
  • Lifecycle: The removed SpillSortLocalState _opened guard was stale and was never set true in the changed code path; removing it does not alter lifecycle semantics.
  • Configuration and compatibility: No configuration, storage format, thrift protocol, or mixed-version compatibility concern was introduced.
  • Parallel paths: Remaining references to the removed helper/counters were checked; no stale uses were found.
  • Error handling: Status/exception propagation remains equivalent at the touched call sites.
  • Memory and spill accounting: SpillFile-owned byte accounting and profile counters still used by SpillFileWriter remain in place; the removed cumulative writer fields were not used by behavior.
  • Tests: Test helpers were updated to stop registering the removed counter. No new test result correctness issue was found.
  • Observability: Removing the unused current-count counter reduces obsolete profile state; required write byte/file-count counters remain.
  • User focus: No additional user-provided review focus was specified.

Overall opinion: The PR quality looks acceptable for this cleanup, with no critical blocking issue identified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants