[fix](mtmv) Refresh MTMV after excluded trigger tables change#64041
Open
seawinde wants to merge 1 commit into
Open
[fix](mtmv) Refresh MTMV after excluded trigger tables change#64041seawinde wants to merge 1 commit into
seawinde wants to merge 1 commit into
Conversation
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary: Changing the effective excluded_trigger_tables set changes which base tables participate in MTMV freshness decisions. Reusing the old refreshSnapshot can make a later AUTO refresh skip work under the new exclude rules, leaving MV data based on the old snapshot.
Root cause: In MTMVTask.calculateNeedRefreshPartitions(), AUTO refresh calls MTMVPartitionUtil.isMTMVSync() before verifying whether the refresh baseline is still complete. After ALTER excluded_trigger_tables, the new exclude rules can skip changed base tables and incorrectly mark the MV as fresh.
This change resets the refresh snapshot and bumps the MTMV schema change version when the effective excluded_trigger_tables set changes, without setting MTMVStatus.state to SCHEMA_CHANGE. AUTO refresh now treats an incomplete refresh snapshot as requiring a complete refresh, while manual specified-partition refresh keeps the existing behavior.
### Release note
Changing excluded_trigger_tables on an MTMV now invalidates the refresh baseline so the next AUTO refresh rebuilds it with a complete refresh.
### Check List (For Author)
- Test: Unit Test
- ./run-fe-ut.sh --run org.apache.doris.mtmv.MTMVTest,org.apache.doris.mtmv.MTMVTaskTest
- Regression test not run because no local FE/BE cluster is available to generate the updated .out file.
- Behavior changed: Yes. ALTER excluded_trigger_tables now invalidates the refresh baseline, and AUTO refresh performs a complete refresh when the baseline is incomplete.
- Does this need documentation: No
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
|
/review |
Contributor
|
run buildall |
morrySnow
approved these changes
Jun 3, 2026
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Review completed for the actual GitHub PR diff. I did not find a concrete blocking issue.
Critical checkpoint conclusions:
- Goal/test coverage: the change addresses AUTO refresh correctness after excluded_trigger_tables changes by invalidating the refresh baseline and forcing a full refresh when the snapshot baseline is incomplete. Unit and regression coverage were added for the changed scenarios.
- Scope/focus: the actual PR is small and focused on MTMV excluded trigger table refresh behavior.
- Concurrency/locking: MTMV property and snapshot mutations are performed under the MV write lock; the added read path uses the existing MV read-lock pattern.
- Lifecycle/persistence: ALTER_PROPERTY replay recomputes the same property-change transition, and the mutated MTMV fields are persisted through the table image/state. No master/follower mismatch was identified.
- Configuration/incompatibility: no new config item or incompatible storage/protocol change was introduced.
- Parallel paths/special checks: manual refresh behavior remains prioritized before the incomplete-baseline AUTO check; COMPLETE refresh behavior remains unchanged.
- Test results: I did not run tests in this review runner; I reviewed the added unit/regression coverage for determinism and relevance.
- Observability/performance: no new observability requirement or material performance issue was identified.
- Transaction/data correctness: the baseline invalidation prevents stale AUTO refresh decisions after excluded trigger rules change.
User focus: no additional user-provided review focus was supplied.
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
TPC-H: Total hot run time: 29234 ms |
Contributor
TPC-DS: Total hot run time: 169870 ms |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
Changing the effective
excluded_trigger_tablesset changes which base tables participate in MTMV freshness decisions. Reusing the oldrefreshSnapshotcan make a later AUTO refresh skip work under the new exclude rules, leaving MV data based on the old snapshot.Root cause: In
MTMVTask.calculateNeedRefreshPartitions(), AUTO refresh callsMTMVPartitionUtil.isMTMVSync()before verifying whether the refresh baseline is still complete. AfterALTER MATERIALIZED VIEW ... SET ("excluded_trigger_tables"=...), the new exclude rules can skip changed base tables and incorrectly mark the MV as fresh.MTMV.javaexcluded_trigger_tablesset changes, bumpschemaChangeVersion, and clearrefreshSnapshotwithout settingMTMVStatus.statetoSCHEMA_CHANGE.MTMVTask.javaisMTMVSync(). Manual specified-partition refresh keeps existing behavior.MTMVTest.javaMTMVTaskTest.javatest_excluded_trigger_table_mtmv.groovyDesign rationale:
This uses the existing
refreshSnapshotcompleteness as the invalid-baseline signal instead of adding a new persisted flag. That keeps the change scoped to the ALTER property path and AUTO refresh decision path. Query rewrite behavior is unchanged.graph TD A[ALTER excluded_trigger_tables] --> B[Compare effective table-name set] B -->|changed| C[Clear refreshSnapshot] C --> D[Next AUTO refresh] D --> E{refreshSnapshot complete?} E -->|No| F[Refresh all MV partitions] E -->|Yes| G[Run isMTMVSync with current exclude rules]Release note
Changing
excluded_trigger_tableson an MTMV now invalidates the refresh baseline so the next AUTO refresh rebuilds it with a complete refresh.Check List (For Author)
./run-fe-ut.sh --run org.apache.doris.mtmv.MTMVTest,org.apache.doris.mtmv.MTMVTaskTestRegression test was not run locally because no FE/BE cluster is available to generate the updated
.outfile.Behavior changed:
ALTER MATERIALIZED VIEW ... SET ("excluded_trigger_tables"=...)now invalidates the refresh baseline when the effective excluded table set changes. The next AUTO refresh performs a complete refresh if the baseline is incomplete.Does this need documentation?
Check List (For Reviewer who merge this PR)