Skip to content

Do not read metadata.json on initial host with remote initiator#1760

Open
ianton-ru wants to merge 1 commit intoantalya-26.3from
feature/antalya-26.3/remote_initiator_optimization
Open

Do not read metadata.json on initial host with remote initiator#1760
ianton-ru wants to merge 1 commit intoantalya-26.3from
feature/antalya-26.3/remote_initiator_optimization

Conversation

@ianton-ru
Copy link
Copy Markdown

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Do not read metadata.json on initial host with remote initiator

Documentation entry for user-facing changes

With remote initiator for Iceberg table source node reads metadata.json but does not use.
This PR turns off this reading.

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Performance tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All with Aarch64
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

@ianton-ru ianton-ru added antalya port-antalya PRs to be ported to all new Antalya releases antalya-26.3 labels May 8, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 8, 2026

Workflow [PR], commit [263b9e0]

@ianton-ru
Copy link
Copy Markdown
Author

@codex review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 263b9e0dfb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +549 to +550
if (query_context->getSettingsRef()[Setting::object_storage_remote_initiator])
return;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Refresh metadata even when using remote initiator

Returning early when object_storage_remote_initiator is enabled skips all dynamic metadata refresh on the initiating node, so its in-memory schema/state can become stale after Iceberg/DeltaLake snapshot changes. The initiator still performs query analysis and builds storage_snapshot locally (e.g., before rewriting to remote(...)), so a query that references newly added columns can fail at analysis with unknown-column errors even though the remote node has up-to-date metadata. Previously this path always called configuration->update, so this is a regression for sessions that keep object_storage_remote_initiator=1 across evolving lakehouse tables.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

antalya antalya-26.3 port-antalya PRs to be ported to all new Antalya releases

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant