[fix](s3) preserve explicit path style for object storage#64034
Open
sollhui wants to merge 1 commit into
Open
Conversation
sollhui
requested review from
CalvinKirs,
gavinchou and
morningman
as code owners
Contributor
Author
|
run buildall |
sollhui
changed the title
sollhui
force-pushed
the
fix_preserve_use_path_style
branch
from
bd3c424 to
fdf08ce
Compare
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage `` 🎉 |
Contributor
TPC-H: Total hot run time: 29313 ms |
Contributor
FE UT Coverage ReportIncrement line coverage `` 🎉 |
Contributor
TPC-DS: Total hot run time: 169466 ms |
Contributor
Author
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Review result: request changes.
I found one correctness issue in the path-style preservation logic: the change preserves only the canonical use_path_style key, but still overrides the supported s3.path-style-access alias by injecting use_path_style=false. This affects COS, OBS, and OSS provider/translation paths.
Critical checkpoint conclusions:
- Goal/test: The PR aims to preserve explicit path-style configuration for OSS/COS/OBS. It covers canonical use_path_style=true, but not the existing s3.path-style-access alias, so the goal is incomplete.
- Scope/focus: The code change is small and focused.
- Concurrency/lifecycle/config/persistence/transactions/data writes: Not applicable to this filesystem property mapping change.
- Parallel paths: The same provider/toS3Props pattern exists for COS, OBS, and OSS and needs the same fix in all affected paths.
- Test coverage: Tests cover canonical use_path_style but miss the alias regression; please add alias coverage for provider-created storage and/or toS3Props.
- Observability/performance: No additional observability or performance concerns found for this change.
User focus: No additional user-provided review focus was specified.
sollhui
force-pushed
the
fix_preserve_use_path_style
branch
from
fdf08ce to
ea84c38
Compare
Contributor
Author
|
run buildall |
Contributor
Author
|
/review |
Contributor
There was a problem hiding this comment.
No blocking issues found in the current PR diff.
Critical checkpoint conclusions:
- Goal/test coverage: The PR preserves explicit path-style settings for COS/OBS/OSS provider creation and toS3Props conversion, including the s3.path-style-access alias. Added unit tests cover direct use_path_style, alias handling, and provider-created S3ObjStorage behavior for all three providers.
- Scope/focus: The implementation is small and focused on avoiding insertion of the default use_path_style=false when either supported path-style key is already present.
- Concurrency/lifecycle: No new shared mutable state, lifecycle management, or lock behavior is introduced.
- Configuration/compatibility: No new config item or storage/protocol format is introduced. Existing default behavior remains false when neither path-style key is provided, while explicit caller values are preserved.
- Parallel paths: Checked COS, OBS, and OSS provider paths and their toS3Props helpers. The previously raised alias concern is addressed in this head and was not duplicated.
- Security/threat-model: Object storage endpoints and credentials remain admin-provided external connection properties under the repository threat model; this change does not add a new trust boundary or broaden access.
- Observability/performance: No additional observability is needed for this config-preservation path; performance impact is negligible.
User focus: No additional user-provided review focus was specified.
Testing: Not run locally because thirdparty/installed/bin/protoc is missing, and FE instructions require that prerequisite before FE builds/tests.
Contributor
TPC-H: Total hot run time: 29609 ms |
Contributor
TPC-DS: Total hot run time: 170013 ms |
Contributor
FE Regression Coverage ReportIncrement line coverage |
2 similar comments
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Related PR: #62023
OSS/COS/OBS filesystem providers translated their storage properties to S3-compatible properties, but always overwrote use_path_style with false. As a result, a user-provided use_path_style=true was lost before S3FileSystem and S3ObjStorage consumed the configuration. This made path-style behavior impossible to preserve through the provider path and could hide invalid path-style URL cases.
This change keeps the existing OSS/COS/OBS default of use_path_style=false when the option is not provided, but preserves an explicit user value by using putIfAbsent during provider creation and property translation. Unit tests cover both the translation helper and the final provider-created S3ObjStorage configuration.