Point users at AOBaseConfig replacement when passing string quant_type to TorchAoConfig#13784
Open
adityasingh2400 wants to merge 1 commit into
Open
Conversation
…e to TorchAoConfig PR huggingface#13291 removed the string-based quant_type path from TorchAoConfig in favour of passing an AOBaseConfig subclass instance from torchao.quantization directly. The remaining TypeError ("quant_type must be an AOBaseConfig instance, got str") does not name a replacement, so users running existing code, especially against torchao >= 0.16 where the legacy lowercase factories (float8_weight_only, int8_weight_only, float8_dynamic_activation_float8_weight, ...) were removed upstream, hit the rename without a concrete migration path. Map the common legacy strings to their Config-class replacements and surface that mapping in the TypeError so the error itself tells the user which import to add and how to instantiate it. Strings outside the mapping still raise but point at the torchao quantization docs. The non-string branch is unchanged. Fixes huggingface#13286 Fixes huggingface#13266
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #13286. Also covers the diffusers-side root cause behind #13266.
PR #13291 removed the string-based
quant_typepath fromTorchAoConfigin favour of anAOBaseConfigsubclass instance fromtorchao.quantization(e.g.Int8WeightOnlyConfig()). That cleanup is good, but the remaining error for a string input,doesn't name a replacement. Users running existing code, especially against
torchao >= 0.16where the legacy lowercase factories (float8_weight_only,int8_weight_only,float8_dynamic_activation_float8_weight, ...) were removed upstream, hit the rename without a concrete migration path.This PR maps the common legacy strings to their
Configclass replacements and surfaces that mapping in theTypeError, so the error itself tells the user which import to add and how to instantiate it. Unknown strings still raise but point at the torchao quantization docs. The non-string branch is unchanged, and the public surface (TorchAoConfig(AOBaseConfig)) is untouched.Before this PR
After this PR
The suggestion does not auto-instantiate the Config for the user: the new API exposes options (granularity, dtype, version, ...) that the legacy strings hard-coded (e.g.
float8dq_e4m3_rowsetgranularity=(PerRow(), PerRow())), and silently picking a default would be surprising. Naming the class is enough to unblock migration.Tests
Added
test_string_quant_type_error_includes_migration_hinttotests/quantization/torchao/test_torchao.pycovering both reproductions from the linked issues (float8dq_e4m3_row,int8wo,float8_weight_only, ...) and an unknown-string fallback. The pre-existingtest_post_init_checkstill passes unchanged.Before submitting
Who can review?
@sayakpaul @DN6 - tagging the reviewers from PR #13291 (the original string-removal PR) since this is a follow-up on the same surface.