Skip to content

Point users at AOBaseConfig replacement when passing string quant_type to TorchAoConfig#13784

Open
adityasingh2400 wants to merge 1 commit into
huggingface:mainfrom
adityasingh2400:fix-torchao-string-config-migration-error
Open

Point users at AOBaseConfig replacement when passing string quant_type to TorchAoConfig#13784
adityasingh2400 wants to merge 1 commit into
huggingface:mainfrom
adityasingh2400:fix-torchao-string-config-migration-error

Conversation

@adityasingh2400
Copy link
Copy Markdown

What does this PR do?

Fixes #13286. Also covers the diffusers-side root cause behind #13266.

PR #13291 removed the string-based quant_type path from TorchAoConfig in favour of an AOBaseConfig subclass instance from torchao.quantization (e.g. Int8WeightOnlyConfig()). That cleanup is good, but the remaining error for a string input,

TypeError: quant_type must be an AOBaseConfig instance, got str

doesn't name a replacement. Users running existing code, especially against torchao >= 0.16 where the legacy lowercase factories (float8_weight_only, int8_weight_only, float8_dynamic_activation_float8_weight, ...) were removed upstream, hit the rename without a concrete migration path.

This PR maps the common legacy strings to their Config class replacements and surfaces that mapping in the TypeError, so the error itself tells the user which import to add and how to instantiate it. Unknown strings still raise but point at the torchao quantization docs. The non-string branch is unchanged, and the public surface (TorchAoConfig(AOBaseConfig)) is untouched.

Before this PR

>>> from diffusers import TorchAoConfig
>>> TorchAoConfig("float8dq_e4m3_row")
TypeError: quant_type must be an AOBaseConfig instance, got str

After this PR

>>> from diffusers import TorchAoConfig
>>> TorchAoConfig("float8dq_e4m3_row")
TypeError: TorchAoConfig no longer accepts string quant_type values (got 'float8dq_e4m3_row');
pass an AOBaseConfig instance from torchao.quantization instead. For 'float8dq_e4m3_row', use
`from torchao.quantization import Float8DynamicActivationFloat8WeightConfig;
 TorchAoConfig(Float8DynamicActivationFloat8WeightConfig())`. See
https://huggingface.co/docs/diffusers/main/en/quantization/torchao for the full list of
supported AOBaseConfig classes.

The suggestion does not auto-instantiate the Config for the user: the new API exposes options (granularity, dtype, version, ...) that the legacy strings hard-coded (e.g. float8dq_e4m3_row set granularity=(PerRow(), PerRow())), and silently picking a default would be surprising. Naming the class is enough to unblock migration.

Tests

Added test_string_quant_type_error_includes_migration_hint to tests/quantization/torchao/test_torchao.py covering both reproductions from the linked issues (float8dq_e4m3_row, int8wo, float8_weight_only, ...) and an unknown-string fallback. The pre-existing test_post_init_check still passes unchanged.

$ pytest tests/quantization/torchao/test_torchao.py::TorchAoConfigTest -v
tests/quantization/torchao/test_torchao.py::TorchAoConfigTest::test_post_init_check PASSED
tests/quantization/torchao/test_torchao.py::TorchAoConfigTest::test_repr PASSED
tests/quantization/torchao/test_torchao.py::TorchAoConfigTest::test_string_quant_type_error_includes_migration_hint PASSED
tests/quantization/torchao/test_torchao.py::TorchAoConfigTest::test_to_dict PASSED
4 passed

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case). Improves an error message.
  • Did you read the contributor guideline?
  • Did you read our philosophy doc (important for complex PRs)?
  • Was this discussed/approved via a GitHub issue or the forum? Issues torchao >= 0.16.0 quantization not supported #13286 and Torchao SD3 int8wo #13266.
  • Did you make sure to update the documentation with your changes? Error message links to the existing torchao quantization doc page.
  • Did you write any new necessary tests?

Who can review?

@sayakpaul @DN6 - tagging the reviewers from PR #13291 (the original string-removal PR) since this is a follow-up on the same surface.

…e to TorchAoConfig

PR huggingface#13291 removed the string-based quant_type path from TorchAoConfig in favour of
passing an AOBaseConfig subclass instance from torchao.quantization directly. The
remaining TypeError ("quant_type must be an AOBaseConfig instance, got str") does
not name a replacement, so users running existing code, especially against
torchao >= 0.16 where the legacy lowercase factories (float8_weight_only,
int8_weight_only, float8_dynamic_activation_float8_weight, ...) were removed
upstream, hit the rename without a concrete migration path.

Map the common legacy strings to their Config-class replacements and surface that
mapping in the TypeError so the error itself tells the user which import to add
and how to instantiate it. Strings outside the mapping still raise but point at
the torchao quantization docs. The non-string branch is unchanged.

Fixes huggingface#13286
Fixes huggingface#13266
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

torchao >= 0.16.0 quantization not supported

1 participant