Skip to content

[core] Refactor condition embedders (and others) modules from pipelines#13781

Open
sayakpaul wants to merge 17 commits into
mainfrom
refactor/condition-embedders-submodule
Open

[core] Refactor condition embedders (and others) modules from pipelines#13781
sayakpaul wants to merge 17 commits into
mainfrom
refactor/condition-embedders-submodule

Conversation

@sayakpaul
Copy link
Copy Markdown
Member

@sayakpaul sayakpaul commented May 21, 2026

What does this PR do?

We have a bunch of model-level components under src/diffusers/pipelines module. This PR factors them out of the pipelines module and places them under src/diffusers/models module strategically.

The main changes are in src/pipelines/pipeline_loading_utils.py which basically remaps to the components that were moved to src/diffusersmodels/. Open to other ways of handling it.

For reviewers

Main changes can be summarized as follows:

  • Move an existing model-level module (e.g., audio_tokenizer_ace_step.py) to src/diffusers/models and deprecate gracefully.
  • I have used my best judgment to determine which model gets to either of autoencoders and condition_embedders. When I wasn't sure, I simply placed them under others. No strong opinions here.
  • scripts/smoke_test_relocated_pretrained.py is for transient testing until the PR is merged. Will delete it after. I have run the tests and models and pipelines load as expected.
  • There is a tests/models/test_relocated_class_loading.py which will check if deprecation warnings are raised and will also help to remove those deprecation paths once the cycle is over.
  • Similarly, there is a tests/pipelines/test_relocated_class_loading.py. It ensures that no deprecation warnings for the refactored modules are raised when loading the affected pipelines.

@github-actions github-actions Bot added size/L PR with diff > 200 LOC models tests utils pipelines and removed size/L PR with diff > 200 LOC labels May 21, 2026
@github-actions github-actions Bot added the size/L PR with diff > 200 LOC label May 21, 2026
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

# See the License for the specific language governing permissions and
# limitations under the License.

import torch
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open to suggestions on where to place these other than others.

@@ -0,0 +1,476 @@
# Copyright 2025 The ACE-Step Team and The HuggingFace Team. All rights reserved.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going by the implementation of the module, it encodes into a tokenized representation (not the existing sense of tokenization done in language models). So, decided keep them under autoencoders.

@@ -0,0 +1,192 @@
# Copyright 2025 Lightricks and The HuggingFace Team. All rights reserved.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open to moving them elsewhere. Since upsampling could be interpreted as decoding to a higher resolution, I thought of keeping them under autoencoders.

@sayakpaul sayakpaul marked this pull request as ready for review May 22, 2026 06:24
@sayakpaul sayakpaul requested a review from yiyixuxu May 22, 2026 07:55
@sayakpaul sayakpaul requested review from dg845 and yiyixuxu and removed request for yiyixuxu May 22, 2026 07:55

# Reroutes pretrained loads past pipeline-local deprecation shims onto the canonical
# top-level diffusers export.
_RELOCATED_PIPELINE_CLASSES: dict[tuple[str, str], tuple[str, str]] = {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so that we don't hit deprecation warnings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants