Skip to content

Enable MPS backend for bitsandbytes quantization #13361

@LucasSte

Description

@LucasSte

Is your feature request related to a problem? Please describe.

Bitsandbytes now has basic support for the Apple MPS backend, as I can tell by bitsandbytes-foundation/bitsandbytes#1818 and
bitsandbytes-foundation/bitsandbytes#1875.

The issue is that diffusers does not allow me to use the quantization on Apple hardware, because of the error No GPU found. A GPU is needed for quantization. from here

if not (torch.cuda.is_available() or torch.xpu.is_available()):
raise RuntimeError("No GPU found. A GPU is needed for quantization.")
.

Describe the solution you'd like.

I modified the above lines to

if not (torch.cuda.is_available() or torch.xpu.is_available() or torch.mps.is_available()):
    raise RuntimeError("No GPU found. A GPU is needed for quantization.")

and tested the change with the quantized version of FLUX.2-dev as described in https://github.com/black-forest-labs/flux2/blob/main/docs/flux2_dev_hf.md#4-bit-transformer-and-4-bit-text-encoder-20g-of-vram, and all worked fine.

I am wondering if that solution could be adopted in the repository. I can open a PR if that is the case.

Describe alternatives you've considered.

None.

Additional context.
Add any other context or screenshots about the feature request here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions