Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't generate anything on 5090 #7

Open
aahmyu opened this issue Feb 15, 2025 · 8 comments · May be fixed by #8
Open

Doesn't generate anything on 5090 #7

aahmyu opened this issue Feb 15, 2025 · 8 comments · May be fixed by #8

Comments

@aahmyu
Copy link

aahmyu commented Feb 15, 2025

It seems the new graphic cards don't work on the current installed cuda version

Take a look here
lllyasviel/stable-diffusion-webui-forge#2601

I tried running the update command directly inside the container

pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

Now I get another error about xformers need to be updated or something
I think we need to set env variable for the xformers here
like mentioned here AUTOMATIC1111/stable-diffusion-webui#16818
but now sure how for your image.

jim60105 added a commit that referenced this issue Feb 15, 2025
Update Dockerfile and entrypoint.sh to support new graphic cards with updated CUDA version and xformers environment variable. #7

* **Dockerfile**
  - Update the CUDA version to 128 in the `pip install` command.
  - Update the `torch`, `torchvision` versions to the latest nightly builds and `xformers` to `0.0.29.post2`.

* **entrypoint.sh**
  - Update the `pip install` command to use the latest nightly builds of `torch`, `torchvision`, and `xformers==0.0.29.post2`.
@jim60105 jim60105 linked a pull request Feb 15, 2025 that will close this issue
jim60105 added a commit that referenced this issue Feb 15, 2025


Update Dockerfile and entrypoint.sh to support new graphic cards with updated CUDA version and xformers environment variable. #7

* **Dockerfile**
  - Update the CUDA version to 128 in the `pip install` command.
  - Update the `torch`, `torchvision` versions to the latest nightly builds and `xformers` to `0.0.29.post2`.

* **entrypoint.sh**
  - Update the `pip install` command to use the latest nightly builds of `torch`, `torchvision`, and `xformers==0.0.29.post2`.

Signed-off-by: CHEN, CHUN <[email protected]>
@jim60105 jim60105 linked a pull request Feb 15, 2025 that will close this issue
@jim60105
Copy link
Owner

Please check if these two images work on your 5090

  • ghcr.io/jim60105/stable-diffusion-webui:v1.10.1-128test
  • ghcr.io/jim60105/stable-diffusion-webui:forge-128test

(Note that they are temporary tags for testing and will be removed after #8 merge into master.)

@aahmyu
Copy link
Author

aahmyu commented Feb 15, 2025

So I updated the forge docker compose with the new tag, enabled --reninstall-torch in the commands section the compose file, but now the container fails to start.

Did I mess a step or something?

stable-diffusion-webui-forge  | If you want to use the NVIDIA GeForce RTX 5090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
stable-diffusion-webui-forge  |
stable-diffusion-webui-forge  |   warnings.warn(
stable-diffusion-webui-forge  | WARNING:xformers:WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
stable-diffusion-webui-forge  |     PyTorch 2.6.0+cu124 with CUDA 1204 (you have 2.3.1+cu121)
stable-diffusion-webui-forge  |     Python  3.10.16 (you have 3.10.16)
stable-diffusion-webui-forge  |   Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
stable-diffusion-webui-forge  |   Memory-efficient attention, SwiGLU, sparse and more won't be available.
stable-diffusion-webui-forge  |   Set XFORMERS_MORE_DETAILS=1 for more details
stable-diffusion-webui-forge  | Python 3.10.16 (main, Feb  4 2025, 04:33:05) [GCC 12.2.0]
stable-diffusion-webui-forge  | Version: f2.0.1v1.10.1-1.10.1
stable-diffusion-webui-forge  | Commit hash: <none>
stable-diffusion-webui-forge  | Installing torch and torchvision
stable-diffusion-webui-forge  | Installing requirements
stable-diffusion-webui-forge  | Legacy Preprocessor init warning: Unable to install insightface automatically. Please try run `pip install insightface` manually.
stable-diffusion-webui-forge  | Launching Web UI with arguments: --listen --port 7860 --data-dir /data --gradio-allowed-path . --xformers --api --reinstall-torch --allow-code --enable-insecure-extension-access --opt-sdp-attention --opt-sdp-no-mem-attention --opt-split-attention --opt-channelslast
stable-diffusion-webui-forge  | Total VRAM 32607 MB, total RAM 15716 MB
stable-diffusion-webui-forge  | pytorch version: 2.3.1+cu121
stable-diffusion-webui-forge  | Set vram state to: NORMAL_VRAM
stable-diffusion-webui-forge  | Device: cuda:0 NVIDIA GeForce RTX 5090 : native
stable-diffusion-webui-forge  | Hint: your device supports --cuda-malloc for potential speed improvements.
stable-diffusion-webui-forge  | VAE dtype preferences: [torch.bfloat16, torch.float32] -> torch.bfloat16
stable-diffusion-webui-forge  | CUDA Using Stream: False
stable-diffusion-webui-forge  | Using pytorch cross attention
stable-diffusion-webui-forge  | Using pytorch attention for VAE
stable-diffusion-webui-forge  | Traceback (most recent call last):
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/utils/import_utils.py", line 853, in _get_module
stable-diffusion-webui-forge  |     return importlib.import_module("." + module_name, self.__name__)
stable-diffusion-webui-forge  |   File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
stable-diffusion-webui-forge  |     return _bootstrap._gcd_import(name[level:], package, level)
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/models/autoencoders/__init__.py", line 1, in <module>
stable-diffusion-webui-forge  |     from .autoencoder_asym_kl import AsymmetricAutoencoderKL
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/models/autoencoders/autoencoder_asym_kl.py", line 23, in <module>
stable-diffusion-webui-forge  |     from .vae import DecoderOutput, DiagonalGaussianDistribution, Encoder, MaskConditionDecoder
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/models/autoencoders/vae.py", line 24, in <module>
stable-diffusion-webui-forge  |     from ..attention_processor import SpatialNorm
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/models/attention_processor.py", line 35, in <module>
stable-diffusion-webui-forge  |     import xformers.ops
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/xformers/ops/__init__.py", line 9, in <module>
stable-diffusion-webui-forge  |     from .fmha import (
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/xformers/ops/fmha/__init__.py", line 10, in <module>
stable-diffusion-webui-forge  |     from . import (
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/xformers/ops/fmha/flash3.py", line 10, in <module>
stable-diffusion-webui-forge  |     from torch.utils.flop_counter import (
stable-diffusion-webui-forge  | ImportError: cannot import name '_flash_attention_backward_flop' from 'torch.utils.flop_counter' (/home/1001/.local/lib/python3.10/site-packages/torch/utils/flop_counter.py)
stable-diffusion-webui-forge  |
stable-diffusion-webui-forge  | The above exception was the direct cause of the following exception:
stable-diffusion-webui-forge  |
stable-diffusion-webui-forge  | Traceback (most recent call last):
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/utils/import_utils.py", line 853, in _get_module
stable-diffusion-webui-forge  |     return importlib.import_module("." + module_name, self.__name__)
stable-diffusion-webui-forge  |   File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
stable-diffusion-webui-forge  |     return _bootstrap._gcd_import(name[level:], package, level)
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 44, in <module>
stable-diffusion-webui-forge  |     from ..models import AutoencoderKL
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/utils/import_utils.py", line 843, in __getattr__
stable-diffusion-webui-forge  |     module = self._get_module(self._class_to_module[name])
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/utils/import_utils.py", line 855, in _get_module
stable-diffusion-webui-forge  |     raise RuntimeError(
stable-diffusion-webui-forge  | RuntimeError: Failed to import diffusers.models.autoencoders.autoencoder_kl because of the following error (look up to see its traceback):
stable-diffusion-webui-forge  | cannot import name '_flash_attention_backward_flop' from 'torch.utils.flop_counter' (/home/1001/.local/lib/python3.10/site-packages/torch/utils/flop_counter.py)
stable-diffusion-webui-forge  |
stable-diffusion-webui-forge  | The above exception was the direct cause of the following exception:
stable-diffusion-webui-forge  |
stable-diffusion-webui-forge  | Traceback (most recent call last):
stable-diffusion-webui-forge  |   File "/app/launch.py", line 54, in <module>
stable-diffusion-webui-forge  |     main()
stable-diffusion-webui-forge  |   File "/app/launch.py", line 50, in main
stable-diffusion-webui-forge  |     start()
stable-diffusion-webui-forge  |   File "/app/modules/launch_utils.py", line 546, in start
stable-diffusion-webui-forge  |     import webui
stable-diffusion-webui-forge  |   File "/app/webui.py", line 23, in <module>
stable-diffusion-webui-forge  |     initialize.imports()
stable-diffusion-webui-forge  |   File "/app/modules/initialize.py", line 32, in imports
stable-diffusion-webui-forge  |     from modules import processing, gradio_extensions, ui  # noqa: F401
stable-diffusion-webui-forge  |   File "/app/modules/processing.py", line 19, in <module>
stable-diffusion-webui-forge  |     from modules import devices, prompt_parser, masking, sd_samplers, lowvram, infotext_utils, extra_networks, sd_vae_approx, scripts, sd_samplers_common, sd_unet, errors, rng, profiling
stable-diffusion-webui-forge  |   File "/app/modules/sd_samplers.py", line 5, in <module>
stable-diffusion-webui-forge  |     from modules import sd_samplers_kdiffusion, sd_samplers_timesteps, sd_samplers_lcm, shared, sd_samplers_common, sd_schedulers
stable-diffusion-webui-forge  |   File "/app/modules/sd_samplers_kdiffusion.py", line 5, in <module>
stable-diffusion-webui-forge  |     from modules import sd_samplers_common, sd_samplers_extra, sd_samplers_cfg_denoiser, sd_schedulers, devices
stable-diffusion-webui-forge  |   File "/app/modules/sd_samplers_common.py", line 6, in <module>
stable-diffusion-webui-forge  |     from modules import devices, images, sd_vae_approx, sd_samplers, sd_vae_taesd, shared, sd_models
stable-diffusion-webui-forge  |   File "/app/modules/sd_models.py", line 21, in <module>
stable-diffusion-webui-forge  |     from backend.loader import forge_loader
stable-diffusion-webui-forge  |   File "/app/backend/loader.py", line 9, in <module>
stable-diffusion-webui-forge  |     from diffusers import DiffusionPipeline
stable-diffusion-webui-forge  |   File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/utils/import_utils.py", line 844, in __getattr__
stable-diffusion-webui-forge  |     value = getattr(module, name)
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/utils/import_utils.py", line 843, in __getattr__
stable-diffusion-webui-forge  |     module = self._get_module(self._class_to_module[name])
stable-diffusion-webui-forge  |   File "/home/1001/.local/lib/python3.10/site-packages/diffusers/utils/import_utils.py", line 855, in _get_module
stable-diffusion-webui-forge  |     raise RuntimeError(
stable-diffusion-webui-forge  | RuntimeError: Failed to import diffusers.pipelines.pipeline_utils because of the following error (look up to see its traceback):
stable-diffusion-webui-forge  | Failed to import diffusers.models.autoencoders.autoencoder_kl because of the following error (look up to see its traceback):
stable-diffusion-webui-forge  | cannot import name '_flash_attention_backward_flop' from 'torch.utils.flop_counter' (/home/1001/.local/lib/python3.10/site-packages/torch/utils/flop_counter.py)
stable-diffusion-webui-forge  | WebUI stopped.
stable-diffusion-webui-forge  | Correcting user data permissions... Please wait a second.
stable-diffusion-webui-forge  | Done.
stable-diffusion-webui-forge exited with code 0

@jim60105
Copy link
Owner

Do not reinstall torch and xformers, it is the purpose of this issue.
Reinstalling them at runtime would render this fix pointless.

And if you are using the local volume (like what I did with runtime-deps images), please remove it.

@aahmyu
Copy link
Author

aahmyu commented Feb 15, 2025

I'm pretty much using your default yaml files

services:
  stable-diffusion-webui:
    container_name: stable-diffusion-webui
    image: ghcr.io/jim60105/stable-diffusion-webui:v1.10.1-128test
    user: "1001:0"
    build:
      context: .      
      target: final
      args:
        - UID=1001
        # - SKIP_REQUIREMENTS_INSTALL=true
      cache_from:
        - ghcr.io/jim60105/stable-diffusion-webui:cache
      cache_to:
        - type=inline
    command:
      # https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Command-Line-Arguments-and-Settings
      # https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/cmd_args.py
      #
      # Already configured in the entrypoint and should remain unchanged!!
      # --listen --port 7860 --data-dir /data
      #
      # Recommanded:
      - --xformers
      - --api
      - --allow-code
      - --no-half-vae
      #
      # You may or may not need:
      ## Enable installing/modifying extensions from the UI, please pay attention to security considerations.
      - --enable-insecure-extension-access
      #
      ## Optimizations
      ## https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Optimizations
      # - --lowvram
      # - --medvram
      - --opt-sdp-attention
      - --opt-sdp-no-mem-attention
      - --opt-split-attention
      - --opt-channelslast
      #
      ## Debug logging
      # - --loglevel
      # - DEBUG
      # - --log-startup
    ports:
      - 7860:7860
    environment:
      # Use the volume to get better performance then bind mount.
      # And users should not touch these files in general.
      - SD_WEBUI_CACHE_FILE=/.cache/cache.json
      - SD_WEBUI_CACHE_DIR=/.cache
    tmpfs:
      - /tmp
    volumes:
      - cache:/.cache
      - repositories:/app/repositories
      - ./data:/data
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
              count: all

volumes:
  cache:
  repositories:

I usually use forge, I tried again with the base a1111 file and got the same result.

@jim60105
Copy link
Owner

Please try

docker compose down -v

and then

docker compose up -d --pull always --no-build

to ensure you use the correct container image 🤔

@aahmyu
Copy link
Author

aahmyu commented Feb 16, 2025

Please try

docker compose down -v

and then

docker compose up -d --pull always --no-build

to ensure you use the correct container image 🤔

I got the same error sadly.

I also tried going inside the container and running

pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

since the xformers are updated. But I got dependecies issue

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
xformers 0.0.29.post2 requires torch==2.6.0, but you have torch 2.7.0.dev20250215+cu128 which is incompatible.

EDIT:
Also I got a similar error for open-clip-torch so I tried uninstalling xformers and open-clip-torch and now it generates "fine", but I think you need to look into the dependencies since there seems to be a few conflicts.

I used protobuf 4.25.6 "didn't try latest" and mediapipe 0.10.21 and open-clip-torch 2.30.0
And uninstalled xformers. With these changes pip check is not showing any issues, but sadly it seems there is no xformers package yet for latest cuda.

jim60105 added a commit that referenced this issue Feb 20, 2025


Update Dockerfile and entrypoint.sh to support new graphic cards with updated CUDA version and xformers environment variable. #7

* **Dockerfile**
  - Update the CUDA version to 128 in the `pip install` command.
  - Update the `torch`, `torchvision` versions to the latest nightly builds and `xformers` to `0.0.29.post2`.

* **entrypoint.sh**
  - Update the `pip install` command to use the latest nightly builds of `torch`, `torchvision`, and `xformers==0.0.29.post2`.

Signed-off-by: CHEN, CHUN <[email protected]>
@jim60105
Copy link
Owner

You are correct.

I just noticed in my docker build log that it initially collected torch 2.7.0, but actually used torch 2.6.0 from the default source after collected xformers.

Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/nightly/cu128, https://pypi.nvidia.com
Collecting torch
Using cached https://download.pytorch.org/whl/nightly/cu128/torch-2.7.0.dev20250215%2Bcu128-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (28 kB)
Collecting torchvision
Using cached https://download.pytorch.org/whl/nightly/cu128/torchvision-0.22.0.dev20250215%2Bcu128-cp310-cp310-linux_x86_64.whl.metadata (6.2 kB)
Collecting xformers==0.0.29.post2
Using cached xformers-0.0.29.post2-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (1.0 kB)
Collecting numpy (from xformers==0.0.29.post2)
Using cached numpy-2.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (62 kB)
Collecting torch
Using cached torch-2.6.0-cp310-cp310-manylinux1_x86_64.whl.metadata (28 kB)
Collecting filelock (from torch)
Using cached filelock-3.17.0-py3-none-any.whl.metadata (2.9 kB)
...

I found this issue pointing out that we need to build it from source at the moment, and this is already a bit too far from this project.
For cu128, I think it's more appropriate to remove xformers and wait for the official release.

However considering other users outside the Blackwell series, I believe xformers should not be removed from my default images.
So I will put this issue #7 and PR #8 on hold until xformers officially release updates.

@jim60105
Copy link
Owner

Please check if these two images work on your 5090

  • ghcr.io/jim60105/stable-diffusion-webui:v1.10.1-128test
  • ghcr.io/jim60105/stable-diffusion-webui:forge-128test

(Note that they are temporary tags for testing and will be removed after #8 merge into master.)

I have fixed these two images to use torch 2.7.0

If you still get the same pip dependency error, it is very likely a configuration issue with your container rather than a problem with the docker image itself.

Image

@jim60105 jim60105 pinned this issue Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants