Can't load tokenizer

[RealArtGames]

BAT FILE:

set MODEL_NAME=stabilityaistable-diffusion-3-medium
set INSTANCE_DIR=MIX
set OUTPUT_DIR=trained-sd3

accelerate launch train_dreambooth_sd3.py --pretrained_model_name_or_path=%MODEL_NAME% --instance_data_dir=%INSTANCE_DIR% --output_dir=%OUTPUT_DIR% --mixed_precision="no" --instance_prompt="a photo of a naked girl" --resolution=1024 --train_batch_size=1 --gradient_accumulation_steps=1 --learning_rate=1e-4 --report_to="wandb" --lr_scheduler="constant" --lr_warmup_steps=0 --max_train_steps=760 --validation_epochs=1 --seed="0" --use_8bit_adam --caption_column "text"
set /p DUMMY=Hit ENTER to continue..

File:
stabilityaistable-diffusion-3-medium:
-sd3_medium.safetensors
-sd3_medium_incl_clips.safetensors
-stableDiffusion3SD3_textEncoderClipG.safetensors
-stableDiffusion3SD3_textEncoderClipL.safetensors
-stableDiffusion3SD3_textEncoderT5E4m3fn.safetensors
-text_encoders:
--stableDiffusion3SD3_textEncoderClipG.safetensors
--stableDiffusion3SD3_textEncoderClipL.safetensors

Log:

G:\AI\diffusers-0.29.1\diffusers-0.29.1\examples\dreambooth>set MODEL_NAME=stabilityaistable-diffusion-3-medium

G:\AI\diffusers-0.29.1\diffusers-0.29.1\examples\dreambooth>set INSTANCE_DIR=MIX

G:\AI\diffusers-0.29.1\diffusers-0.29.1\examples\dreambooth>set OUTPUT_DIR=trained-sd3

G:\AI\diffusers-0.29.1\diffusers-0.29.1\examples\dreambooth>accelerate launch train_dreambooth_sd3.py --pretrained_model_name_or_path=stabilityaistable-diffusion-3-medium --instance_data_dir=MIX --output_dir=trained-sd3 --mixed_precision="no" --instance_prompt="a photo of a naked girl" --resolution=1024 --train_batch_size=1 --gradient_accumulation_steps=1 --learning_rate=1e-4 --report_to="wandb" --lr_scheduler="constant" --lr_warmup_steps=0 --max_train_steps=760 --validation_epochs=1 --seed="0" --use_8bit_adam --caption_column "text"
08/07/2024 22:40:54 - INFO - main - Distributed environment: DistributedType.NO
Num processes: 1
Process index: 0
Local process index: 0
Device: cuda

Mixed precision type: no

Traceback (most recent call last):
File "G:\AI\diffusers-0.29.1\diffusers-0.29.1\examples\dreambooth\train_dreambooth_sd3.py", line 1760, in
main(args)
File "G:\AI\diffusers-0.29.1\diffusers-0.29.1\examples\dreambooth\train_dreambooth_sd3.py", line 1048, in main
tokenizer_one = CLIPTokenizer.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ZeroTwo\AppData\Roaming\Python\Python312\site-packages\transformers\tokenization_utils_base.py", line 2243, in from_pretrained
raise EnvironmentError(
OSError: Can't load tokenizer for 'stabilityaistable-diffusion-3-medium'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'stabilityaistable-diffusion-3-medium' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer.
Traceback (most recent call last):
File "C:\Users\ZeroTwo\miniconda3\Scripts\accelerate-script.py", line 9, in
sys.exit(main())
^^^^^^
File "C:\Users\ZeroTwo\miniconda3\Lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
args.func(args)
File "C:\Users\ZeroTwo\miniconda3\Lib\site-packages\accelerate\commands\launch.py", line 1106, in launch_command
simple_launcher(args)
File "C:\Users\ZeroTwo\miniconda3\Lib\site-packages\accelerate\commands\launch.py", line 704, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\Users\ZeroTwo\miniconda3\python.exe', 'train_dreambooth_sd3.py', '--pretrained_model_name_or_path=stabilityaistable-diffusion-3-medium', '--instance_data_dir=MIX', '--output_dir=trained-sd3', '--mixed_precision=no', '--instance_prompt=a photo of a naked girl', '--resolution=1024', '--train_batch_size=1', '--gradient_accumulation_steps=1', '--learning_rate=1e-4', '--report_to=wandb', '--lr_scheduler=constant', '--lr_warmup_steps=0', '--max_train_steps=760', '--validation_epochs=1', '--seed=0', '--use_8bit_adam', '--caption_column', 'text']' returned non-zero exit status 1.

G:\AI\diffusers-0.29.1\diffusers-0.29.1\examples\dreambooth>set /p DUMMY=Hit ENTER to continue..
Hit ENTER to continue..

I try to search for solution but i can't find any.

[RealArtGames]

I download the tokenizer from here: https://huggingface.co/tiovikram/trained-sd3
And its work but i have a new error code:

You set add_prefix_space. The tokenizer needs to be converted from the slow tokenizers
Traceback (most recent call last):
File "G:\AI\diffusers-0.29.1\diffusers-0.29.1\examples\dreambooth\train_dreambooth_sd3.py", line 1760, in
main(args)
File "G:\AI\diffusers-0.29.1\diffusers-0.29.1\examples\dreambooth\train_dreambooth_sd3.py", line 1058, in main
tokenizer_three = T5TokenizerFast.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ZeroTwo\AppData\Roaming\Python\Python312\site-packages\transformers\tokenization_utils_base.py", line 2259, in from_pretrained
return cls._from_pretrained(
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ZeroTwo\AppData\Roaming\Python\Python312\site-packages\transformers\tokenization_utils_base.py", line 2493, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ZeroTwo\AppData\Roaming\Python\Python312\site-packages\transformers\models\t5\tokenization_t5_fast.py", line 119, in init
super().init(
File "C:\Users\ZeroTwo\AppData\Roaming\Python\Python312\site-packages\transformers\tokenization_utils_fast.py", line 106, in init
raise ValueError(
ValueError: Cannot instantiate this tokenizer from a slow version. If it's based on sentencepiece, make sure you have sentencepiece installed.
Traceback (most recent call last):
File "C:\Users\ZeroTwo\miniconda3\Scripts\accelerate-script.py", line 9, in
sys.exit(main())
^^^^^^
File "C:\Users\ZeroTwo\miniconda3\Lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
args.func(args)
File "C:\Users\ZeroTwo\miniconda3\Lib\site-packages\accelerate\commands\launch.py", line 1106, in launch_command
simple_launcher(args)
File "C:\Users\ZeroTwo\miniconda3\Lib\site-packages\accelerate\commands\launch.py", line 704, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)

[asomoza]

Hi, the error is giving the solution:

ValueError: Cannot instantiate this tokenizer from a slow version. If it's based on sentencepiece, make sure you have sentencepiece installed.

[asomoza]

Also for your first error, this stabilityaistable-diffusion-3-medium is not correct, it that case it should be stabilityai/stable-diffusion-3-medium but this one also won't work, you need the diffusers version: stabilityai/stable-diffusion-3-medium-diffusers