Lora training questions

[SPCell]

Hello Kohya! Me and my friend have big troubles with training non-human characters. My friend is making a monster from Digimon (https://wikimon.net/Aero_V-dramon) while I'm making a humanoid demon from Doom Eternal (https://doom.fandom.com/wiki/Marauder). Both of us so far trained many various versions of Lora and Lycoris, but results are not satisfying at all. I got best results on AdamW8bit while my friend got them on Dadaptation. Is Stable Diffusion even suitable for training and generating non-human concepts? We tried various datasets and settings, maybe we missed something important? Maybe Prodigy optimizer will help? However, Prodigy optimizer is not available in Colab Lora trainer. Is there a way to integrate Prodigy into Colab?

[RealBeyondMaster]

Hi SPCell
What is your / your friend JSON File for training, meaning all the presets?
Could you post for us please?
I've done about 1000 training sessions by now, should know a thing or two.
On what model are you training your dataset on?
V1.5 might not be suitable for pokemon styles.

[SPCell]

Both of us are training on SD 1.5. Our goal is to make a universal-styled LoRa which can be used both with anime and photorealistic styles. Both of us are also using 3D model renders as dataset and 2-4 drawn artworks (4 in my case and 2 in his case iirc). Digimons also have much more difficult designs than Pokemons. What kinds of settings do you need? We tried like many dozens of various kinds of settings (alpha, dim, Lora, Lycoris, various optimizers like AdamW8bit, Dadaptation, Lion, various LRs and dataset variations). My friend got best results on Dadaptation which are somewhat satisfying and he wants to try Prodigy since we read that it's a direct upgrade to Dadaptation.