AEIONic.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "49n2AC6spxQS"
      },
      "source": [
        "# 🎨 **AEIONic Diffusion Deluxe Edition** 👨‍🎨️ - Full Featured Fancy Flet/Flutter Framework\n",
        "\n",
        "*...using `🤗HuggingFace🧨diffusers`* and many advanced Open-Sourced AI bonus features...\n",
        "\n",
        "---\n",
        "### Project at [AEIONic.com](https://AEIONic.com) - Designed by [**Skquark**, Inc.](https://www.Skquark.com) 😋\n",
        "<p align=center>\n",
        "<a href=\"https://github.com/Skquark/AEIONic/blob/main/AEIONic_Diffusion_Deluxe.ipynb\"><img src=\"https://badgen.net/badge/icon/github?icon=github&label\" alt=\"Github\"></a> <a href=\"https://github.com/Skquark/AEIONic\"><img src=\"https://badgen.net/github/release/Skquark/AEIONic/stable\" alt=\"Release version\"></a>\n",
        "<a href=\"https://colab.research.google.com/github/Skquark/AEIONic/blob/main/AEIONic_Diffusion_Deluxe.ipynb\"><img src=\"https://img.shields.io/badge/Open-in%20Colab-brightgreen?logo=google-colab&style=flat-square\" alt=\"Open in Google Colab\"/></a>\n",
        "</p>\n",
        "\n",
        "*   Runs in a pretty WebUI using [Flet - Flutter for Python](https://flet.dev) with themes, interactivity & sound\n",
        "*   Saves all settings/parameters in your config file, don't need to Copy to Drive\n",
        "*   Run a batch list of prompts at once, so queue many and walk away\n",
        "*   Option to override any parameter per prompt in queue\n",
        "*   Use Stable Diffusion [XL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9), [2.1](https://huggingface.co/stabilityai/stable-diffusion-2-1), [2.0](https://huggingface.co/stabilityai/stable-diffusion-2), [1.5 ](https://huggingface.co/runwayml/stable-diffusion-v1-5), or [1.4](https://huggingface.co/CompVis/stable-diffusion-v1-4) Checkpoint Model File\n",
        "*   Supports many custom community Finetuned Model checkpoints & DreamBooth Libraries\n",
        "*   Supports Stable Diffusion image2image to use an init_image\n",
        "*   Supports Stable Diffusion [Inpaint](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting) mask_image layer\n",
        "*   Supports Negative Prompts to specify what you don't want\n",
        "*   Supports Long Prompt Weighting to emphasize (positive) & [negative] word strengths\n",
        "*   Prompt tweening to combine latent space of 2 prompts in a series\n",
        "*   Can use Interpolation to walk steps between latent space of prompt list\n",
        "*   Can use CLIP Guidance with LAION & OpenAI ViT models\n",
        "*   Can use Textual Inversion Conceptualizer with 760+ Community Concepts\n",
        "*   Can Centipede prompts as init images feeding down the list\n",
        "*   Can use Composable weighted | prompt | segments for img2img\n",
        "*   Can use iMagic to edit init image with your prompt \n",
        "*   Can use Kakaobrain unCLIP Generator, which gives similar results to DALL-E 2\n",
        "*   Can save all images to your Google Drive (PyDrive support soon)\n",
        "*   Can Upscale automatically with Real-ESRGAN enlarging\n",
        "*   Option to use Stability-API tokens for more samplers, bigger size & CPU runtime\n",
        "*   Embeds exif metadata directly into png files\n",
        "*   Disabled NSFW filtering and added custom sampler options\n",
        "*   Renames image filenames to the prompt text, with options\n",
        "*   OpenAI Prompt Generator, Remixer, Brainstormer & Noodle Soup Prompt Writer included\n",
        "*   Standalone ESRGAN Upscaler for batch uploads and image splitting\n",
        "*   Train your own models with DreamBooth, LoRA or Textual-Inversion & upload to HuggingFace\n",
        "*   Replicate Material Diffusion to make Seamless Tile Textures\n",
        "*   Experimental HarmonAI Dance Diffusion audio generator\n",
        "*   Experimental DreamFusion 3D model generator with texture & video\n",
        "*   Image2Text CLIP-Interrogator to get prompt from images\n",
        "*   Bonus Image Generators DALL-E 2/3 and Kandinsky 2/3 to compare\n",
        "*   Additional features added regularly...\n",
        "\n",
        "If you have a powerful enough GPU with > 8GB VRAM, you can run Desktop app locally with [AEIONic-setup.exe](https://colab.research.google.com/github/Skquark/AEIONic/blob/main/AEIONic-Diffusion-Deluxe/AEIONic-setup.exe) or [aeionic-linux.sh](https://colab.research.google.com/github/Skquark/AEIONic/blob/main/AEIONic-Diffusion-Deluxe/aeionic-linux.sh). Still in beta...\n",
        "\n",
        "Try these other useful notebooks [Enhanced DiscoArt](https://colab.research.google.com/github/Skquark/structured-prompt-generator/blob/main/DiscoArt_%5B_w_Batch_Prompts_%26_GPT_3_Generator%5D.ipynb), [Enhanced Stable Diffusion](https://colab.research.google.com/github/Skquark/structured-prompt-generator/blob/main/Enhanced_Stable_Diffusion_with_diffusers.ipynb) and [Structured Prompt Generator](https://colab.research.google.com/github/Skquark/structured-prompt-generator/blob/main/Structured_Prompt_Generator.ipynb)\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "KTwZrpxEuKR0"
      },
      "outputs": [],
      "source": [
        "#@title ## **▶️ Run AEIONic Diffusion Deluxe** - Easy Web App Launcher\n",
        "#@markdown ## 🧰 Installs Flet/Flutter UI Framework & Initializes AI Friends\n",
        "#markdown If you're running on Google Colab, authorize Google Drive with the popup to connect storage. If you're on a local or cloud Jupyter Notebook, you must get a OAuth json using instructions below to save images to GDrive, however the feature is not currently working and in progress. When set, continue to run the Web UI and experiment away..\n",
        "#@markdown We'll connect to your Google Drive and save all of your preferences in realtime there, as well as your created images. This is the folder location and file name we recommend in your mounted gdrive, will be created if you're new, but you can save elsewhere.  Launches webpage with localtunnel or ngrok, if app doesn't get past loading screen, switch tunnel_type & run again.\n",
        "#markdown <br>Note: You'll need to _Restart Runtime_ the first time you run to force update, then _Reconnect_ and Run this again.\n",
        "storage_type = \"Colab Google Drive\" #@param [\"Colab Google Drive\", \"Local Drive\"]\n",
        "Google_OAuth_client_secret_json = \"/content/client_secrets.json\" #param {'type': 'string'}\n",
        "saved_settings_json = '/content/drive/MyDrive/AI/AI_Projects/aeionic-settings.json' #@param {'type': 'string'}\n",
        "tunnel_type = \"ngrok\" #@param [\"localtunnel\", \"ngrok\"]\n",
        "auto_launch_website = False #@param {'type': 'boolean'}\n",
        "bools = \" --auto_launch_website\" if auto_launch_website else \"\"\n",
        "from IPython.display import clear_output\n",
        "%cd /content\n",
        "!wget -q -N https://github.com/Skquark/AEIONic/raw/main/aeionic_colab.py -P /content/\n",
        "clear_output()\n",
        "%run aeionic_colab.py --storage_type \"{storage_type}\" --saved_settings_json \"{saved_settings_json}\" --tunnel_type {tunnel_type}{bools}\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "gfi-vDp6A8it"
      },
      "outputs": [],
      "source": [
        "#@markdown ---\n",
        "raise(RuntimeError(\"Stopping Execution for Run All to end...\"))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "jf90tMc3tu9q"
      },
      "outputs": [],
      "source": [
        "#@title 🖥️ Check GPU Status (A100 > G100 > V100 > P100 > T4 > K8)\n",
        "import subprocess, psutil, torch\n",
        "simple_nvidia_smi_display = True #@param {type:\"boolean\"}\n",
        "cpu_memory = psutil.virtual_memory().total / (1024 * 1024 * 1024)\n",
        "gpu_memory = torch.cuda.get_device_properties(0).total_memory / (1024 * 1024 * 1024)\n",
        "mem = f\"GPU VRAM {gpu_memory:.0f}GB | CPU RAM {cpu_memory:.0f}GB\"\n",
        "if simple_nvidia_smi_display:\n",
        "    print(subprocess.run(['nvidia-smi', '-L'], stdout=subprocess.PIPE).stdout.decode('utf-8') + mem)\n",
        "else:\n",
        "    print(subprocess.run(['nvidia-smi'], stdout=subprocess.PIPE).stdout.decode('utf-8'))\n",
        "    print(subprocess.run(['nvidia-smi', '-i', '0', '-e', '0'], stdout=subprocess.PIPE).stdout.decode('utf-8') + mem)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "zE0dN7OiVBz0"
      },
      "outputs": [],
      "source": [
        "#@title ## ▶️ **Inpaint Dream Mask Maker** - GradIO WebUI (temporary until I make better Mask Painter in Flet UI)\n",
        "#@markdown Prepared your B&W ***mask_image*** and resizes ***init_image*** to add to your batch prompts list easily.\n",
        "max_image_resolution = 960 #@param {'type':'slider', min:256, max:1024, step:64}\n",
        "init_image_strength_override = False #param {type:'boolean'}\n",
        "import string\n",
        "#import PIL.ImageOps\n",
        "#@title ## 📥 **Install GradIO Web Server** (_if_ you want to use Inpaint Helper interface)\n",
        "try:\n",
        "  import gradio as gr\n",
        "except ImportError:\n",
        "  run_sp(\"pip install gradio datasets tqdm\")\n",
        "  pass\n",
        "finally:\n",
        "  import gradio as gr\n",
        "\n",
        "#import numpy as np\n",
        "import torch\n",
        "from torch import autocast\n",
        "import requests\n",
        "import PIL\n",
        "from PIL import Image, ImageOps\n",
        "from io import BytesIO\n",
        "clear_output()\n",
        "#print(f'{Color.BLUE}Successfully Installed GradIO...{Color.END}')\n",
        "\n",
        "image = None\n",
        "def multiple_of_64(x):\n",
        "    return int(round(x/64)*64)\n",
        "def multiple_of_8(x):\n",
        "    return int(round(x/8)*8)\n",
        "def scale_dimensions(width, height):\n",
        "  max = int(max_image_resolution)\n",
        "  r_width = width\n",
        "  r_height = height\n",
        "  if width < max and height < max:\n",
        "    if width >= height:\n",
        "      ratio = max / width\n",
        "      r_width = max\n",
        "      r_height = int(height * ratio)\n",
        "    else:\n",
        "      ratio = max / height\n",
        "      r_height = max\n",
        "      r_width = int(width * ratio)\n",
        "    width = r_width\n",
        "    height = r_height\n",
        "  if width >= height:\n",
        "    if width > max:\n",
        "      r_width = max\n",
        "      r_height = int(height * (max/width))\n",
        "    else:\n",
        "      r_width = width\n",
        "      r_height = height\n",
        "  else:\n",
        "    if height > max:\n",
        "      r_height = max\n",
        "      r_width = int(width * (max/height))\n",
        "    else:\n",
        "      r_width = width\n",
        "      r_height = height\n",
        "  return multiple_of_64(r_width), multiple_of_64(r_height)\n",
        "\n",
        "def format_filename(s):\n",
        "    valid_chars = \"-_.() %s%s\" % (string.ascii_letters, string.digits)\n",
        "    filename = ''.join(c for c in s if c in valid_chars)\n",
        "    filename = filename.replace(' ','_')\n",
        "    return filename[:24]\n",
        "\n",
        "def available_file(folder, name, idx, mask=False):\n",
        "  available = False\n",
        "  mask_str = '-mask' if mask else ''\n",
        "  while not available:\n",
        "    if os.path.isfile(os.path.join(folder, f'{name}-{idx}{mask_str}.png')):\n",
        "      idx += 1\n",
        "    else: available = True\n",
        "  return os.path.join(folder, f'{name}-{idx}{mask_str}.png')\n",
        "\n",
        "def available_folder(folder, name, idx):\n",
        "  available = False\n",
        "  while not available:\n",
        "    if os.path.isdir(os.path.join(folder, f'{name}-{idx}')):\n",
        "      idx += 1\n",
        "    else: available = True\n",
        "  return os.path.join(folder, f'{name}-{idx}')\n",
        "  \n",
        "def create(prompt, img, option):\n",
        "    global prefs\n",
        "    mask_img = img[\"mask\"]\n",
        "    init_img = img[\"image\"]\n",
        "    w, h = init_img.size\n",
        "    w, h = scale_dimensions(w, h)\n",
        "    mask_img = mask_img.convert(\"1\")\n",
        "    mask_img = mask_img.resize((w, h), resample=PIL.Image.LANCZOS)\n",
        "    if option == \"Replace everything else\":\n",
        "       mask_img = ImageOps.invert(mask_img.convert('RGB'))\n",
        "       #mask = mask.convert('1')\n",
        "    init_img = init_img.convert(\"RGB\")\n",
        "    init_img = init_img.resize((w, h))\n",
        "\n",
        "    fname = format_filename(prompt)\n",
        "    mask_name = f'{root_dir}{fname}-mask.png'\n",
        "    img_name = f'{root_dir}{fname}.png'\n",
        "    if os.path.isfile(img_name):\n",
        "      img_name = available_file(root_dir, fname, 1, False)\n",
        "    if os.path.isfile(mask_name):\n",
        "      mask_name = available_file(root_dir, fname, 1, True)\n",
        "    mask_img.save(mask_name)\n",
        "    init_img.save(img_name)\n",
        "    #print(str(img.name))\n",
        "    #print(str(img.filename))\n",
        "    image_strength = f'init_image_strength=0.4, ' if init_image_strength_override else ''\n",
        "    print(f'    Dream(\"{prompt}\", init_image=\"{img_name}\", mask_image=\"{mask_name}\", {image_strength}width={w}, height={h}),')\n",
        "    if prefs is not None:\n",
        "      prompt_pref = prefs['prompt_list']\n",
        "      new_dream = {'prompt': prompt, 'init_image': img_name, 'mask_image': mask_name, 'width': w, 'height': h}\n",
        "      if init_image_strength_override:\n",
        "        new_dream['init_image_strength':]\n",
        "      prompt_pref.append(new_dream)\n",
        "      prefs['prompt_list'] = prompt_pref\n",
        "      print(\"Added to your AEIONic Diffusion Deluxe Prompts List (in saved preferences)\")\n",
        "    #display(mask)\n",
        "    return f'Dream(\"{prompt}\", init_image=\"{img_name}\", mask_image=\"{mask_name}\", {image_strength}width={w}, height={h}),'\n",
        "\n",
        "block = gr.Blocks(css=\".container { max-width: 1200px; margin: auto; }\")\n",
        "with block as demo:\n",
        "    gr.Markdown(\"<h1><center>Stable Diffusion Inpaint Dream Mask Maker</center></h1>\")\n",
        "    with gr.Group():\n",
        "        with gr.Row().style(mobile_collapse=False, equal_height=False):\n",
        "                text = gr.Textbox(\n",
        "                    label=\"Enter your Inpaint prompt\", show_label=True, max_lines=1\n",
        "                ).style(\n",
        "                    border=(True, False, True, True),\n",
        "                    rounded=(True, False, False, True),\n",
        "                    container=False,\n",
        "                )\n",
        "        with gr.Row().style(mobile_collapse=False, equal_height=True):\n",
        "          option = gr.Radio(label=\"Masking\", choices=[\"Replace selection\", \"Replace everything else\"], value=\"Replace selection\")\n",
        "          btn = gr.Button(\"Prepare Dream\").style(\n",
        "                    margin=True,\n",
        "                    rounded=(True, True, True, True),\n",
        "                )\n",
        "        image = gr.Image(\n",
        "            tool=\"sketch\",\n",
        "            source=\"upload\",\n",
        "            #image_mode=\"L\", shape=(42, 42), invert_colors=True,\n",
        "            label=\"Input Init Image\",\n",
        "            type=\"pil\"\n",
        "            #type=\"numpy\"\n",
        "        )\n",
        "        gallery = gr.Markdown('')\n",
        "        #gallery = gr.Gallery(label=\"Generated mask_image\", show_label=False).style(grid=[2], height=ScrollMode.AUTO)\n",
        "        text.submit(create, inputs=[text,image,option], outputs=gallery)\n",
        "        btn.click(create, inputs=[text,image,option], outputs=gallery)\n",
        "\n",
        "demo.launch(debug=True)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "za8wL7yUJJYI"
      ],
      "include_colab_link": true,
      "private_outputs": true,
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3.8.10 64-bit (microsoft store)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.8.10"
    },
    "vscode": {
      "interpreter": {
        "hash": "aecaad1160391d286dea8adfafabc0106aa931e7deb6a40f00e2f89f1c4ee412"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}