ControlNet

ControlNet は、条件を追加して拡散モデルを制御するニューラルネットワーク構造です。

色塗りしたり画像通りに色々なポーズをとることができます

https://github.com/lllyasviel/ControlNet

ControlNetのインストール

WebU Extensionsタブ

https://github.com/Mikubill/sd-webui-controlnet.git

モデルのダウンロード

下記サイトからダウンロードします

https://huggingface.co/lllyasviel/ControlNet/tree/main/models

ダウンロードしたファイルを下記フォルダーに入れます
/content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/

軽量なものは下記です

https://huggingface.co/webui/ControlNet-modules-safetensors/tree/main

最後にWebUI起動し直します

colab

colaboの例になります
毎回設定が大変なのでこちらを使います

画像の閲覧と日本語化をしています

!pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117
!pip install -U --pre triton
!pip install xformers==0.0.16rc425

#!pip install opencv-python
#!pip install prettytable


!git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui

!git clone https://github.com/AlUlkesh/stable-diffusion-webui-images-browser.git /content/stable-diffusion-webui/extensions/stable-diffusion-webui-images-browser	

!wget  https://raw.githubusercontent.com/yuuki76/stable-diffusion-webui/ja-translation/localizations/ja_JP.json -O   /content/stable-diffusion-webui/localizations/ja_JP.json
!echo {\"localization\": \"ja_JP\"} >/content/stable-diffusion-webui/config.json

!wget https://huggingface.co/camenduru/sd15/resolve/main/v1-5-pruned-emaonly.ckpt -O /content/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned-emaonly.ckpt

%cd /content/stable-diffusion-webui/extensions/
!git clone https://github.com/Mikubill/sd-webui-controlnet
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_canny-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_canny-fp16.safetensors
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_depth-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_depth-fp16.safetensors
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_hed-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_hed-fp16.safetensors
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_mlsd-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_mlsd-fp16.safetensors
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_normal-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_normal-fp16.safetensors
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_openpose-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_openpose-fp16.safetensors
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/control_scribble-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_scribble-fp16.safetensors
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/t2iadapter_keypose-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/t2iadapter_keypose-fp16.safetensors
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/t2iadapter_seg-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/t2iadapter_seg-fp16.safetensors
!wget https://huggingface.co/webui/ControlNet-modules-safetensors/resolve/main/t2iadapter_sketch-fp16.safetensors -O /content/stable-diffusion-webui/extensions/sd-webui-controlnet/models/t2iadapter_sketch-fp16.safetensors


%cd /content/stable-diffusion-webui

!python launch.py --share --xformers --enable-insecure-extension-access

使い方

必要に応じて画像をアップロードした後,enableを選択してプロフェッサーとモデルを選びます。