What is Stable Diffusion?

It's a fully open source competitor to dall-e; it enables you to generate images from text, generate images from images and upscaling with near infinite configurability. Check out the features!

Check out some of the examples of SD produced art on /r/stablediffusion.

You can also check out the prompts which generated SD images through various web services, to get inspiration or to take a shortcut directly to an art style you wish to utilise.

Despite being only a couple months only, this technology has lead to plenty of services and innovations, check out the awesome-stable-diffusion git repo for many links.

Setup guide

I've been using the platform called DataCrunch to create temporary stable diffusion instances, rather than use Google Colab.

Using a 'Spot' instance type, for a 16GB GPU server it's only $0.25/hour, a figure far lower than competitor platforms and colab. This instance type can be hibernated at any time by a full paying user, so if that's unacceptable select the higher on demand instance type.

So, select a GPU instance on datacrunch, and select the image type Ubuntu 22.04 (minimal image type); We require Ubuntu 22.04 as we need python 3.10.

Input an instance storage size of at least 60GB, this will give you several gigs of space to generate art, however if you want more wiggle room boost this up to 100GB and you'll not have to worry about filling up hdd too quickly.

Generate an SSH key with puttygen, upload this key to datacrunch and select this key within the instance creation section.

Once the instance has been provisioned, grab the IP address and load it with your SSH key in an SSH client (e.g. Termius, putty, etc).

Once you're in the server, you're going to need to install dependencies and setup a non-root user.

sudo apt-get update
sudo apt-get install python3-pip python3-venv git-lfs
sudo adduser art

You'll be prompted to provide a password and details for the new user art.

Now let's install CUDA:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda

Cool, now let's switch to the new art user and get started setting things up:

su - art
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
cd stable-diffusion-webui
cd models
cd Stable-diffusion
git clone https://huggingface.co/CompVis/stable-diffusion-v-1-4-original

Note:
You'll need to create a huggingface account & to provide your username and access token for the above git clone step.

Now if you want to use upscaling tech, the repo says to follow these steps to get the required REAL-ESRGAN data:

cd ..
mkdir ERSGAN
cd ERSGAN
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth
wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.1/RealESRGAN_x2plus.pth

However, it's possible these upscaling models can be downloaded after selecting them in the settings, without the above downloads in place. It seems to be a slight inconsistency in the parent repo docs.

Now let's configure our webui:

nano webui-user.sh

Within the file, edit the COMMANDLINE_ARGS line, enabling xformers (speedup), medram (since we're using 16GB not 40/80/160..), --share to activate a simple gradio link & --gradio-auth to set the username/password combo for the gradio page.

export COMMANDLINE_ARGS="--medvram --share --gradio-auth myUserName:mySuperSecretPassword"

Once you've performed all the above steps, the final step to perform is:

./webui.sh

If you run into any issues check out the following page: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues

So what will kind of art will you choose to generate?

I've been trying out this technology, messing about with it and the following are a few of my first initial attempts without any upscaling, each of these started from an embarrassingly simple mspaint image 😅

I plan on making some art for a content creator, then I'll be touching up some of my existing NFTs and looking into creating some new NFTs for giveaway purposes.