Blender + Stable Diffusion experiments

I've been trying out the AI Render plugin in Blender that allows you to use Stable Diffusion directly in Blender. I've done some experimenting with it and got some half-decent results I'd like to share.

The way it works is, you can model out whatever you want, light it however you want and color it however you want, and then describe what it's depicting. Stable Diffusion will take into consideration all of it, including the coordinates, sizes and colors of the objects.

Here's the prompt I used for the thumbnail image; "A sci-fi hibernation tank, glass window, technical room, tiny lights, azure blue". And here's the blender scene it's rendered from - this only took a couple of minutes to throw together.

Some more variations of that prompt in various styles.

How about a medieval ceremonial sacrificial chamber. I'll admit this was heavily inspired by Manos: The Hands of Fate - if you haven't seen that... masterpiece, I'd definitely not recommend it.

Next up: "a portal to another dimension, pillars of energy, lightning, surreal".

This definitely opens up some possibilities, though I'll admit it's still difficult to get exactly what you're envisioning. I had to spend some time getting good results, and several prompts failed to produce anything of interest. Still, this method does offer increased flexibility in the images you can generate using Stability AI, by being able to place the rough shapes, colors and positions of what you're trying to generate.

What is Hive?

To learn more about Hive, this article is a good place to start: What is Hive?. If you don't already own a Hive account, click here to get one.