If you have been using the AI art generator created by Stability AI in the form of Stable Diffusion and have been enjoying the latest release in the form of SDXL. But would like to have more control over the diffusion model, this guide will provide an overview of how to install a custom-built graphical user interface.
The ComfyUI user interface is a fantastic tool that allows users to design and execute advanced stable diffusion pipelines using a graph/nodes/flowchart-based interface. This innovative platform was developed by an anonymous developer who sought to understand the intricacies of Stable Diffusion and desired a clean, powerful tool that would allow them to experiment with Stable Diffusion without limitations.
ComfyUI is designed for anyone interested in creating complex workflows with Stable Diffusion or those who wish to gain a deeper understanding of how Stable Diffusion operates. The interface closely mirrors the workings of Stable Diffusion, and the code is significantly simpler to comprehend than other Stable Diffusion user interfaces.
One of the standout features of ComfyUI is its nodes/graph/flowchart interface, which allows users to experiment and create complex Stable Diffusion workflows without the need for coding. This feature is a game-changer for those who want to delve into the world of Stable Diffusion but lack the coding skills traditionally required.
How to install ComfyUI
ComfyUI supports SD1.x, SD2.x, and SDXL, and it also features an asynchronous queue system. It boasts many optimizations, including the ability to only re-execute the parts of the workflow that change between executions. This feature significantly enhances efficiency and reduces unnecessary computational load.
Other articles you may find of interest on the subject of Stable Diffusion :
- How to create consistent characters in Stable Diffusion AI art
- How to use Stable Diffusion to face swap images for consistency
- How to install Stable Diffusion SDXL 1.0 text-to-image Ai art
- Stable Diffusion XL NVIDIA TensorRT performance improvement
- Learn how to use Stable Diffusion SDXL 1.0 to create AI artwork
- 10 Stable Diffusion extensions for next-level creativity
For users with GPUs that have less than 3GB vram, ComfyUI offers a command line option: –lowvram. This option is automatically enabled on GPUs with low vram, ensuring that the interface remains accessible to a wide range of users. Moreover, ComfyUI can function even without a GPU with the –cpu option, although this does slow down the process.
ComfyUI can load ckpt, safetensors, and diffusers models/checkpoints, standalone VAEs, and CLIP models. It also supports embeddings/textual inversion, Loras (regular, locon, and loha), hypernetworks, and loading full workflows (with seeds) from generated PNG files. Users can save and load workflows as Json files, and the nodes interface can be used to create complex workflows like one for Hires fix or even more advanced ones.
ComfyUI : XY Plot Tutorial
The interface also supports area composition, inpainting with both regular and inpainting models, ControlNet and T2I-Adapter, upscale models (ESRGAN, ESRGAN variants, SwinIR, Swin2SR, etc.), unCLIP Models, GLIGEN, model merging, and latent previews with TAESD. ComfyUI starts up very quickly and works fully offline, ensuring that it will never download anything. A config file is also available to set the search paths for models.
For those looking to increase generation speed, ComfyUI recommends using the regular loaders/Load Checkpoint node to load checkpoints. This will auto pick the right settings depending on your GPU. Users can set this command line setting to disable the upcasting to fp32 in some cross attention operations, which will increase speed. However, it’s important to note that this will likely result in black images on SD2.x models. If users opt for xformers, this option will not have any effect.
ComfyUI is a powerful, user-friendly interface that allows users to explore and experiment with Stable Diffusion without restrictions. Its wide range of features and optimizations make it a valuable tool for anyone interested in Stable Diffusion, regardless of their coding skills or the capabilities of their hardware.
Latest Geeky Gadgets Deals
Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, Geeky Gadgets may earn an affiliate commission. Learn about our Disclosure Policy.