7-ControlNet

Scribble ControlNet

Here is a simple example of how to use ControlNets. This example uses the Scribble ControlNet and the AnythingV3 model. You can load this image into ComfyUI to get the complete workflow.

Here is the input image I used for this workflow:

T2I-Adapter vs ControlNets

T2I-Adapters are much more efficient than ControlNets, so I highly recommend them. ControlNets will significantly slow down the generation speed, while T2I-Adapters have almost no negative impact on speed.

In ControlNets, the ControlNet model is run once per iteration. For the T2I-Adapter, the model runs only once in total.

T2I-Adapters are used in the same way as ControlNets in ComfyUI: by using the ControlNetLoader node.

Here is the input image that will be used in this source example:

Here’s how you use the depth T2I-Adapter:

Here’s how you use the depth ControlNet. Note that this example uses the DiffControlNetLoader node because the ControlNet used is a diff control net. Diff control nets require the model weights to be loaded correctly. The DiffControlNetLoader node can also be used to load regular controlnet models. When loading regular controlnet models, it behaves the same way as the ControlNetLoader node.

You can load these images into ComfyUI to get the complete workflow.

Pose ControlNet

Here is the input image that will be used in this example:

Here is an example using a first pass with AnythingV3 with the ControlNet and a second pass without the ControlNet with AOM3A3 (Abyss Orange Mix 3) and using their VAE.

You can load this image into ComfyUI to get the complete workflow.

Mixing ControlNets

Multiple ControlNets and T2I-Adapters can be applied in this way with interesting results:

You can load this image into ComfyUI to get the complete workflow.

Input images: Input images: