If you are a professional designer, or require precise control of AI generated image, you must learn about ControlNert.

Basic Theory About ControlNet

ControlNet is regarded as an extension of Stable Diffusion, providing specific information to precisely control characteristics of AI-generated images, which are challenging to manage with text2img or img2img methods alone. Without ControlNet, controlling AI-generated images is difficult due to inherent randomness. Control is a crucial factor in productivity when specific requirements must be met. For example, if you input "dancing," the AI will produce various dance gestures. However, if you upload a particular gesture to ControlNet, the AI will generate images featuring only that specific dance gesture.

ControlNet's structure involves a preprocessor that extracts characteristic information from the image, and then the trained ControlNet model reads this information and guides Stable Diffusion in generating the final image.

Basic Installation and Guidance

（If you are using MimicPC, it's all done for you!）

https://www.mimicpc.com/?co-from=HN%3Futm_source%3Dstartuptile.com

Open auto1111, click Extensions-Avaliable, then serach Controlnet, click download.

Model should be download as well, download the model base on your needs. Then upload the model into ControlNet folder.

https://huggingface.co/lllyasviel/sd_control_collection/tree/main

After that, ControlNet is ready to use. Firstly, upload the information image, choose preprocessor and model.

Let's discuss the parameter settings of ControlNet. The Control Weight determines the intensity of the control applied to the image, typically set to a value of 1. The Starting Control Step and Ending Control Step manage the duration during which ControlNet is effective. The Control Mode determines whether the image is closer to the prompt or to ControlNet's guidance, with a balanced setting usually recommended. For beginners, it is advisable to keep other settings at their default values.

Explanation of 5 Main ControlNet Models

The first model, as we previous mentioned, Openpose. Openpose control the gusture, hand and face details. If you choose OpenPose as the preprocessor and select the corresponding model, then click the firework icon, you will see a clear outline of the information image.

The second model is Depth, which control the composition of the space in depth. For example, the black area is far away while the white area is closer.

The third model is Canny, which control line profiles. Canny will draw the shape of objects by lines, and control the image restoration.

The forth model is SoftEdge, which also control line profiles, but in a smooth and relax way.

The last model is Scibble, users could draw by themselves to lead the generated image. Scribble could be extracted from image or draw by users.

Applications of Multi-ControlNet

Users can use multiple ControlNets to achieve the best results. To do this, go to the settings and adjust the number of ControlNets as needed. For example, using Depth and OpenPose together can precisely position a character's hand in front of their face.