[Advanced Skills of Stable Diffusion] LoCon, which is stronger than LoRA, is here!
The relationship between LoRA and LoCon
LoCon is evolved from LoRA, and its full name is LoRA for Convolution Network. The improvement of LoCon relative to LoRA. Simply put, LoRA is a method for fine-tuning a large language model, which reduces the number of parameters and maintains a comparable effect by reducing the rank of the parameter matrix. LoCon is the application of LoRA in the convolutional neural network, which reduces the number of parameters by reducing the dimensionality of the convolution kernel. From the code level: # for convolution
self.lora_down = nn.Conv2d(in_dim, lora_dim, k_size, stride, padding, bias=False) self.lora_up = nn.Conv2d(lora_dim, out_dim, (1, 1), bias=False)# For fully connected layer self .lora_down = nn.Linear(in_dim, lora_dim, bias=False) self.lora_up = nn.Linear(lora_dim, out_dim, bias=False)
LoCon deployment method
The deployment and usage methods are consistent with LoRA. Specific steps are as follows: 1. Make sure the LoCon extension is installed Download github.com/KohakuBlueleaf/a1111-sd-webui-locon 2. Put the file in the folder models/lora click the button below the Generate button (purple icon) show extra networks 3. Go to the tabs and refresh Lora as needed Click the one you want to use and it will be added to the prompt middle 3. Make sure to adjust the weight, the default is 1, which is a bit high, and it is more appropriate to set it to 0.5
Advantages of LoCon 1. The effect of LoCon image output is better than that of LoRA 2. LoCon learning a very small number of samples can have relatively good results, reducing training and operating costs Disadvantages of LoCon
1. The drawing effect is relatively slow 2. At present, LoCon has increased the amount of parameters, at most it is an adjustment similar to hyperparameters, and it has not completely solved the LoRA problem of convolutional neural networks. At present, there is little information about LoCon, and there will be new knowledge and iterations in the future. Welcome to discuss together :)
