[Stable Diffusion Mode] Model merge example with other

[Stable Diffusion Mode] Model merge example with other

AI  Learning Assistant No 1
AI Learning Assistant No 1
July 31st, 2023

Merging two models

To merge two models using AUTOMATIC1111 GUI, go to the Checkpoint Merger tab and select the two models you want to merge in Primary model (A) and Secondary model (B).

Adjust the multiplier (M) to adjust the relative weight of the two models. Setting it to 0.5 would merge the two models with equal importance.

After pressing Run, the new merged model will be available for use.

Example of a merged model

Here are sample images from merging F222 and Anything V3 with equal weight (0.5):

The merged model sits between the realistic F222 and the anime Anything V3 styles. It is a very good model for generating illustration art with human figures.

Model variants

On a model download page, you may see several variants of the model.

  • Pruned
  • Full
  • EMA-only
  • FP16
  • FP32
  • .pt
  • .safetensor

This is confusing! Which one should you download?

Pruned, full, EMA-only models

Some Stable Diffusion checkpoint models consist of two sets of weights: (1) The weights after the last training step, and (2) the average weights over the last few training steps called EMA (exponential moving average).

If you are only interested in using the model, you only need the EMA-only model. These are the weights you actually use when you use the model. They are sometimes called pruned models.

You will only need the full model (i.e. A checkpoint file consisting of two sets of weights) if you want to fine-tune the model with additional training.

So, download the pruned or EMA-only model if you simply want to use it to generate images. This saves you some disk space. Trust me, your hard drive will fill up very soon!

fp16/fp32 models

FP stands for floating point. It is a computer’s way of storing decimal numbers. Here the decimal numbers are the model weights. FP16 takes 16 bits per number and is called half precision. FP32 takes 32 bits and is called full precision.

For deep learning models (such as Stable Diffusion), the training data is pretty noisy. You rarely need full precision when you use the model. The extra precision just stores noise!

So, download the FP16 models if available. They are about half as big. This saves you a few GB!

Safetensor models

The original pytorch model format is .pt. The downside of this format is that it is not secure. Someone can pack some malicious code in it. The code can run on your machine when you use the model.

Safetensor is an improved version of the PT model format. It does the same thing of storing the weights, but it will not execute any codes.

So, download the safetensor version whenever it is available. If not, make sure you download the PT files from a trust-worthy source.

Other model types

Four main types of files can be called “models”. Let’s clarify them so you know what people are talking about.

  • Checkpoint models: These are the real Stable Diffusion models. They contain all you need to generate an image. No additional files are required. They are large, typically 2 – 7 GB. They are the subject of this article.
  • Textual inversions: Also called embeddings. They are small files defining new keywords to generate new objects or styles. They are small, typically 10 – 100 KB. You must use them with a checkpoint model.
  • LoRA models: They are small patch files to checkpoint models for modifying styles. They are typically 10-200 MB. You must use them with a checkpoint model.
  • Hypernetworks: They are additional network modules added to checkpoint models. They are typically 5 – 300 MB. You must use them with a checkpoint model.

To see more content about Stable Diffusion from zero click:https://www.hayo.com/article/64c21001ef669957a0d21e63

Reprinted from View Original


no dataCoffee time! Feel free to comment