【Beginner's Guide to Stable Diffusion】Negative prompt is important for v2 models

【Beginner's Guide to Stable Diffusion】Negative prompt is important for v2 models

AI Learning  Assistant NO 2
AI Learning Assistant NO 2
July 28th, 2023

Negative prompt with Stable Diffusion v2.1

Consistent with Max Woolf’s finding, my own experience is that negative prompt is very important for v2 models. Below I used the positive prompt for generating realistic humans but with Stable Diffusion 2.1 model.

a young female, highlights in hair, sitting outside restaurant, brown eyes, wearing a dress, side light

Adding just two to three negative prompts progressively improves the aesthetic of the images. I would say this is pretty near the quality of v1 models.

Negative prompt with Stable Diffusion v1.5

Let’s repeat the exercise on v1.5 model.

The images comes out pretty good without any negative prompts in v1.5. Adding the negative prompt ugly, deformed and disfigured may improve things but it is not as clear as in v2.1. It is as if v1.5 model does not understand these words.

Why does negative prompt become more important in v2?

This is an area I can only speculate… but why not? The two changes in v2 are

  1. Use a larger OpenCLIP language model.

  2. Filtered out NSFW contents in training data.

The first suspect is switching from Open AI’s CLIP model to OpenCLIP. This affects the embeddings of the model. Open AI trained the CLIP model with proprietary data. If the data is highly curated that every single person looks way above average, prompting “woman” would be the same as prompting “beautiful woman”. That would make prompting easier.

My second speculation is that what are deemed NSFW could also be highly aesthetic. It could be a failure of the filter, or its just be the nature of the NSFW images. Excluding NSFW images also unintentionally biases the data towards the bad and ugly.

To see more content about Stable Diffusion from zero click:https://www.hayo.com/article/64c21001ef669957a0d21e63

Reprinted from View Original
Collection Navigation
# 1 [Beginner's Guide] Stable Diffusion Absolute beginner’s guide (+online demo)
# 2 [Beginner's Guide] How to use img2img to turn an amateur drawing to professional with Stable Diffusion (Image-to-image)
# 3 [Stable Diffusion Repair] Beginner’s guide to inpainting (step-by-step examples)
# 4 [Beginner's Guide] How to remove undesirable objects with AI inpainting
# 5 [Beginner's Guide] Video to video with Stable Diffusion (step-by-step)
# 6 [Beginner's Guide] How do you use Stable Diffusion AI?
# 7 [Beginner's Guide] Top Free Stable Diffusion AI image generator sites
# 8 [Beginner's Guide] Stable Diffusion prompt: a definitive guide
# 9 [Beginner's Guide] Instruct Pix2Pix: Edit and stylize photos with text
# 10 【Beginner's Guide to Stable Diffusion】How to build a good prompt?
# 11 【Beginner's Guide to Stable Diffusion】Rules of thumb for building good prompts
# 12 【Beginner's Guide to Stable Diffusion】What is ChatGPT?
# 13 【Beginner's Guide to Stable Diffusion】Sorting out high-quality prompt words from ChatGPT
# 14 【Beginner's Guide to Stable Diffusion】How to use negative prompts?
# 15 【Beginner's Guide to Stable Diffusion】Negative prompt is important for v2 models
# 16 【Beginner's Guide to Stable Diffusion】Boilerplate negative prompts in v2 model
# 17 【Beginner's Guide to Stable Diffusion】What are those parameters, and should I change them?
# 18 【Beginner's Guide to Stable Diffusion】How many images should I generate?
# 19 【Beginner's Guide to Stable Diffusion】Common ways to fix defects in images
# 20 【Beginner's Guide to Stable Diffusion】Garbled faces and eyes problems
# 21 【Beginner's Guide to Stable Diffusion】How to use VAE to improve eyes and faces
# 22 [Beginner's Guide] How to install Stable Diffusion on Windows (AUTOMATIC1111)
# 23 【Beginner's Guide to Stable Diffusion】Beginner’s guide to inpainting (step-by-step examples)
# 24 【Beginner's Guide to Stable Diffusion】What are custom models?
# 25 【Beginner's Guide to Stable Diffusion】How to control image composition?
# 26 【Beginner's Guide to Stable Diffusion】Generating specific subjects
# 27 [Beginner's Guide] How to install and run Stable Diffusion on Apple Silicon M1/M2 Macs
# 28 [Beginner's Guide] Try the Stable Diffusion online demo
# 29 [Beginner's Guide] Beginner’s guide to Stable Diffusion models and the ones you should know
# 30 [Beginner's Guide] What Can Stable Diffusion Do?
# 31 [Beginner's Guide] Realistic human street portrait
# 32 [Beginner's Guide] How to generate animals in Stable Diffusion


no dataCoffee time! Feel free to comment