HomeTutorials
[SD Advanced Tutoria]ChatGPT: How to generate prompts for Stable Diffusion
4

[SD Advanced Tutoria]ChatGPT: How to generate prompts for Stable Diffusion

AI Learning  Assistant NO 2
AI Learning Assistant NO 2
August 22nd, 2023

Can ChatGPT generate high-quality prompts for Stable Diffusion? The answer is yes. But you will need special methods to tease them out. Once you learn how to do it, the results will surprise you. In this article, I will show you how to do that step-by-step.

What is ChatGPT?

ChatGPT is a chatbot released by Open AI in November 2022. It is built on the GPT-3 family of Large Language Model which was trained with text containing hundreds of billions of words on the internet.

How long is a book with hundreds of billions words? It would take a person more than five thousand years to read it all out.

As a result, ChatGPT contains vast amount of human knowledge. it can generate essays, poems, fictions and even computer codes.

How was ChatGPT trained?

ChatGPT was fine-tuned on top of GPT-3.5. GPT-3 itself is a powerful large language model. It operates like autocomplete. You type in a few words or a few sentences, the model predicts the most likely next words. Because the large amount of data it was trained on, the autocomplete of GPT-3 is supercharged. It can produce an essay of coherent and (sometimes) accurate content.

Human trainers played an important role in fine-tuning ChatGPT’s response. The model was trained to provide answers human trainers would give. Human trainers also ranked ChatGPT’s answers during the training phase so that it would prioritize answers humans prefer.

Does ChatGPT know anything about Stable Diffusion?

Unfortunately, the answer is no. You can easily check by asking ChatGPT the question “What is Stable Diffusion?”. Its answer is

Stable diffusion refers to a type of stochastic process that describes the spread of a certain quantity, such as information or a disease, through a network of individuals or nodes. The term “stable” is used to indicate that the process reaches a steady state, in which the rate at which the quantity spreads through the network is balanced by the rate at which it decays. The most common example of stable diffusion is the spread of a rumor through a social network.

We can debate whether this is complete nonsense, but we should all agree this is NOT Stable Diffusion. Its training data likely predates the release of Stable Diffusion.

Luckily, it knows what text-to-image models and DALL·E are (You can verify). So we can piggy-back on them in our prompt engineering.

How to use ChatGPT

To start using ChatGPT, go to chat.openai.com. Type a question in the input box at the bottom to start a conversation.

ChatGPT is aware of the history of your current conversation. Press “+ New Chat” button on the left panel to start a new conversation.

Asking for prompts

I am going to test ChatGPT by asking prompts for illustration portrait, one of the hardest to get right. Let’s see how it does.

Close-up illustration

There are three important techniques to tease out high-quality prompts for Stable Diffusion from ChatGPT:

  • Specify image style. You need to ask for a specific kind of image. In my example, I will ask for “photorealistic close-up illustration”.

  • Fix the subject. I will fix the subject as the actress “Ana de Armas”. I am not (too) obsessed with Ana de Armas. Reason for doing this is actress name is a strong keyword. Not only will it change the face (intended) but it will also heavily influence the style (not intended). If the subject is not fixed, ChatGPT will use a different actress in each prompt. The style and composition can be changed simply due to that.

  • Prime with examples. Finally, give two example prompts.

Putting these together, the ChatGPT prompt is

Examples of high quality prompt for stunning close-up photorealistic illustration of Ana de Armas for text-to-image models (Stable Diffusion, midjourney or Dalle2) are

– portrait of beautiful happy young ana de armas, ethereal, realistic anime, trending on pixiv, detailed, clean lines, sharp lines, crisp lines, award winning illustration, masterpiece, 4k, eugene de blaas and ross tran, vibrant color scheme, intricately detailed

– alberto seveso and geo2099 style, A highly detailed and hyper realistic portrait of a gorgeous young ana de armas, lisa frank, trending on artstation, butterflies, floral, sharp focus, studio photo, intricate details, highly detailed, by Tvera and wlop and artgerm

Give me more examples.

ChatGPT’s response:

  • A stunning close-up illustration of Ana de Armas in a dramatic, dark and moody style, inspired by the work of Simon Stålenhag, with intricate details and a sense of mystery.

  • A photorealistic, highly detailed illustration of Ana de Armas in a vintage Hollywood style, reminiscent of the golden age of cinema, with a focus on glamour and elegance.

  • A close-up illustration of Ana de Armas in a neo-noir style, reminiscent of the film noir genre, with a focus on shadows, contrasts, and a sense of danger.

Let’s put these prompts in Stable Diffusion v1.5. I will use 25 steps of DPM++ 2M Karras sampler, CFG scale 7, image size 512×704. I will use the following universal negative prompt.

ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, extra limbs, disfigured, deformed, body out of frame, bad anatomy, watermark, signature, cut off, low contrast, underexposed, overexposed, bad art, beginner, amateur, distorted face

ChatGPT Prompt #1: A stunning close-up illustration of Ana de Armas in a dramatic, dark and moody style, inspired by the work of Simon Stålenhag, with intricate details and a sense of mystery.

ChatGPT Prompt #2: A photorealistic, highly detailed illustration of Ana de Armas in a vintage Hollywood style, reminiscent of the golden age of cinema, with a focus on glamour and elegance.

ChatGPT Prompt #3: A close-up illustration of Ana de Armas in a neo-noir style, reminiscent of the film noir genre, with a focus on shadows, contrasts, and a sense of danger.

These are all good stuffs! I am pleasantly surprised by how creative the prompts are, and how diverse the styles it gives.

And I surely learned some new keywords that I am going to put in the prompt generator.

Full-body illustration

Now let’s refine the ChatGPT prompt to tease out prompts for full-body portrait. All you need to do is to add “full-body” in your question and prompt examples.

Examples of high quality prompt for stunning photorealistic full body illustration of ana de armas for text-to-image models (Stable Diffusion, midjourney or Dalle2) are

full body portrait of beautiful happy young ana de armas, ethereal, realistic anime, trending on pixiv, detailed, clean lines, sharp lines, crisp lines, award winning illustration, masterpiece, 4k, eugene de blaas and ross tran, vibrant color scheme, intricately detailed

full body portrait of a gorgeous young ana de armas, A highly detailed and hyper realistic lisa frank, trending on artstation, butterflies, floral, sharp focus, studio photo, intricate details, highly detailed, by Tvera and wlop and artgerm, alberto seveso and geo2099 style,

Give me more examples

There are the prompts ChatGPT generated.

ChatGPT Prompt #4: Full body illustration of Ana de Armas in a stunning red dress, posing elegantly against a beautiful sunset background, with realistic lighting and shadows, by a master digital artist known for their photorealistic style, such as Jomaro Kindred or Gleb Alexandrov

ChatGPT Prompt #5: A full body illustration of Ana de Armas as a powerful warrior queen, with intricate armor and weapons, in a dynamic and action-packed pose, rendered in a highly detailed and photorealistic style by top fantasy artists such as Simon Stålenhag or Jora Vision

ChatGPT Prompt #6: A full body illustration of Ana de Armas as a sultry and mysterious sorceress, with detailed and realistic clothing and accessories, in a dramatic and captivating pose, by leading fantasy and sci-fi artists such as Yoshitaka Amano or Dave Rapoza.

These are good, stylish full-body portraits!

Ask for more

If you like a particular prompt ChatGPT gave, you can ask for more by something like

I like the third one. Can you give me more examples like that?

ChatGPT will give you more variants of the same style.

Summary

Can ChatGPT generate high-quality prompts for Stable Diffusion? The answer is a solid YES. But you need to follow these three guidelines: (1) Specify the image style, (2) fix the subject, and (3) prime with examples.

ChatGPT is still free to use as of time of writing. Go generate some prompts before they start charging!

Reprinted from View Original
Collection Navigation
Collection
# 1 [SD Advanced Tutoria]How to fix parameters?
# 2 [SD Advanced Tutorial]How to automatically fix faces and hands
# 3 [SD Advanced Tutorial]How to run SDXL models
# 4 [SD Advanced Tutorial]Speed up Stable Diffusion
# 5 [SD Advanced Tutorial]3 ways to control lighting in Stable Diffusion
# 6 [SD Advanced Tutorial]Video to video with Stable Diffusion
# 7 [SD Advanced Tutorial]3 methods to upscale images in SD
# 8 [SD Advanced Tutorial]Control image composition in SD
# 9 [SD Advanced Tutorial]How to generate animals in SD
# 10 [SD Advanced Tutorial]How to make a video with SD
# 11 [SD Advanced Tutorial]How to generate realistic people in SD
# 12 [SD Advanced Tutorial]Stable Diffusion SDXL Beta Model
# 13 [SD Advanced Tutorial]SD Samplers: A Comprehensive Guide
# 14 [SD Advanced Tutorial]Midjourney vs SD: Which one should you pick?
# 15 [SD Advanced Tutorial]What is hypernetwork
# 16 [SD Advanced Tutorial]AUTOMATIC1111: A Beginner’s Guide
# 17 [SD Advanced Tutoria]What are LoRA models and how to use them in AUTOMATIC1111
# 18 [SD Advanced Tutoria]ControlNet v1.1: A complete guide
# 19 [SD Advanced Tutoria]How to remove extra limbs with Stable Diffusion inpainting
# 20 [SD Advanced Tutoria]Stable Diffusion prompt: a definitive guide
# 21 [SD Advanced Tutoria]Instruct Pix2Pix: Edit and stylize photos with text
# 22 [SD Advanced Tutoria]How to use outpainting to extend images
# 23 [SD Advanced Tutoria]ChatGPT: How to generate prompts for Stable Diffusion
# 24 [SD Advanced Tutoria]How to run Stable Diffusion on Google Colab (AUTOMATIC1111)
# 25 [SD Advanced Tutoria]Stable Diffusion Installation Tutorial (Mac M1/M2)
# 26 [SD Advanced Tutoria]How to use negative prompts?
# 27 [SD Advanced Tutoria]How does negative prompt work?
# 28 [SD Advanced Tutoria]How does Stable Diffusion work?
# 29 [SD Advanced Tutoria]Stable Diffusion Workflow
# 30 [SD Advanced Tutoria]Depth-to-image in SD 2
# 31 [SD Advanced Tutoria]How to install SD on Windows
# 32 [SD Advanced Tutoria]How to use embeddings in Stable Diffusion
# 33 [SD Advanced Tutoria]How to install Stable Diffusion 2.1 in AUTOMATIC1111 GUI
# 34 [SD Advanced Tutoria]How to cartoonize photo with Stable Diffusion
# 35 [SD Advanced Tutoria]How to use Dreambooth to put anything in Stable Diffusion (Colab notebook)
# 36 [SD Advanced Tutoria]How to use VAE to improve eyes and faces
# 37 [SD Advanced Tutoria]Turn amateur into professional with img2img
# 38 [SD Advanced Tutoria]How to use AI image upscaler to improve details
# 39 [SD Advanced Tutoria]How to run Stable Diffusion 2.0 and a first look
# 40 [SD Advanced Tutoria]A Beginner's Guide to SD Models
# 41 [SD Advanced Tutoria]How to stylize images using Stable Diffusion AI
# 42 [SD Advanced Tutoria]Basic information of Stable Diffusion
# 43 [SD Advanced Tutoria]Top Free Stable Diffusion AI Image Generator Sites
# 44 [SD Advanced Tutoria]Image AI generates fashion ideas
# 45 [SD Advanced Tutoria]Make an Animated GIF Using SD
# 46 [SD Advanced Tutoria]Common Problems in AI
# 47 [SD Advanced Tutoria]Change prompt word parameters
# 48 [SD Advanced Tutoria]AI removes unwanted objects
# 49 [SD Advanced Tutoria]Fine-tune AI images with tips
# 50 [SD Advanced Tutoria]What is Stable Diffusion?
 

Comments

no dataCoffee time! Feel free to comment