HomeAI News
[Beginner's Guide] How to use img2img to turn an amateur drawing to professional with Stable Diffusion (Image-to-image)
350

[Beginner's Guide] How to use img2img to turn an amateur drawing to professional with Stable Diffusion (Image-to-image)

AI  Learning Assistant No 1
AI Learning Assistant No 1
July 28th, 2023

Not a born-artist? Stable Diffusion can help. Img2img (image-to-image) can improve your drawing while keeping the color and composition.

What is image-to-image?

Image-to-image (img2img for short) is a method to generate new AI images from an input image and text prompt. The output image will follow the color and composition of the input image.

The input image is just a guide. It does not need to be pretty or have any details. The important part is the color and the composition.

The prompt requirement is the same as text-to-image.

Software setup

We will use AUTOMATIC1111 GUI for this tutorial.

Enable color sketch tool

The color sketch tool is not turned on by default. Using it makes img2img a lot easier.

To enable the color sketch feature in AUTOMATC1111 GUI, add the following argument option when you run

webui.py.

--gradio-img2img-tool color-sketch Alternatively, on Windows system, edit the file webui-user.bat under stable-diffusion-webui folder. Change the line

set COMMANDLINE_ARGS=

to

set COMMANDLINE_ARGS=--gradio-img2img-tool color-sketch

If there’s already something after the equal sign, add --gradio-img2img-tool color-sketch to the end (Make sure there’s a space in the beginning).

Double click the file to start the GUI.

The option is already enabled in Colab Notebook in the Quick Start Guide.

Using color sketch tool

To use the color sketch tool, go to img2img tab, upload a starting image to the canvas. Click on the color palette icon and the then the solid color button. The color sketch tool should appear like below.

Now you can draw in color!

The color picker tool is especially useful when making small modifications.

Color sketch tool

Transform an amateur drawing to professional

I just saw an Youtube video of a professional artist drawing a realistic apple with a black background on iPad. Thought this would be a good subject to show how to use img2img.

Step 1: Create the background

You can use this black or white background. They are both 512×512 pixels, the same as the default image size of Stable Diffusion v1.5.

Upload the background to the canvas.

Drawing canvas with background uploaded.

Step 2: Draw an apple

Let’s draw the apple with the color palette tool.

Don’t spend too much time on what you draw. Just aim at getting the color, shape and composition in the right neighborhood.

This is the apple I drew. (The little light green strips are water drops… just so you know…)

My amateur drawing of an apple.

Step 3: img2img

In Stable Diffusion checkpoint dropbox, select v1-5-pruned-emaonly.ckpt for using v1.5 model. (You can also experiment with other models.)

Come up with a prompt that describe your final picture as accurately as possible. photo of perfect green apple with stem, water droplets, dramatic lighting Put this in the prompt text box.

img2img settings.

Set image width and height to 512.

Set sampling steps to 20 and sampling method to DPM++ 2M Karras.

Set batch size to 4 so that you can cherry pick the best one.

Set seed to -1 (random).

The two parameters you want to play with are CFG scale and denoising strength. In the beginning, you can set CFG scale to 11 and denoising strength to 0.75.

Hit Generate to get a set of four new images.

Increase denoising strength if you want the images to change more. Decrease if you want them to be closer to your original drawing.

img2img

Once you are happy with what you get, save the image.

Step 4: Second img2img

You can stop here if you are happy with the result. But doing one more round of img2img adds more details and allow you to use a different prompt.

Upload the image you just generated.

For mine, I feel the stem is a bit too dark so I paint it a bit lighter. (I used the color picker tool to get the color of the stem, and dial up the color values.)

I simply reused the same settings, including the prompt.

2nd round of img2img.

In the new batch, I get something I like:

Final drawing.

It has a lot more realistic details and better lighting. Doing a second round of img2img adds complexity to the scene.

Next steps

img2img is a versatile technique to control the composition and color of the image. It provides additional control to text-to-image.

In the next tutorials, I will walk you through a few more techniques to use with img2img. Stay tuned!

To see more content about Stable Diffusion from zero click:https://www.hayo.com/article/64c21001ef669957a0d21e63

Reprinted from View Original
Collection Navigation
Collection
# 1 [Beginner's Guide] Stable Diffusion Absolute beginner’s guide (+online demo)
# 2 [Beginner's Guide] How to use img2img to turn an amateur drawing to professional with Stable Diffusion (Image-to-image)
# 3 [Stable Diffusion Repair] Beginner’s guide to inpainting (step-by-step examples)
# 4 [Beginner's Guide] How to remove undesirable objects with AI inpainting
# 5 [Beginner's Guide] Video to video with Stable Diffusion (step-by-step)
# 6 [Beginner's Guide] How do you use Stable Diffusion AI?
# 7 [Beginner's Guide] Top Free Stable Diffusion AI image generator sites
# 8 [Beginner's Guide] Stable Diffusion prompt: a definitive guide
# 9 [Beginner's Guide] Instruct Pix2Pix: Edit and stylize photos with text
# 10 【Beginner's Guide to Stable Diffusion】How to build a good prompt?
# 11 【Beginner's Guide to Stable Diffusion】Rules of thumb for building good prompts
# 12 【Beginner's Guide to Stable Diffusion】What is ChatGPT?
# 13 【Beginner's Guide to Stable Diffusion】Sorting out high-quality prompt words from ChatGPT
# 14 【Beginner's Guide to Stable Diffusion】How to use negative prompts?
# 15 【Beginner's Guide to Stable Diffusion】Negative prompt is important for v2 models
# 16 【Beginner's Guide to Stable Diffusion】Boilerplate negative prompts in v2 model
# 17 【Beginner's Guide to Stable Diffusion】What are those parameters, and should I change them?
# 18 【Beginner's Guide to Stable Diffusion】How many images should I generate?
# 19 【Beginner's Guide to Stable Diffusion】Common ways to fix defects in images
# 20 【Beginner's Guide to Stable Diffusion】Garbled faces and eyes problems
# 21 【Beginner's Guide to Stable Diffusion】How to use VAE to improve eyes and faces
# 22 [Beginner's Guide] How to install Stable Diffusion on Windows (AUTOMATIC1111)
# 23 【Beginner's Guide to Stable Diffusion】Beginner’s guide to inpainting (step-by-step examples)
# 24 【Beginner's Guide to Stable Diffusion】What are custom models?
# 25 【Beginner's Guide to Stable Diffusion】How to control image composition?
# 26 【Beginner's Guide to Stable Diffusion】Generating specific subjects
# 27 [Beginner's Guide] How to install and run Stable Diffusion on Apple Silicon M1/M2 Macs
# 28 [Beginner's Guide] Try the Stable Diffusion online demo
# 29 [Beginner's Guide] Beginner’s guide to Stable Diffusion models and the ones you should know
# 30 [Beginner's Guide] What Can Stable Diffusion Do?
# 31 [Beginner's Guide] Realistic human street portrait
# 32 [Beginner's Guide] How to generate animals in Stable Diffusion
 

Comments

no dataCoffee time! Feel free to comment