[SD Advanced Tutoria]How to cartoonize photo with Stable Diffusion

[SD Advanced Tutoria]How to cartoonize photo with Stable Diffusion

AI  Learning Assistant No 1
AI Learning Assistant No 1
August 21st, 2023

Software setup

We will use AUTOMATIC1111 GUI for this tutorial.

For the purpose of demonstration, we will use the following cartoon-style models. The technique will work for any custom models with unique styles. (That’s why they made custom models, right?)

Anything v3 – anime style model.

Inkpunk Diffusion – Japanese illustration style. (keyword: nvinkpunk)

Mo-Di Diffusion – Modern Disney style. (keywords: modern disney style)

Follow the model links above to download the model. Check out this article to learn how to install and use custom models.

Note for using Quick Start Guide

These 3 models are available in Colab Notebook in the Quick Start Guide.

If you are using the Colab Notebook in the Quick Start Guide, make sure those models are selected. Run the cell to start AUTOMATIC1111 GUI.

Cartoonize – step by step

I will cartoonize a photo of my friend I met in the latent space. You can find her other friends here.

We will use Inkpunk Diffusion as our cartoon model. See Software section for set up instructions.

In AUTOMATIC1111 GUI, select the Inpunk Diffusion model in the Stable Diffusion checkpoint dropdown menu.

Switch to img2img tab by clicking img2img. Upload the photo you want to be cartoonized to the canvas in the img2img sub-tab.

Put in a prompt describing your photo. You also need to include the special keywords that trigger the style, if any. For Inkpunk Diffusion, it is nvinkpunk. So my prompt is

nvinkpunk A woman sitting outside

Negative prompt is optional. Out of habit (or superstition), I did use one:

disfigured, ugly

Set either width or height to 512 and adjust the other to closely match the aspect ratio of your photo. Since my photo is a portrait, I set width to 512 and height to 704. Select Crop and resize to get rid of the part that doesn’t fit.

Set sampling steps to 30 to get a good-quality image.

Set batch size to 4 so that you will have a few images to choose from. (Cherry picking is an important skill for using Stable Diffusion)

The setting you will need to play with is denoising strength. Carefully adjust this setting to get the amount of change you want. Nothing would change when it is set to 0. Your original image would be ignored when you set it to 1. Around 0.4 is a good starting point.

Press Generate to produce images.

You will see something like below when you use different values of denoising strength. Pick the value to get the effect you want to achieve!

Multiple img2img steps

One trick I often use is doing multiple rounds of img2img to fill in finer details.

Remember low value of img2img produces small change? You can do a step of big denoising strength followed by a series of small values. Below is an example.

Using multiple rounds of img2img, you can stylize local areas without changing the overall composition.

Final products

Here are the images cartoonized by the three custom models.

These are, of course, just one example from each model. You can control how strong the effect you want to your taste.



Anything v3

Next steps

If you want to use the cartoonized character for some other creative works, you can remove the background by using one of those AI background removal sites, e.g. remove.bg.

Perhaps you can make a comic strip by asking your friends to act out the scenes, and cartoonize them.

You can combine this technique with inpainting if you only want part of the image changed. Or different parts of images stylized with different models.

The possibility is endless here.

Reprinted from https://stable-diffusion-art.com/cartoonize-photo/View Original
Collection Navigation
# 1 [SD Advanced Tutoria]How to fix parameters?
# 2 [SD Advanced Tutorial]How to automatically fix faces and hands
# 3 [SD Advanced Tutorial]How to run SDXL models
# 4 [SD Advanced Tutorial]Speed up Stable Diffusion
# 5 [SD Advanced Tutorial]3 ways to control lighting in Stable Diffusion
# 6 [SD Advanced Tutorial]Video to video with Stable Diffusion
# 7 [SD Advanced Tutorial]3 methods to upscale images in SD
# 8 [SD Advanced Tutorial]Control image composition in SD
# 9 [SD Advanced Tutorial]How to generate animals in SD
# 10 [SD Advanced Tutorial]How to make a video with SD
# 11 [SD Advanced Tutorial]How to generate realistic people in SD
# 12 [SD Advanced Tutorial]Stable Diffusion SDXL Beta Model
# 13 [SD Advanced Tutorial]SD Samplers: A Comprehensive Guide
# 14 [SD Advanced Tutorial]Midjourney vs SD: Which one should you pick?
# 15 [SD Advanced Tutorial]What is hypernetwork
# 16 [SD Advanced Tutorial]AUTOMATIC1111: A Beginner’s Guide
# 17 [SD Advanced Tutoria]What are LoRA models and how to use them in AUTOMATIC1111
# 18 [SD Advanced Tutoria]ControlNet v1.1: A complete guide
# 19 [SD Advanced Tutoria]How to remove extra limbs with Stable Diffusion inpainting
# 20 [SD Advanced Tutoria]Stable Diffusion prompt: a definitive guide
# 21 [SD Advanced Tutoria]Instruct Pix2Pix: Edit and stylize photos with text
# 22 [SD Advanced Tutoria]How to use outpainting to extend images
# 23 [SD Advanced Tutoria]ChatGPT: How to generate prompts for Stable Diffusion
# 24 [SD Advanced Tutoria]How to run Stable Diffusion on Google Colab (AUTOMATIC1111)
# 25 [SD Advanced Tutoria]Stable Diffusion Installation Tutorial (Mac M1/M2)
# 26 [SD Advanced Tutoria]How to use negative prompts?
# 27 [SD Advanced Tutoria]How does negative prompt work?
# 28 [SD Advanced Tutoria]How does Stable Diffusion work?
# 29 [SD Advanced Tutoria]Stable Diffusion Workflow
# 30 [SD Advanced Tutoria]Depth-to-image in SD 2
# 31 [SD Advanced Tutoria]How to install SD on Windows
# 32 [SD Advanced Tutoria]How to use embeddings in Stable Diffusion
# 33 [SD Advanced Tutoria]How to install Stable Diffusion 2.1 in AUTOMATIC1111 GUI
# 34 [SD Advanced Tutoria]How to cartoonize photo with Stable Diffusion
# 35 [SD Advanced Tutoria]How to use Dreambooth to put anything in Stable Diffusion (Colab notebook)
# 36 [SD Advanced Tutoria]How to use VAE to improve eyes and faces
# 37 [SD Advanced Tutoria]Turn amateur into professional with img2img
# 38 [SD Advanced Tutoria]How to use AI image upscaler to improve details
# 39 [SD Advanced Tutoria]How to run Stable Diffusion 2.0 and a first look
# 40 [SD Advanced Tutoria]A Beginner's Guide to SD Models
# 41 [SD Advanced Tutoria]How to stylize images using Stable Diffusion AI
# 42 [SD Advanced Tutoria]Basic information of Stable Diffusion
# 43 [SD Advanced Tutoria]Top Free Stable Diffusion AI Image Generator Sites
# 44 [SD Advanced Tutoria]Image AI generates fashion ideas
# 45 [SD Advanced Tutoria]Make an Animated GIF Using SD
# 46 [SD Advanced Tutoria]Common Problems in AI
# 47 [SD Advanced Tutoria]Change prompt word parameters
# 48 [SD Advanced Tutoria]AI removes unwanted objects
# 49 [SD Advanced Tutoria]Fine-tune AI images with tips
# 50 [SD Advanced Tutoria]What is Stable Diffusion?


no dataCoffee time! Feel free to comment