In the field of AI drawing, DragGAN is undoubtedly a bright new star. This tool, jointly developed by scholars from MPII, MIT, Penn University, Google and other institutions, has been recognized at SIGGRAPH 2023. The emergence of DragGAN allows us to achieve precise image editing through simple drag and drop.

Recently, the open source code of DragGAN has been officially released, so that more people can experience the magic of this tool.

How to install DragGAN

First, we need to download the weight file of the pre-trained StyleGAN2:

sh scripts/download_model.sh

If you want to try the StyleGAN-Human and Landscapes HQ (LHQ) datasets, you need to download the corresponding weight files to the ./checkpoints directory. Then, we can start the DragGAN GUI to edit the image generated by GAN:

sh scripts/gui.sh

If you want to edit real images, you also need to use tools like PTI for GAN inversion, and then load the new latent code and model weights into the DragGAN GUI for editing.

Features of DragGAN

The biggest feature of DragGAN is that it can realize precise editing of images by dragging and dropping. Whether it is the shape, layout, posture, expression, position, or size of the object, it can be precisely controlled.

This is due to the two main components of DragGAN: feature-based motion monitoring and a new point tracking method.

With these two approaches, DragGAN can operate on the learned-to-generate image manifold of GANs to generate realistic outputs.

Applications of DragGAN

The application of DragGAN is very extensive. Whether it is an image of various categories such as animals, cars, humans, landscapes, etc., it can be edited by DragGAN.

Moreover, DragGAN also has a strong generalization ability, which can create images beyond the distribution of training images. For example, a huge mouth, and a wheel that is magnified to the extreme.

DragGAN experience

The experience of using DragGAN is very good. The user only needs to set a few operating points (red dots), target points (blue dots), and optionally a mask (bright area) representing the movable area.

Then, the model will automatically complete the subsequent image generation. This process continues until the operating point reaches its corresponding target point.

In general, DragGAN is a very potential AI drawing tool, and its appearance has opened a new chapter for AI drawing.