SeaArt Guide
How to SeaArt AI - Office Tutorial
How to SeaArt AI - Office Tutorial
  • ✨How to Use an AI Image Generator
  • ✨1-SeaArt AI Basic Page
  • ✨2-SeaArt AI Basic Function
    • 2-1 Text to Image
    • 2-2 Image to Image
    • 2-3 ControlNet
    • 2-4 AI Apps
      • How to publish as App
      • Swift AI Apps
    • 2-5 AI Characters
      • How to create your own character?
      • Character Description Writing Tips
      • Conversation Tips
    • 2-6 Models
    • 2-7 Post
    • 2-8 AI Video Generation
      • Txt2Vid
      • Img2Vid
      • Camera Movement
      • Start and End Frames
    • 2-9 AI Audio
    • 2-10 Workflow
      • Text to Image Workflow
      • IMG2IMG+Partial Repainting
      • Core Nodes
      • Tips
    • 2-11 Canvas
    • 2-12 LoRA Training
      • Flux Lora Training
  • ✨3-Advanced Guide
    • 3-1 Principles of AI art
    • 3-2 LoRA Training (Advance)
      • How To Create Dataset For Training
    • 3-3 Workflow Guide
      • Image Conversion
      • Inpainting
    • 3-4 Canvas Guide
    • Composite Poster
  • ✨4-Parameters
    • 4-1 Model
    • 4-2 Mode
    • 4-3 Basic Settings
    • 4-4 Advanced Config
    • 4-5 Advanced Repair
    • 4-6 Complete Prompting Guide
    • 4-7 Prompt Edit | Keyword Blending Guide
  • ✨5-Practical Examples
    • AI Influencer
    • LOGO Design
    • E-commerce Poster
    • How To Make Multiple Characters
    • Prompt Templates
    • How to Maintain Character Consistency
    • Useful Prompts
  • ✨6-Permanent Events
    • SeaArt.AI Creator Incentive Program
      • Creator Incentive Program FAQ
    • High-Quality Models Recommendation
      • SeaArt Infinity
      • Stable Diffusion 3.5
      • SeaArt Realism
      • NOOBAI XL
      • T-Ponynai3 V6
      • Counterfeit V3.0
      • Temporal Paradox Mix
    • High-Quality AI Apps Recommendation
    • High-Quality Character Recommendation
    • High-Quality Workflow Recommendation
  • ✨7-FAQ
Powered by GitBook
On this page
  • Basic Image to Image
  • Partial Repainting
  1. 2-SeaArt AI Basic Function
  2. 2-10 Workflow

IMG2IMG+Partial Repainting

Learn about ComfyUI's image-to-image workflow and four powerful partial repainting methods: VAE Encode, Set Latent Noise Mask, ControlNet Inpaint, and CLIPSeg.

PreviousText to Image WorkflowNextCore Nodes

Last updated 9 months ago

Basic Image to Image

Img to Img can be adjusted based on , adding the "Load Image" and "VAE Encode" nodes.

Since the input image is just a pixel image, it can't be directly placed into the latent space. Therefore, it requires a VAE encoder to encode the image so that the latent space can recognize it. Here, the final generated image size is consistent with the original image.

Empty Latent Image: The previous Text to Image must denoise through Empty Latent Image before generating a new image. Now that an image has been added, the Empty Latent Image is no longer needed.

Workflow: Upload an Image → Select Models → Enter Prompts → Adjust Parameters → Generate.

Parameters:

denoise: Equivalent to Denoising Strength, can be adjusted between 0-1.

Using Img2Img allows for changing styles, repairing images, extending images, high-definition restoration, etc.

Pre-processing Image

You can scale or crop the image by adding different nodes, or choose not to add any.

Upscale Image/Upscale Image By: Upscale the image.

ImageCrop: Crop the image.

Partial Repainting

Four Methods: VAE Encode (for Inpainting), Set Latent Noise Mask, ControlNet Inpaint, and CLIPSeg.

  1. VAE Encode (for Inpainting)

Add VAE Encode (for Inpainting), connect Mask, right-click on the image, and select Open in MaskEditor to draw a mask. If there are issues with a section of the drawn mask, you can erase it by holding down the right mouse button.

Workflow: Choose a model similar to the original image, and enter prompts for the masked part.

VAE Encode (for Inpainting): Equivalent to repainting, with higher randomness, and the original masked area will not be preserved.

  1. Set Latent Noise Mask

First, encode the image through VAE to turn it into content recognizable by the latent space, then regenerate the masked area as noise content.

Set Latent Noise Mask: It will refer to the original image for repainting, ensuring a better understanding of the generated content, with a lower probability of generating incorrect images, thus suitable for fine-tuning while maintaining similarity to the original image.

  1. ControlNet Inpaint

Add the ControlNet, select the Inpaint model, and preprocess the image accordingly (Inpaint Preprocessor).

Note: Don't forget to add the VAE encoder so the image can enter the latent space.

  1. CLIPSeg

Enter prompts to automatically divide the mask areas, eliminating the need for manual paint-over. It can be used together with Set Latent Noise Mask.

Parameters:

text: Input the area you want to repaint.

threshold: The precision level of content recognition.

dilation_factor: The diffusion degree of content recognition.

Output:

Heatmap Mask: Heatmap image.

BW Mask: Black and white image.

You can preview the recognized mask areas separately.

Differences between the four repainting methods:

1. VAE Encode (for Inpainting): Equivalent to erasing and repainting with higher randomness, suitable for generation from scratch.

2. Set Latent Noise Mask: It will refer to the original image for repainting, ensuring a certain similarity to the original image, suitable for fine-tuning.

3. ControlNet Inpaint: Relatively stable and refined.

4. CLIPSeg: Automatically recognizes the mask areas, so there's no need for manual paint-over, making it more convenient.

✨
Text to Image
Image to Image workflow panel
Image to image workflow - pre-processing Image
Partial Repainting workflow - VAE Encode
Partial Repainting workflow - draw a mask
Partial Repainting workflow - parameters panel
Partial Repainting - Set Latent Noise Mask
Partial Repainting - ControlNet Inpaint
Partial Repainting - CLIPSeg