Text to Image Workflow

Explore the text-to-image workflow in SeaArt's ComfyUI, from adding nodes like KSampler and LoRA to setting parameters and generating stunning images based on your text prompts.

1. Understanding the Text to Image Workflow

start by clicking on ComfyUI, create a new ComfyUI workflow

add the basic workflow for Text to Image

Process of text-to-image workflow

The workflow in ComfyUI is similar to that in Web UI:

select a model → enter prompt → set parameters → generate image

Parameters

Text to Image Workflow - KSampler parameters

control_after_generate: control over seed generation

fixed: fixing the seed

increment: adding 1 from the existing seed

decrement: subtracting 1 from the existing seed

randomize: random seed

Text to Image Workflow - scheduler parameters

scheduler: usually choose between normal or karras denoise: which indicates the strength of noise reduction in image generation, the higher the value, the greater the impact and change on the image

Text to Image Workflow - image size settings

size recommendation:

SD1.5: 512512

SDXL: 10241024

2. How to add a node?

Add KSampler

First, add a core node: the KSampler

  • Right-click: Add Node → sampling → KSampler

Steps for adding KSampler

Pull out the node

then pull out the sampler node and add the corresponding nodes

Text to Image Workflow - pull out the sampler node and add the corresponding nodes

model→CheckpointLoaderSimple

positive→CLIPTextEncode

negative→CLIPTextEncode

latent_image→EmptyLatentImage

LATENT→VAEDecode

IMAGE→SaveImage

Text to Image Workflow - Steps of pulling out the node

How to add LoRA?

Loaders: mainly used for loading the diffusion model, including model and LoRA.

Add Node→loaders→Load LoRA

Steps to add LoRA

How to add Clip Skip?

Conditioning: guides the diffusion model to generate specific outputs, including Prompt, ControINet, Clip Skip, etc.

It is recommended to add Clip Skip to control the layers skipped, which helps adjust the final image details.

Add Node→conditioning→CLIP Set Last Layer

Steps to add Clip Skip

Connect nodes

After adding nodes, many nodes might not be connected yet. They need to be connected in order according to the sequence and matching colors.

Checkpoint→LoRA→Clip Skip→Prompt→KSampler、Empty Latent Image→VAE Decode→Save Image

Process to connect nodes
Steps to connect nodes

3. Starting the Text to Image

Choose a Checkpoint

Text to image workflow - select a model

Select LoRA and adjust the weights

Text to image workflow - select LoRA and adjust the weights

Set Clip Skip, typically choosing to set it to -2

Text to image workflow - set Clip Skip

Enter prompt

Text to image workflow - enter prompts

Set relevant parameters

Since the SDXL model is chosen here, set the sampling steps to around 40, and image size to 1024*1024

Text to image workflow - set relevant parameters

Click Generate

Text to image workflow - click the Generate button

View the results

View the results in History

Last updated