Img2Vid

In this guide, we will learn how to write effective prompts, covering how to describe the main subject, control camera dynamics, and even add more special effects to your video.

By simply uploading an image, you can convert it into a video. During this process, accurate and detailed prompts are crucial. If no prompts are provided, the AI will randomly generate video content based on the image. However, with prompts, the AI will create a video based on the combination of the image and the prompts.

Since the image already provides the basic subject and atmosphere, compared to text-to-video, image-to-video can reduce the number of prompts accordingly.

Prompt Framework

Subject + Action + Camera + Light and Atmosphere

• Subject: Objects appearing in the image, such as people, objects, environmental information, etc.

• Action: Description of the movement of the objects, such as a person running, environmental changes, spatial transformations, etc.

• Camera: Camera descriptions are not necessary; after describing the subject and action, SeaArt will generally generate a reasonable dynamic video based on the prompts. If you want to control the camera, you can add descriptions of the camera in the prompts, such as: camera zoom out, etc.

• Light and Atmosphere: Although the image already has basic lighting and atmosphere, SeaArt can still adjust the overall atmosphere of the video based on the prompts.

Tips:

1. Accurately describe the subject, and use smooth and concise sentences.

2. Ensure the completeness of the prompts, and use Prompt Refinement to expand the prompts.

3. Use short sentences for descriptions.

Prompt Analysis

Before generating a video from an image, we need to determine what kind of video we want. For example, if we want to make the dog in the image move:

Simple prompt:

A dog running on the grass.

Prompt based on the framework:

A dog is sprinting across the grass, making quick turns and leaping with its tail held high. The camera follows closely from a low angle, adjusting its focus as the dog runs, capturing each leap and running moment. The sunlight casts a warm glow on the dog's fur, while the grass is bathed in a warm light. The blue sky and white clouds gently drift in the light breeze in the background, creating a scene full of energy and the warm atmosphere of a peaceful morning.

Application examples

1. A man talking to a woman.

1. An anthropomorphic cat walking down the runway.

1. Glowing text, smoke, camera moving left.

1. A woman facing the camera.

1. A ship sailing on the sea, with a shaky camera.

1. A woman raising her hand to drink coffee.