inference.minimax.h3.chili.img2vid.v0

View markdown

The inference.minimax.h3.chili.img2vid.v0 job animates an uploaded image into a 24 fps video with synchronized stereo audio.

Upload the starting image as a job input and set first_frame to its filename. You can also upload an ending image and name it in last_frame to guide the closing frame. If these fields are omitted, the first upload is the starting frame and the second upload, when present, is the ending frame.

{
    "type": "inference.minimax.h3.chili.img2vid.v0",
    "config": {
        "prompt": "the scene comes to life, camera slowly pushing in",
        "first_frame": "input.jpg",
        "duration": 8
    }
}

Prompting

The prompt is used exactly as written: nothing rewrites or expands it before it reaches the model, which was trained on descriptive captions. Describe the shot rather than making a request: the subject, setting, time of day, weather, lighting, camera move and motion. For example, “show me what christmas would be like in Tokyo” tends to produce a generic winter scene, while “Shibuya crossing in Tokyo on Christmas Eve at night, light snow falling, crowds with umbrellas, a giant illuminated Christmas tree, neon signs reflecting in the wet street, slow cinematic dolly shot” produces the intended scene. The input image already fixes the subject and composition, so use the prompt to describe the motion and change you want from it. This job type runs a distilled 8-evaluation recipe, so it follows the prompt less closely than full-step generation; vague prompts steer it even less, so a detailed description matters more here.

Structured prompt format

MiniMax trains H3 on a structured prompt, not on free text. Sending that structure yourself is the most reliable way to control the soundtrack: on a 24-prompt bank rendered on this tier, 10 of 24 free-text prompts came back with an unrequested music bed, and 0 of 24 did when the same prompts were sent in this format (same seeds, same duration). The format adds three things free text usually leaves to the model’s own judgement: an explicit soundscape, an explicit music decision, and an explicit statement of whether anyone speaks.

Rules that apply to every section below:

A line is spoken only when it is written as dialogue: she (S1) says softly: <d>[English] Don't tell me you forgot again.</d>. (S1) numbers the speaker, <d>…</d> encloses the spoken words, and the language tag is always [English] (the model voices other languages inside the tag as written). Put dialogue last in the shot: prose after a spoken line tends to be voiced too.
When nobody speaks, say so: She does not speak., He does not speak. or, for a scene with no single subject, No spoken dialogue. Without this sentence the model may invent speech.
overall_soundscape: is one sentence of diegetic sound, i.e. what the scene itself produces. For a person: Soft room tone, her quiet breathing and the faint rustle of fabric as she moves. For a scene: Natural ambient sound matching the scene and its visible actions.
non_diegetic_music: is N/A unless you want a score, in which case describe it in one sentence, e.g. Calm ambient piano plays throughout.
Separate sections with a blank line. The whole prompt, format included, must stay within the 7000-character limit.
Write the visual description exactly as the Prompting section above recommends; the format wraps it, it does not replace it.

Beyond the soundtrack, the same format carries the rest of what the model was trained to read:

Write the description as a timeline, not a caption: what is visible and audible first, then the actions, reactions and state changes in the order they happen. Every detail should be something that can be seen or heard; requests like “make it cinematic” steer little.
Give camera movement as type, amplitude and speed: “the camera pushes in with small amplitude at slow speed”, “trucks right slowly”, “holds a locked-off shot”.
For more than one shot, keep [Shot 1] without a timestamp and start each later shot with a strictly increasing cut time inside the duration: [Shot 2] At 00:05.000, the camera cuts to a close-up of steam rising from the sliced bread.
Keep a speaker’s id across shots. For narration that is not spoken on screen, use the exact phrase “says in an off-screen voiceover” and add that the on-screen character’s lips remain closed. When a line crosses a cut, mark both parts with <scenetrans> and say the audio continues across the cut; when the clip ends mid-sentence, mark the line with <cutoff>.
When you do want a score, describe it as instrumentation, tempo and dynamics (“sparse piano at a slow tempo, joined by low strings that build and fade”) rather than mood words.
Text that must appear on screen is spelled out verbatim in the prompt.
An unspecified clip gets natural ambience; write overall_soundscape: N/A only when you want silence.

Image-to-video adds one line before the three sections that binds the uploaded first frame to the shot as <Picture 1>, and the shot opens by saying the frame’s appearance, setting and framing are kept. Describe the motion and change you want from the frame; the picture already fixes the subject.

For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.

integrated_multimodal_description: [Shot 1] The subject shown in <Picture 1> is the first frame; its appearance, setting and framing are kept. She hugs her knees tighter, turns her head toward the lens and lets a slow, warm smile spread. She does not speak.

overall_soundscape: Soft room tone, her quiet breathing and the faint rustle of fabric as she moves.

non_diegetic_music: N/A

When last_frame is also uploaded, it is <Picture 2>; mention it the same way if the ending should match it.

prompt is required and must contain 3–7000 characters. Describe both the visual scene and the sounds you want to hear.

duration accepts whole seconds from 4 to 15 and defaults to 6. The actual video may be slightly longer than requested: 4 seconds produces about 4.46 seconds, and 15 seconds produces about 15.08 seconds.

resolution supports only 768P, which is also the default. Exact output dimensions depend on the aspect ratio. seed accepts an integer from 0 to 2147483647 for reproducible generation; omit it to use a random seed.

The output aspect ratio follows the starting image. aspect_ratio accepts only adaptive, which is also the default.

Schema

{
  "type": "object",
  "required": [
    "type",
    "config"
  ],
  "additionalProperties": false,
  "properties": {
    "type": {
      "type": "string",
      "enum": [
        "inference.minimax.h3.base.img2vid.v0",
        "inference.minimax.h3.fast.img2vid.v0",
        "inference.minimax.h3.base.img2vid.v1",
        "inference.minimax.h3.fast.img2vid.v1",
        "inference.minimax.h3.turbo.img2vid.v0",
        "inference.minimax.h3.hyper.img2vid.v0",
        "inference.minimax.h3.chili.img2vid.v0"
      ]
    },
    "config": {
      "type": "object",
      "required": [
        "prompt"
      ],
      "additionalProperties": false,
      "properties": {
        "first_frame": {
          "type": "string",
          "format": "filename",
          "example": "first_frame.png",
          "description": "Set the starting frame for the video. Optional: defaults to the first uploaded input."
        },
        "last_frame": {
          "type": "string",
          "format": "filename",
          "example": "last_frame.png",
          "description": "Set the ending frame for the video. Optional: defaults to the second uploaded input, when present."
        },
        "prompt": {
          "type": "string",
          "minLength": 3,
          "maxLength": 7000,
          "default": "a dancing cat under moonlight",
          "description": "Description of the desired video and audio."
        },
        "duration": {
          "type": "integer",
          "minimum": 4,
          "maximum": 15,
          "default": 6,
          "description": "Duration of the video in seconds. Rounded up to the model's frame grid (4 s returns 4.46 s, 15 s returns 15.08 s)."
        },
        "aspect_ratio": {
          "type": "string",
          "enum": [
            "adaptive"
          ],
          "default": "adaptive",
          "description": "The aspect ratio always derives from the input image, so only \"adaptive\" is accepted."
        },
        "resolution": {
          "type": "string",
          "enum": [
            "768P"
          ],
          "default": "768P",
          "description": "Only 768P is supported: the canvas targets a 768px short edge with a 768x1344 pixel area cap and 32px rounding; exact width and height follow the aspect ratio of the input image."
        },
        "seed": {
          "type": "integer",
          "minimum": 0,
          "maximum": 2147483647,
          "description": "Seed for reproducible generation. Random when omitted."
        }
      }
    }
  }
}