inference.sam3.segment.video.v1

The inference.sam3.segment.video.v1 job performs text-prompted video segmentation using Meta’s SAM 3 Video Predictor. A single text prompt applied at frame 0 is tracked forwards and backwards across the video and one output mp4 is returned.

Basic Usage

{
    "type": "inference.sam3.segment.video.v1",
    "config": {
        "prompt": "fish"
    }
}

This returns a single mp4 where every detected object mask is merged (per-frame pixel-wise union) into one black-and-white channel.

Configuration Options

Parameter	Type	Default	Description
prompt	string	(required)	Text describing what to segment and track across the video.
confidence_threshold	number	0.5	Minimum SAM 3 score an object must reach to be included in the merged mask or overlay (range 0.0–1.0).
mode	enum	mask	mask returns a merged B&W mp4. overlay returns the SAM 3 README-style colored overlay over the input.
alpha	number	0.5	Mask alpha used when mode is overlay. Ignored otherwise (range 0.0–1.0).

Examples

Merged B&W mask video

{
    "type": "inference.sam3.segment.video.v1",
    "config": {
        "prompt": "fish"
    }
}

Colored overlay video

{
    "type": "inference.sam3.segment.video.v1",
    "config": {
        "prompt": "fish",
        "mode": "overlay",
        "alpha": 0.5
    }
}

Only high-confidence detections

{
    "type": "inference.sam3.segment.video.v1",
    "config": {
        "prompt": "person",
        "confidence_threshold": 0.9
    }
}

Input Requirements

• Format: MP4 (video/mp4)
• Max file size: 100 MB

Input resolution and fps are preserved on the output. Common sizes (832×480, 1280×720) are warmed into the torch.compile cache on bootstrap.

Output

One MP4 file (H.264 / yuv420p) sent back as the output form field. The mp4 has the same resolution and fps as the input.

• mode=mask (default): grayscale-style, where any pixel covered by any detected mask is white (255) and everything else is black (0).
• mode=overlay: alpha-blended colored masks over the original frames, plus per-object bounding boxes and id=<N>, p=<score> labels, matching the SAM 3 README example.

Schema

{
  "type": "object",
  "required": [
    "type",
    "config"
  ],
  "additionalProperties": false,
  "properties": {
    "type": {
      "enum": [
        "inference.sam3.segment.video.v1"
      ]
    },
    "config": {
      "type": "object",
      "required": [
        "prompt"
      ],
      "additionalProperties": false,
      "properties": {
        "prompt": {
          "type": "string",
          "minLength": 1,
          "maxLength": 500,
          "description": "Text prompt describing what to track and segment across the video (e.g., 'fish', 'yellow school bus', 'person')."
        },
        "confidence_threshold": {
          "type": "number",
          "default": 0.5,
          "minimum": 0,
          "maximum": 1,
          "description": "Confidence threshold applied to per-object scores before the mask is merged or rendered."
        },
        "mode": {
          "type": "string",
          "enum": [
            "mask",
            "overlay"
          ],
          "default": "mask",
          "description": "Output style. 'mask' produces a black-and-white mp4 where white means any detected object covers that pixel. 'overlay' composites the colored masks, bounding boxes, and id/score labels from the SAM 3 visualization over the original video."
        },
        "alpha": {
          "type": "number",
          "default": 0.5,
          "minimum": 0,
          "maximum": 1,
          "description": "Mask alpha used when mode is 'overlay'. Ignored otherwise."
        }
      }
    }
  }
}