unslothai/unsloth · error · ValueError

No captioned video clips found. MiniMax-H3 trains from clips

Error message

No captioned video clips found. MiniMax-H3 trains from clips with sound, not stills: provide .mp4 / .mov / .mkv / .webm files plus a metadata.jsonl / captions.jsonl, per-clip .txt captions, or an instance prompt.

What it means

Raised by H3 clip discovery when the directory exists but zero (clip, caption) pairs were assembled. Clips only count when they carry a caption: a <stem>.txt/.caption sidecar, a metadata.jsonl/captions.jsonl row, or the dreambooth instance_prompt fallback. It also encodes that MiniMax-H3 trains from video with sound, not stills, so image-only or caption-less folders fail here.

Source

Thrown at studio/backend/core/training/diffusion_h3_clips.py:250

            sidecar = clip.with_suffix(ext)
            if sidecar.is_file():
                sidecar_present = True
                try:
                    caption = sidecar.read_text(encoding = "utf-8").strip()
                except (OSError, UnicodeError):
                    caption = ""
                break
        if not sidecar_present:
            caption = meta_caption.get(clip.name) or meta_caption.get(
                clip.relative_to(root).as_posix()
            )
        if not caption and instance_prompt:
            caption = instance_prompt
        if caption:
            pairs.append((str(clip), caption))

    if not pairs:
        raise ValueError(
            "No captioned video clips found. MiniMax-H3 trains from clips with sound, not "
            "stills: provide .mp4 / .mov / .mkv / .webm files plus a metadata.jsonl / "
            "captions.jsonl, per-clip .txt captions, or an instance prompt."
        )
    return pairs


def decode_clip(
    path: str | os.PathLike[str],
    *,
    num_frames: int,
    width: int,
    height: int,
    on_note: Optional[Callable[[str], None]] = None,
) -> tuple[Any, Any]:
    """Decode one training clip to ``(frames, waveform)``.

    ``frames`` is a uint8 numpy array of shape ``(num_frames, height, width, 3)`` resampled onto

View on GitHub (pinned to 203007d190)

Solutions

1. Add per-clip <stem>.txt caption sidecars or a valid UTF-8 metadata.jsonl/captions.jsonl with file_name + caption_column rows.
2. Or set an instance_prompt so every clip falls back to it.
3. Verify the folder actually contains .mp4/.mov/.mkv/.webm files and that caption keys exactly match clip names (or relative POSIX paths).

Example fix

# before
# data/myclip.mp4 with no caption anywhere, instance_prompt=None -> ValueError

# after
# data/myclip.txt containing the caption text
cfg.instance_prompt = "a skateboarder in slow motion"  # or per-clip sidecars

from pathlib import Path
VIDEO_EXTS = {".mp4", ".mov", ".mkv", ".webm", ".m4v", ".avi"}

def dataset_has_captioned_clips(data_dir: str, instance_prompt: str | None) -> bool:
    root = Path(data_dir).expanduser()
    clips = [p for p in root.iterdir() if p.is_file() and p.suffix.lower() in VIDEO_EXTS]
    if not clips:
        return False
    if instance_prompt and instance_prompt.strip():
        return True
    return any((p.with_suffix(".txt").is_file() or p.with_suffix(".caption").is_file()) for p in clips)

When it happens

Trigger: An empty directory; a folder of .mp4 files with no captions and no instance_prompt; captions keyed by a filename that doesn't match any clip's name or relative path; only images (.jpg) present; a metadata.jsonl that failed to parse (its OSError/UnicodeError is swallowed, leaving no captions).

Common situations: Pointing data_dir at the parent folder instead of the clips folder; caption files named clip-1.txt for clip_1.mp4; a UTF-16-encoded metadata.jsonl that silently fails to read; forgetting to set an instance prompt for dreambooth-style runs.

Related errors

• data_dir is not a directory: {data_dir}
• No captioned images found. Provide a metadata.jsonl / captio
• MiniMax-H3 was trained on aspect ratios from 1:4 to 4:1; thi
• {Path(path).name} carries no video track.
• {Path(path).name} carries no audio track. MiniMax-H3 denoise

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/b6d50864c6e7a7af. Report an issue: GitHub.