unslothai/unsloth · error · FileNotFoundError

data_dir is not a directory: {data_dir}

Error message

data_dir is not a directory: {data_dir}

What it means

Raised by the H3 clip discovery routine when data_dir does not exist or is not a directory (FileNotFoundError). It is the first thing checked after expanduser(), before any clip scanning or caption resolution. A file path, a broken symlink, or an unmounted volume all land here.

Source

Thrown at studio/backend/core/training/diffusion_h3_clips.py:199

    data_dir: str | os.PathLike[str],
    *,
    instance_prompt: Optional[str] = None,
    caption_column: str = "text",
) -> list[tuple[str, str]]:
    """Resolve ``(clip_path, caption)`` pairs from a dataset directory.

    The caption rules are exactly ``discover_image_caption_pairs``' -- a per-clip ``<stem>.txt``
    / ``<stem>.caption`` sidecar wins, then a ``metadata.jsonl`` / ``captions.jsonl`` row keyed
    by ``file_name`` (or ``video`` / ``image`` / ``file``) carrying ``caption_column``, then the
    dreambooth ``instance_prompt`` -- so a user who has captioned an image dataset already knows
    this layout. Only the file extensions differ.

    An empty sidecar is the same deliberate tombstone it is for images: it suppresses the
    metadata caption and leaves the clip uncaptioned, so the ``instance_prompt`` fallback applies.
    """
    root = Path(data_dir).expanduser()
    if not root.is_dir():
        raise FileNotFoundError(f"data_dir is not a directory: {data_dir}")

    clips = sorted(p for p in root.iterdir() if p.is_file() and p.suffix.lower() in _VIDEO_EXTS)

    meta_caption: dict[str, str] = {}
    for meta_name in ("metadata.jsonl", "captions.jsonl"):
        meta_path = root / meta_name
        if not meta_path.is_file():
            continue
        try:
            meta_lines = meta_path.read_text(encoding = "utf-8").splitlines()
        except (OSError, UnicodeError):
            continue
        for line in meta_lines:
            line = line.strip()
            if not line:
                continue
            try:
                row = json.loads(line)

View on GitHub (pinned to 203007d190)

Solutions

1. Correct data_dir to the folder that actually contains the video files.
2. Use an absolute path to avoid working-directory ambiguity.
3. Confirm the volume is mounted and the process has permission to stat the directory.

Example fix

# before
pairs = discover_video_caption_pairs("~/trainng-data")  # typo, missing dir

# after
from pathlib import Path
data = Path("~/training-data").expanduser().resolve()
assert data.is_dir(), f"missing dataset dir: {data}"
pairs = discover_video_caption_pairs(data)

from pathlib import Path

def dataset_dir_ok(data_dir: str) -> bool:
    return Path(data_dir).expanduser().is_dir()

When it happens

Trigger: Passing a wrong/misspelled data_dir; passing a path to a file instead of its parent directory; a tilde path that expanduser resolves differently than expected; an external drive or network share not mounted.

Common situations: Typos in long dataset paths; relative paths evaluated from a different working directory; moving the dataset after saving the config; unmounted USB/NAS storage at run time.

Related errors

• No captioned video clips found. MiniMax-H3 trains from clips
• data_dir is not a directory: {data_dir}
• The training checkpoint at '{directory}' could not be read;
• MiniMax-H3 was trained on aspect ratios from 1:4 to 4:1; thi
• {Path(path).name} carries no video track.

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/a48d874aa5567dc6. Report an issue: GitHub.