{"record":{"id":"a48d874aa5567dc6","repo":"unslothai/unsloth","slug":"data-dir-is-not-a-directory-data-dir","errorCode":null,"errorMessage":"data_dir is not a directory: {data_dir}","messagePattern":"data_dir is not a directory: (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_h3_clips.py","lineNumber":199,"sourceCode":"    data_dir: str | os.PathLike[str],\n    *,\n    instance_prompt: Optional[str] = None,\n    caption_column: str = \"text\",\n) -> list[tuple[str, str]]:\n    \"\"\"Resolve ``(clip_path, caption)`` pairs from a dataset directory.\n\n    The caption rules are exactly ``discover_image_caption_pairs``' -- a per-clip ``<stem>.txt``\n    / ``<stem>.caption`` sidecar wins, then a ``metadata.jsonl`` / ``captions.jsonl`` row keyed\n    by ``file_name`` (or ``video`` / ``image`` / ``file``) carrying ``caption_column``, then the\n    dreambooth ``instance_prompt`` -- so a user who has captioned an image dataset already knows\n    this layout. Only the file extensions differ.\n\n    An empty sidecar is the same deliberate tombstone it is for images: it suppresses the\n    metadata caption and leaves the clip uncaptioned, so the ``instance_prompt`` fallback applies.\n    \"\"\"\n    root = Path(data_dir).expanduser()\n    if not root.is_dir():\n        raise FileNotFoundError(f\"data_dir is not a directory: {data_dir}\")\n\n    clips = sorted(p for p in root.iterdir() if p.is_file() and p.suffix.lower() in _VIDEO_EXTS)\n\n    meta_caption: dict[str, str] = {}\n    for meta_name in (\"metadata.jsonl\", \"captions.jsonl\"):\n        meta_path = root / meta_name\n        if not meta_path.is_file():\n            continue\n        try:\n            meta_lines = meta_path.read_text(encoding = \"utf-8\").splitlines()\n        except (OSError, UnicodeError):\n            continue\n        for line in meta_lines:\n            line = line.strip()\n            if not line:\n                continue\n            try:\n                row = json.loads(line)","sourceCodeStart":181,"sourceCodeEnd":217,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_h3_clips.py#L181-L217","documentation":"Raised by the H3 clip discovery routine when data_dir does not exist or is not a directory (FileNotFoundError). It is the first thing checked after expanduser(), before any clip scanning or caption resolution. A file path, a broken symlink, or an unmounted volume all land here.","triggerScenarios":"Passing a wrong/misspelled data_dir; passing a path to a file instead of its parent directory; a tilde path that expanduser resolves differently than expected; an external drive or network share not mounted.","commonSituations":"Typos in long dataset paths; relative paths evaluated from a different working directory; moving the dataset after saving the config; unmounted USB/NAS storage at run time.","solutions":["Correct data_dir to the folder that actually contains the video files.","Use an absolute path to avoid working-directory ambiguity.","Confirm the volume is mounted and the process has permission to stat the directory."],"exampleFix":"# before\npairs = discover_video_caption_pairs(\"~/trainng-data\")  # typo, missing dir\n\n# after\nfrom pathlib import Path\ndata = Path(\"~/training-data\").expanduser().resolve()\nassert data.is_dir(), f\"missing dataset dir: {data}\"\npairs = discover_video_caption_pairs(data)","handlingStrategy":"validation","validationCode":"from pathlib import Path\n\ndef dataset_dir_ok(data_dir: str) -> bool:\n    return Path(data_dir).expanduser().is_dir()","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Store absolute, resolved dataset paths in configs.","Verify mounts exist right before launching the run, not only at config time."],"tags":["filesystem","dataset","video","training"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}