{"record":{"id":"40ad02ac4ad05d1f","repo":"unslothai/unsloth","slug":"data-dir-is-not-a-directory-data-dir-40ad02","errorCode":null,"errorMessage":"data_dir is not a directory: {data_dir}","messagePattern":"data_dir is not a directory: (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_train_common.py","lineNumber":1382,"sourceCode":"      3. ``instance_prompt`` (dreambooth) for any remaining image.\n\n    A sidecar wins over the metadata row because it is the user's explicit per-image edit\n    (the labeling grid writes a .txt sidecar), which must override the bulk metadata file.\n    Must agree with ``routes.training._image_record``, which resolves captions the same way.\n\n    Images with no caption from any source are skipped. Pure filesystem + JSON, so it is\n    unit-testable without torch. Raises FileNotFoundError for a missing dir and ValueError\n    when nothing is captionable.\n\n    ``verify_images`` (opt-in) additionally runs a cheap PIL header probe on each captioned\n    image and raises ValueError on a corrupt/zero-byte/truncated file. The start route enables\n    it so a bad upload is rejected BEFORE the resident GPU models are freed, instead of crashing\n    the spawned trainer after teardown; the trainers leave it off (they decode every image\n    anyway, so a second probe pass would be redundant).\n    \"\"\"\n    root = Path(data_dir).expanduser()\n    if not root.is_dir():\n        raise FileNotFoundError(f\"data_dir is not a directory: {data_dir}\")\n\n    images = sorted(p for p in root.iterdir() if p.is_file() and p.suffix.lower() in _IMAGE_EXTS)\n\n    # 1. metadata.jsonl / captions.jsonl (either name accepted).\n    meta_caption: dict[str, str] = {}\n    for meta_name in (\"metadata.jsonl\", \"captions.jsonl\"):\n        meta_path = root / meta_name\n        if not meta_path.is_file():\n            continue\n        # Tolerate a bad upload (invalid UTF-8, or non-object JSON): skip the record so the instance_prompt fallback still applies.\n        try:\n            meta_lines = meta_path.read_text(encoding = \"utf-8\").splitlines()\n        except (OSError, UnicodeError):\n            continue\n        for line in meta_lines:\n            line = line.strip()\n            if not line:\n                continue","sourceCodeStart":1364,"sourceCodeEnd":1400,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_train_common.py#L1364-L1400","documentation":"The dataset builder expands the data_dir path and requires it to be an existing directory. A missing or non-directory path raises FileNotFoundError before any image discovery runs. This is deliberately filesystem-only so it is unit-testable without torch.","triggerScenarios":"Calling the caption-pair collection function with a typo'd path, a file path instead of a directory, a path on an unmounted volume, or '~' that fails expanduser resolution; also paths whose containing directory was deleted between upload and training start.","commonSituations":"Upload race (training starts before the dataset dir is materialized), docker volume mount mistakes, a stale absolute path after the workspace moved, or a Windows path used on Linux.","solutions":["Check the path exists and is a directory before starting training: Path(data_dir).expanduser().is_dir().","If it is an upload race, poll for the dataset directory's existence before invoking the trainer.","Verify the mount/volume and that the path points at the dataset root (the directory containing the images), not at an image file."],"exampleFix":"# before\nbuild_caption_pairs('/data/mydateset', ...)\n# after\nfrom pathlib import Path\nroot = Path('/data/mydataset').expanduser()\nassert root.is_dir(), f'{root} missing'\nbuild_caption_pairs(root, ...)","handlingStrategy":"validation","validationCode":"from pathlib import Path\nroot = Path(data_dir).expanduser()\nif not root.is_dir():\n    raise FileNotFoundError(f'data_dir is not a directory: {data_dir}')","typeGuard":null,"tryCatchPattern":"try:\n    pairs = build_caption_pairs(data_dir, ...)\nexcept FileNotFoundError as e:\n    # surface to the user: dataset path missing/unmounted\n    abort_run(str(e))","preventionTips":["Verify the dataset directory exists immediately before the start call, after uploads settle.","Log the resolved absolute path so mount mistakes are obvious."],"tags":["training","dataset","filesystem","file-not-found"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}