unslothai/unsloth · error · HTTPException

Unsupported local dataset directory (expected parquet/json/j

Error message

Unsupported local dataset directory (expected parquet/json/jsonl/csv files)

What it means

HTTP 400 from local dataset preview loading when the target is a directory that contains no files matching LOCAL_FILE_EXTS (parquet/json/jsonl/csv) after the parquet-multi-file branch also found nothing. The directory exists but holds no recognizable dataset files, so Studio refuses to guess.

Source

Thrown at studio/backend/hub/services/datasets/local.py:251

            if (dataset_path / "parquet-files").exists()
            else dataset_path
        )
        parquet_files = sorted(parquet_dir.glob("*.parquet"))
        if parquet_files:
            dataset = load_dataset(
                "parquet",
                data_files = [str(path) for path in parquet_files],
                split = train_split,
            )
            total_rows = len(dataset)
            preview_slice = dataset.select(range(min(preview_size, total_rows)))
            return preview_slice, total_rows

        candidate_files: list[Path] = []
        for ext in LOCAL_FILE_EXTS:
            candidate_files.extend(sorted(dataset_path.glob(f"*{ext}")))
        if not candidate_files:
            raise HTTPException(
                status_code = 400,
                detail = "Unsupported local dataset directory (expected parquet/json/jsonl/csv files)",
            )
        dataset_path = candidate_files[0]

    suffix = dataset_path.suffix.lower()
    # Parquet/Arrow give a cheap exact total_rows; JSON/CSV carry none, so stream and report None.
    if suffix == ".parquet":
        dataset = load_dataset("parquet", data_files = str(dataset_path), split = train_split)
        total_rows = len(dataset)
        preview_slice = dataset.select(range(min(preview_size, total_rows)))
        return preview_slice, total_rows

    if suffix in (".json", ".jsonl", ".csv"):
        preview = _stream_file_preview_slice(dataset_path, preview_size)
        if preview is None:
            raise HTTPException(
                status_code = 400,

View on GitHub (pinned to 203007d190)

Solutions

  1. Move or place at least one .parquet/.json/.jsonl/.csv file directly (non-nested) in the directory.
  2. If files sit in a subfolder, select/point at that subfolder instead of the parent.
  3. If files are git-LFS pointer stubs, run `git lfs pull` so real data files exist on disk.
  4. Convert unsupported formats (e.g. .xlsx, .arrow) to parquet or csv before previewing.

Example fix

# before: my_dataset/README.md, my_dataset/data/part-0.parquet
# after (point at the folder holding the files, or flatten):
# my_dataset/part-0.parquet
Defensive patterns

Strategy: validation

Validate before calling

from pathlib import Path

LOCAL_FILE_EXTS = (".parquet", ".json", ".jsonl", ".csv")

def dir_has_dataset_files(root: Path) -> bool:
    return any(p.is_file() and p.suffix.lower() in LOCAL_FILE_EXTS for p in root.glob("*"))

Type guard

def is_previewable_dir(root: Path) -> TypeGuard[Path]:
    return root.is_dir() and dir_has_dataset_files(root)

Try / catch

try:
    load_preview(path)
except HTTPException as e:
    if e.status_code == 400 and "Unsupported local dataset directory" in e.detail:
        show_format_help(allowed=LOCAL_FILE_EXTS)
    else:
        raise

Prevention

When it happens

Trigger: Previewing a directory of images, text files, .arrow files not in the ext list, nested subdirectories (glob is `*{ext}`, non-recursive), or an empty dir.

Common situations: User points Studio at a folder of raw .txt/.jpg files; data lives one level deeper (e.g. `data/`); dataset repo cloned with git LFS pointers instead of real parquet files.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/22fb541183784a17. Report an issue: GitHub.