unslothai/unsloth · error · ValueError

Unsupported local dataset format: {all_files[0]}

Error message

Unsupported local dataset format: {all_files[0]}

What it means

Raised when the first collected local dataset file has an extension other than .json/.jsonl/.csv/.parquet. The loader type is chosen from the first file's suffix, so an unsupported first file aborts the load even if later files are supported.

Source

Thrown at studio/backend/core/training/worker.py:5059

                    if candidates:
                        all_files.extend(str(c) for c in candidates)
                        continue
                    raise ValueError(f"No supported data files in directory: {file_path_obj}")
                else:
                    all_files.append(file_path)

            if not all_files:
                raise ValueError("No local dataset files found")

            first_ext = Path(all_files[0]).suffix.lower()
            if first_ext in (".json", ".jsonl"):
                loader = "json"
            elif first_ext == ".csv":
                loader = "csv"
            elif first_ext == ".parquet":
                loader = "parquet"
            else:
                raise ValueError(f"Unsupported local dataset format: {all_files[0]}")
            return load_dataset(loader, data_files = all_files, split = "train")

        if hf_dataset:
            dataset = _load_embedding_hf_dataset(
                config,
                load_dataset,
                lambda message: _send_status(event_queue, message),
            )
        elif local_datasets:
            dataset = _load_local_embedding_dataset(local_datasets)
        elif config.get("s3_config"):
            from core.training.s3_dataset import (
                S3DownloadCancelled,
                prepare_s3_dataset_download,
            )

            _send_status(event_queue, "Downloading dataset from S3...")
            s3_download = None

View on GitHub (pinned to 203007d190)

Solutions

1. Remove or relocate unsupported files from the dataset directory so the first (sorted) file is .json/.jsonl/.csv/.parquet.
2. Rename .tsv to .csv (with correct delimiter handling) or convert .arrow/.txt to jsonl/parquet.
3. Ensure the directory holds a single consistent format.

Example fix

# before
# /data contains: notes.txt, train.jsonl -> first_ext == '.txt' -> raises

# after
# /data contains only: train.jsonl -> loader = 'json'

from pathlib import Path

SUPPORTED = {".json", ".jsonl", ".csv", ".parquet"}
files = sorted(f for f in Path(data_dir).iterdir() if f.is_file())
assert files and files[0].suffix.lower() in SUPPORTED, f"first file {files[0].name} has unsupported format"

def is_supported_dataset_file(path: str) -> bool:
    return Path(path).suffix.lower() in {".json", ".jsonl", ".csv", ".parquet"}

try:
    ds = _load_local_embedding_dataset(paths)
except ValueError as e:
    if "Unsupported local dataset format" in str(e):
        # filter/convert the offending files, then retry once
        raise
    raise

When it happens

Trigger: all_files[0] ends in something like .arrow, .txt, .tsv, .xlsx, or no extension at all. Because only the first file's extension is inspected, a mixed directory where an unsupported file sorts first (sorted glob order) triggers this even when supported files exist alongside it.

Common situations: datasets.save_to_disk output (.arrow) pointed at the loader; .tsv files mistaken for .csv; mixed-format directories where e.g. notes.txt sorts before data.jsonl; files with compound suffixes like data.jsonl.tmp.

Related errors

• No captioned images found. Provide a metadata.jsonl / captio
• dataset_slice_end must be greater than or equal to dataset_s
• hf_dataset is too long (max 256 chars)
• hf_dataset contains invalid characters or path segments
• subset is too long (max {MAX_HF_DATASET_OPTION_LENGTH} chars

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/0d86798bc3fa91d5. Report an issue: GitHub.