hiyouga/LlamaFactory · error · ValueError

Unknown dataset filetype: {filetype}.

Error message

Unknown dataset filetype: {filetype}.

What it means

When loading a local dataset, the loader derives the HF datasets builder from the file extension. Only arrow, csv, json, jsonl, parquet and txt are recognized; any other extension raises this ValueError before load_dataset is attempted.

Source

Thrown at src/llamafactory/v1/plugins/data_plugins/loader.py:49

        split = dataset_info.get("split", "train")
        streaming = dataset_info.get("streaming", False)
        return super().__call__(path, split, streaming)


def _get_builder_name(path: str) -> Literal["arrow", "csv", "json", "parquet", "text"]:
    """Get dataset builder name.

    Args:
        path (str): Dataset path.

    Returns:
        Literal["arrow", "csv", "json", "parquet", "text"]: Dataset builder name.
    """
    filetype = os.path.splitext(path)[-1][1:]
    if filetype in ["arrow", "csv", "json", "jsonl", "parquet", "txt"]:
        return filetype.replace("jsonl", "json").replace("txt", "text")
    else:
        raise ValueError(f"Unknown dataset filetype: {filetype}.")


@DataLoaderPlugin("local").register()
def load_data_from_file(filepath: str, split: str, streaming: bool) -> HFDataset:
    if os.path.isdir(filepath):
        filetype = _get_builder_name(os.listdir(filepath)[0])
        dataset = load_dataset(filetype, data_dir=filepath, split=split)
    elif os.path.isfile(filepath):
        filetype = _get_builder_name(filepath)
        dataset = load_dataset(filetype, data_files=filepath, split=split)
    else:
        raise ValueError(f"Can not load dataset from {filepath}.")

    if streaming:  # faster when data is streamed from local files
        dataset = dataset.to_iterable_dataset()

    return dataset

View on GitHub (pinned to f28afaf635)

Solutions

1. Convert the data to a supported format: JSON/JSONL is usually the least friction.
2. Rename .tsv to .csv only if it is genuinely comma-separated (TSV is not valid CSV).
3. If loading from a directory, ensure every file inside has a supported extension; move READMEs and metadata out.
4. For arbitrary files, pre-convert with pandas: df.to_json('data.jsonl', orient='records', lines=True).

Example fix

# before
dataset_dir: data/my_notes.md

# after
# convert first:
import pandas as pd
pd.read_excel('data.xlsx').to_json('data/train.jsonl', orient='records', lines=True)
# then:
dataset_dir: data/train.jsonl

import os
SUPPORTED = {'.arrow', '.csv', '.json', '.jsonl', '.parquet', '.txt'}

def path_loadable(p: str) -> bool:
    if os.path.isdir(p):
        return bool(os.listdir(p)) and all(os.path.splitext(f)[-1] in SUPPORTED for f in os.listdir(p))
    return os.path.splitext(p)[-1] in SUPPORTED

assert path_loadable(dataset_dir)

def has_supported_extension(path: str) -> bool:
    """True when the path (or every file in the dir) uses a builder-supported extension."""
    return path_loadable(path)

try:
    ds = load_data_from_file(path, split='train', streaming=False)
except ValueError as e:
    if 'Unknown dataset filetype' in str(e):
        raise SystemExit(f'convert {path} to json/parquet/csv first') from None
    raise

When it happens

Trigger: Passing a dataset_dir/data_files path whose extension is not in [arrow, csv, json, jsonl, parquet, txt] — e.g. .tsv, .xlsx, .md, .pkl, or a file with no extension. Also fires when loading from a directory whose first file has such an extension.

Common situations: Exporting data from Excel/pandas as .xlsx or .tsv and pointing dataset_dir at it; pointing at a README.md or metadata file inside a data directory (os.listdir order makes the first file decide).

Related errors

• More {segment} tags than provided media files.
• Fewer {tag} tags than provided media files ({unused} unused)
• Can not load dataset from {filepath}.
• DPO training dataset is empty: {dataset_path}
• DPO training requires pair-format samples containing chosen/

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/464a6c2784203bc6. Report an issue: GitHub.