unslothai/unsloth · error · HTTPException

Dataset appears to be empty or could not be read

Error message

Dataset appears to be empty or could not be read

What it means

HTTP 400 from `_load_local_preview_slice` when a .json/.jsonl/.csv file's streaming previewer (`_stream_file_preview_slice`) returned None — it opened the file but could not extract a single valid record within the preview size. Distinct from 809 (no candidate file) and 811 (unknown suffix): here the extension was accepted but content was unreadable or empty.

Source

Thrown at studio/backend/hub/services/datasets/local.py:268

        if not candidate_files:
            raise HTTPException(
                status_code = 400,
                detail = "Unsupported local dataset directory (expected parquet/json/jsonl/csv files)",
            )
        dataset_path = candidate_files[0]

    suffix = dataset_path.suffix.lower()
    # Parquet/Arrow give a cheap exact total_rows; JSON/CSV carry none, so stream and report None.
    if suffix == ".parquet":
        dataset = load_dataset("parquet", data_files = str(dataset_path), split = train_split)
        total_rows = len(dataset)
        preview_slice = dataset.select(range(min(preview_size, total_rows)))
        return preview_slice, total_rows

    if suffix in (".json", ".jsonl", ".csv"):
        preview = _stream_file_preview_slice(dataset_path, preview_size)
        if preview is None:
            raise HTTPException(
                status_code = 400,
                detail = "Dataset appears to be empty or could not be read",
            )
        return preview

    raise HTTPException(status_code = 400, detail = f"Unsupported file format: {dataset_path.suffix}")


def _sanitize_filename(filename: str) -> str:
    name = Path(filename).name.strip().replace("\x00", "")
    if not name:
        return "dataset_upload"
    return name


def _upload_too_large(limit_label: str) -> HTTPException:
    return HTTPException(
        status_code = 413,

View on GitHub (pinned to 203007d190)

Solutions

1. Open the file and confirm it actually contains records — not just a header or `{...}` metadata object.
2. For JSON, convert to JSON Lines (one object per line) or a top-level array, which the streamer can read.
3. Re-export from the source tool (UTF-8, no BOM) or re-upload — truncated transfers are a common cause.
4. If the file is fine, convert to parquet, which takes the exact-count path instead of the streaming previewer.

Example fix

# before: meta.json = {"columns": [...], "rows": []}
# after: rows.jsonl = one {"col": value} object per line

def file_has_records(path: Path, sample: bytes = 4096) -> bool:
    if path.stat().st_size == 0:
        return False
    head = path.open("rb").read(sample).strip()
    return len(head) > 0 and not head.startswith(b"{}")  # bare metadata object will not stream

try:
    slice_, total = load_local_preview_slice(path, ...)
except HTTPException as e:
    if e.status_code == 400 and "could not be read" in e.detail:
        convert_to_parquet_then_retry(path)
    else:
        raise

When it happens

Trigger: Zero-byte or whitespace-only json/csv file; JSON not in a layout `datasets` can stream (e.g. arbitrary JSON object, not records/lines); CSV with only a header; BOM/encoding corruption; NDJSON with malformed first lines.

Common situations: Truncated uploads; exports that write a JSON dict instead of an array/lines; CSVs with header-only; wrong-encoding exports from Excel.

Related errors

• Unsupported local dataset directory (expected parquet/json/j
• Unsupported file format: {dataset_path.suffix}
• Dropped dataset is empty
• Empty upload payload
• Empty file not allowed

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/38c163171aadcef8. Report an issue: GitHub.