unslothai/unsloth · error · HTTPException

Failed to check dataset format: {scrubbed}

Error message

Failed to check dataset format: {scrubbed}

What it means

HTTP 500 raised at the end of the dataset format-check handler when the caught exception matched none of the mapped client-error branches (ENAMETOOLONG→400 invalid name, FileNotFoundError/DatasetNotFound→404, ValueError→400, hf_error_status mappings). The original message is scrubbed of secrets (`scrub_secrets`) and appended; the raw error is also logged at error level with the scrubbed text.

Source

Thrown at studio/backend/hub/services/datasets/formatting.py:531

    except Exception as e:
        scrubbed = download_registry.scrub_secrets(str(e), hf_token = hf_token)
        # Missing/gated/bad-token and malformed names are client errors, not 500s.
        status = hf_error_status(e)
        if (
            status is None
            and isinstance(e, OSError)
            and getattr(e, "errno", None) == errno.ENAMETOOLONG
        ):
            status, scrubbed = 400, "Invalid dataset name"
        elif status is None and isinstance(e, FileNotFoundError):
            # datasets raises DatasetNotFoundError (FileNotFoundError) for missing/gated.
            status = 404
        elif status is None and isinstance(e, ValueError):
            status = 400
        if status is not None:
            raise HTTPException(status_code = status, detail = scrubbed)
        logger.error("Error checking dataset format: %s", scrubbed)
        raise HTTPException(
            status_code = 500,
            detail = "Failed to check dataset format: " + scrubbed,
        )


def ai_assist_mapping_response(
    request: AiAssistMappingRequest, hf_token: Optional[str] = None
) -> AiAssistMappingResponse:
    """
    Run the LLM-assisted dataset conversion advisor (user-triggered).

    Multi-pass analysis with a 7B helper model: classify dataset type, generate
    a conversion strategy, then validate it. Falls back to simple column
    classification if the advisor fails.
    """
    try:
        from hub.utils.llm_assist import llm_conversion_advisor

View on GitHub (pinned to 203007d190)

Solutions

1. Read the scrubbed message in the 500 detail and the full traceback the server logged (`Error checking dataset format: ...`) — the detail is only the exception string.
2. If it names a corrupt file, delete the dataset cache and re-download.
3. If it names a version/library issue, pin matching pyarrow/datasets versions and restart the backend.
4. For transient hub errors, retry after a short wait.

try:
    fmt = check_dataset_format(client, req)
except HTTPStatusError as e:
    if e.response.status_code >= 500:
        log_server_trace_id(); notify_user_retryable()  # scrubbed detail + server log has traceback
    else:
        handle_client_error(e)  # 400/404 mappings are actionable

When it happens

Trigger: Any unexpected exception during format inspection: a corrupt parquet file raising pyarrow errors, an unexpected HTTP status from the hub, an arrow memory error, or a library incompatibility — anything that is not NotFound/ValueError/OSError-ENAMETOOLONG.

Common situations: Corrupt or truncated downloaded shards; pyarrow/datasets version mismatch after an upgrade; huge datasets hitting OOM during schema inference; transient hub 5xx that hf_error_status does not classify.

Related errors

• This execution artifact is outside the Recipe Studio dataset
• Execution artifacts are no longer available.
• Execution artifact path is not a dataset folder.
• NeMo Data Designer Hugging Face integration is not installed
• '{repo}' is gated on Hugging Face and this model cannot be d

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/31b29155fbd9f460. Report an issue: GitHub.