{"record":{"id":"e219bc0151b3e78b","repo":"unslothai/unsloth","slug":"streaming-chat-format-standardization-failed-on-th","errorCode":null,"errorMessage":"Streaming chat-format standardization failed on the first row: {exc}","messagePattern":"Streaming chat-format standardization failed on the first row: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/utils/datasets/format_conversion.py","lineNumber":186,"sourceCode":"\n        if num_proc is None or type(num_proc) is not int:\n            num_proc = dataset_map_num_proc()\n        else:\n            num_proc = dataset_map_num_proc(num_proc)\n\n        dataset_map_kwargs[\"num_proc\"] = num_proc\n        dataset_map_kwargs[\"desc\"] = \"Standardizing chat format\"\n\n    result = dataset.map(_standardize_dataset, **dataset_map_kwargs)\n\n    # For streaming, force the first mapped row through now so any\n    # column/format errors surface before training begins (not mid-iteration).\n    # IterableDataset re-iterates from the generator source, so this is safe.\n    if is_streaming_dataset(dataset):\n        try:\n            next(iter(result))\n        except Exception as exc:\n            raise ValueError(\n                f\"Streaming chat-format standardization failed on the first row: {exc}\"\n            ) from exc\n\n    return result\n\n\ndef convert_chatml_to_alpaca(\n    dataset,\n    batch_size = 1000,\n    num_proc = None,\n    chat_column: str | None = None,\n):\n    \"\"\"\n    Convert ChatML (messages OR conversations) to Alpaca format.\n\n    Supports:\n    - \"messages\" or \"conversations\" column\n    - \"role\"/\"content\" (standard) or \"from\"/\"value\" (ShareGPT)","sourceCodeStart":168,"sourceCodeEnd":204,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/utils/datasets/format_conversion.py#L168-L204","documentation":"ValueError raised eagerly for streaming datasets after dataset.map(_standardize_dataset): the first mapped row is forced through (next(iter(result))) so per-row schema/type errors surface now, at setup time, instead of mid-training when iteration first hits them. The original exception is chained (from exc), so the real cause — usually a missing chat column or malformed turn in row 0 — is in exc, and IterableDataset re-iterates from the source generator making this probe safe/non-destructive.","triggerScenarios":"Calling standardize_chat_format on a streaming dataset where the source rows lack chat_column, contain turns that are not dicts, or where _standardize_dataset raises KeyError/TypeError on the very first row.","commonSituations":"Streaming remote datasets whose first shard has a different schema; conversation fields nested under a different name than chat_column; server-side data changes after the stream handle was created.","solutions":["Read the chained exception (raise ... from exc) — fix the underlying per-row error it names, not this wrapper","Materialize and inspect the first row before standardizing: row = next(iter(dataset)); print(row[chat_column])","Verify chat_column actually exists in the streaming schema (dataset.features) and matches the turn structure"],"exampleFix":"# before\nresult = standardize_chat_format(stream_ds, tok, ..., chat_column='messages')\n# raises 'failed on the first row: KeyError ...' because column is 'conversation'\n\n# after\nprint(next(iter(stream_ds)).keys())  # -> 'conversation'\nresult = standardize_chat_format(stream_ds, tok, ..., chat_column='conversation')","handlingStrategy":"try-catch","validationCode":"def first_row_scan_ready(stream_ds, chat_column: str) -> bool:\n    row = next(iter(stream_ds), None)\n    if row is None or chat_column not in row:\n        return False\n    turns = row[chat_column] or []\n    return bool(turns) and isinstance(turns[0], dict)","typeGuard":null,"tryCatchPattern":"try:\n    result = standardize_chat_format(stream_ds, tok, ..., chat_column=col)\nexcept ValueError as e:\n    if \"failed on the first row\" in str(e) and e.__cause__ is not None:\n        diagnose_from(e.__cause__)  # the real per-row error\n        raise\n    raise","preventionTips":["Always inspect the chained __cause__ — the wrapper message alone hides the real error","Print next(iter(dataset)) and check the conversation field before streaming pipelines","Pass chat_column explicitly on streaming datasets; fallback probing is riskier there"],"tags":["datasets","streaming","format-conversion","fail-fast"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}