{"record":{"id":"23f38ce1772df485","repo":"unslothai/unsloth","slug":"could-not-infer-role-content-keys-for-chat-column","errorCode":null,"errorMessage":"Could not infer role/content keys for chat column '{chat_column}'","messagePattern":"Could not infer role/content keys for chat column '(.+?)'","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/utils/datasets/format_conversion.py","lineNumber":115,"sourceCode":"\n    if \"from\" in uniques and \"value\" in uniques:\n        role_key = \"from\"\n        content_key = \"value\"\n    elif \"role\" in uniques and \"content\" in uniques:\n        role_key = \"role\"\n        content_key = \"content\"\n    elif len(uniques.keys()) == 2:\n        keys = list(uniques.keys())\n        length_first = len(set(uniques[keys[0]]))\n        length_second = len(set(uniques[keys[1]]))\n        if length_first < length_second:\n            role_key = keys[0]\n            content_key = keys[1]\n        else:\n            role_key = keys[1]\n            content_key = keys[0]\n    else:\n        raise ValueError(f\"Could not infer role/content keys for chat column '{chat_column}'\")\n\n    # Mapping for aliases\n    aliases_mapping = {}\n    for x in aliases_for_system:\n        aliases_mapping[x] = \"system\"\n    for x in aliases_for_user:\n        aliases_mapping[x] = \"user\"\n    for x in aliases_for_assistant:\n        aliases_mapping[x] = \"assistant\"\n\n    def _standardize_dataset(examples):\n        convos = examples[chat_column]\n        all_convos = []\n        for convo in convos:\n            if not isinstance(convo, list):\n                all_convos.append([])\n                continue\n","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/utils/datasets/format_conversion.py#L97-L133","documentation":"ValueError from standardize_chat_format's key-inference step. It inspects the distinct keys found across turns in chat_column: one unique key is treated as content-only, exactly two are split into role/content by cardinality (the key with fewer distinct values becomes the role), and anything else — zero keys or three or more — cannot be inferred and raises. The error names the chat column so you know which column to fix.","triggerScenarios":"Turns carrying extra keys beyond the role/content pair, e.g. {'from','value','weight'}, {'role','content','tool_calls'}, or function-call datasets with {'from','value','function_call'}; also empty conversations or turns that are not dicts so no keys are collected.","commonSituations":"DPO/ORPO datasets with a 'weight' or 'score' per turn; agent/tool-use datasets with extra fields; a messages column containing None or string rows after a bad merge — all defeat the two-key inference heuristic.","solutions":["Pre-clean the column so every turn has exactly the role/content (or from/value) pair; drop auxiliary keys before standardizing","Filter out non-dict or empty turns: dataset.filter(lambda r: all(isinstance(t, dict) and t for t in r[chat_column]))","Move extra metadata to the row level (sibling columns) instead of inside each turn dict"],"exampleFix":"# before (turns: {'from','value','weight'} -> 3 unique keys -> raises)\nresult = standardize_chat_format(dataset, tokenizer, ..., chat_column='conversations')\n\n# after\ndataset = dataset.map(lambda r: {'conversations': [\n    {'from': t['from'], 'value': t['value']} for t in r['conversations']\n]})\nresult = standardize_chat_format(dataset, tokenizer, ..., chat_column='conversations')","handlingStrategy":"validation","validationCode":"def inferable_chat_keys(dataset, chat_column: str) -> bool:\n    \"\"\"True when turns use exactly one or two distinct keys (inference works).\"\"\"\n    uniques = set()\n    for row in list(dataset.select(range(min(50, len(dataset))))[chat_column]):\n        for turn in row or []:\n            if isinstance(turn, dict):\n                uniques.update(turn.keys())\n    return 1 <= len(uniques) <= 2","typeGuard":"def is_clean_turn(turn) -> bool:\n    \"\"\"Turn carries only a role/content (or from/value) pair.\"\"\"\n    return isinstance(turn, dict) and len(turn) == 2 and (\n        {\"role\", \"content\"} <= set(turn) or {\"from\", \"value\"} <= set(turn)\n    )","tryCatchPattern":null,"preventionTips":["Strip auxiliary per-turn keys (weight, score, tool_calls) before standardize_chat_format","Sample a few turns and count distinct keys — anything other than 1 or 2 will raise","Keep turn metadata at the row level, not inside the turn dicts"],"tags":["datasets","format-conversion","chatml","data-quality"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}