oobabooga/textgen · error · RuntimeError

Dataset row must contain "messages" or "conversations" key…

Error message

Dataset row must contain "messages" or "conversations" key. Found: {list(data_point.keys())}

What it means

Thrown by normalize_messages() during LoRA/dataset training when a dataset row contains neither a 'messages' key (OpenAI chat format) nor a 'conversations' key (ShareGPT format). The training tokenizer path requires one of these schemas to build chat-template input, so any other column layout aborts preprocessing. The message lists the actual keys found, which tells you exactly what schema the row does have.

Solutions

1. Convert the dataset to one of the accepted schemas: either [{'role': 'user'|'assistant'|'system', 'content': str}] under a 'messages' key, or [{'from': 'human'|'gpt', 'value': str}] under a 'conversations' key.
2. If the source is Alpaca-style, map instruction/output columns to messages programmatically before training (e.g. df.apply to build the messages list).
3. Check for typos or nested structures in the dataset column names by printing dataset[0].keys() and comparing against the two accepted keys.
4. As a last resort, pre-transform the dataset file offline (jq/pandas) so every row literally has 'messages' or 'conversations' at the top level.

Example fix

# before (row only has Alpaca columns)
# {"instruction": "Hi", "output": "Hello!"}  -> RuntimeError

# after: convert to messages format
import json
rows = [json.loads(l) for l in open('data.jsonl')]
with open('data_fixed.jsonl', 'w') as f:
    for r in rows:
        f.write(json.dumps({"messages": [
            {"role": "user", "content": r["instruction"]},
            {"role": "assistant", "content": r["output"]},
        ]}) + '\n')

REQUIRED_KEYS = {"messages", "conversations"}

def row_is_trainable(row: dict) -> bool:
    return bool(REQUIRED_KEYS & set(row.keys()))

# before mapping the dataset:
bad = [i for i, row in enumerate(dataset) if not row_is_trainable(row)]
if bad:
    raise ValueError(f'{len(bad)} rows lack messages/conversations; first bad index: {bad[0]}, '
                     f'keys: {list(dataset[bad[0]].keys())}')

from typing import TypedDict

class Message(TypedDict):
    role: str
    content: str

class MessagesRow(TypedDict):
    messages: list[Message]

class Turn(TypedDict):
    from_: str  # 'human' | 'gpt'
    value: str

class ConversationsRow(TypedDict):
    conversations: list[Turn]

def is_chat_row(row: dict) -> bool:
    if "messages" in row:
        msgs = row["messages"]
        return isinstance(msgs, list) and all(
            isinstance(m, dict) and m.get("role") in {"user", "assistant", "system"} and isinstance(m.get("content"), str)
            for m in msgs
        )
    if "conversations" in row:
        turns = row["conversations"]
        return isinstance(turns, list) and all(
            isinstance(t, dict) and t.get("from") in {"human", "gpt", "system"} and isinstance(t.get("value"), str)
            for t in turns
        )
    return False

When it happens

Trigger: Loading a Hugging Face dataset whose rows use a different schema (e.g. only 'text', 'prompt'/'response', 'instruction'/'input'/'output' Alpaca-style columns) and starting training with any apply_chat_template-based dataset path. Also triggered by datasets that nest the chat under another column name, or by a mapping step that was skipped.

Common situations: User points --dataset at an Alpaca-format JSON or a raw completion corpus while the trainer expects chat-format data; dataset was regenerated with renamed columns; a ShareGPT export uses 'from'/'value' but the top-level key is 'chat' or 'history' instead of 'conversations'.

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/ab73ec63fb3b7bc0. Report an issue: GitHub.

Appendix: source

Thrown at modules/training.py:343

        target_mods = [f"{name}_proj" for name, enabled in {
            "q": q_proj_en, "k": k_proj_en, "v": v_proj_en, "o": o_proj_en,
            "gate": gate_proj_en, "down": down_proj_en, "up": up_proj_en,
        }.items() if enabled]
        return target_mods

    def normalize_messages(data_point):
        """Convert a dataset row to OpenAI messages format for apply_chat_template()."""
        if "messages" in data_point:
            return data_point["messages"]

        if "conversations" in data_point:
            role_map = {"human": "user", "gpt": "assistant"}
            return [
                {"role": role_map.get(turn.get("from", ""), turn.get("from", "")), "content": turn["value"]}
                for turn in data_point["conversations"]
            ]

        raise RuntimeError(
            f'Dataset row must contain "messages" or "conversations" key. '
            f'Found: {list(data_point.keys())}'
        )

    def tokenize_conversation(data_point):
        """Tokenize using apply_chat_template() with assistant-only label masking."""
        messages = normalize_messages(data_point)
        full_ids = list(shared.tokenizer.apply_chat_template(messages, tokenize=True, return_dict=False))

        # Build labels: -100 for everything, then unmask assistant turns.
        # This assumes apply_chat_template(messages[:i]) is a token-for-token
        # prefix of apply_chat_template(messages[:i+1]), which holds for all
        # standard chat templates (Llama, ChatML, Mistral, etc.).
        labels = [-100] * len(full_ids)
        for i, msg in enumerate(messages):
            if msg["role"] == "assistant":
                # Tokens up to where this assistant turn starts
                header_ids = shared.tokenizer.apply_chat_template(

View on GitHub (pinned to ed888c71f2)