hiyouga/LlamaFactory · error · NotImplementedError

Unexpected role: {}

Error message

Unexpected role: {}

What it means

While encoding a conversation, each message's role must be one of Role.USER, Role.ASSISTANT, Role.OBSERVATION, or Role.FUNCTION. A message with any other role string (including typos and wrong casing) raises NotImplementedError in Template._encode. The role is compared against the Role enum, so the value must match exactly.

Source

Thrown at src/llamafactory/data/template.py:166

            if i == 0:
                elements += self.format_prefix.apply()
                if system or tools:
                    tool_text = self.format_tools.apply(content=tools)[0] if tools else ""
                    elements += self.format_system.apply(content=(system + tool_text))

            if message["role"] == Role.USER:
                elements += self.format_user.apply(content=message["content"], idx=str(i // 2))
            elif message["role"] == Role.ASSISTANT:
                elements += self.format_assistant.apply(content=message["content"])
            elif message["role"] == Role.OBSERVATION:
                elements += self.format_observation.apply(content=message["content"])
            elif message["role"] == Role.FUNCTION:
                elements += self.format_function.apply(
                    content=message["content"], thought_words=self.thought_words, tool_call_words=self.tool_call_words
                )
            else:
                raise NotImplementedError("Unexpected role: {}".format(message["role"]))

            encoded_messages.append(self._convert_elements_to_ids(tokenizer, elements))

        return encoded_messages

    @staticmethod
    def _add_or_replace_eos_token(tokenizer: "PreTrainedTokenizer", eos_token: str) -> None:
        r"""Add or replace eos token to the tokenizer."""
        if tokenizer.eos_token == eos_token:
            return

        is_added = tokenizer.eos_token_id is None
        num_added_tokens = tokenizer.add_special_tokens({"eos_token": eos_token})

        if is_added:
            logger.info_rank0(f"Add eos token: {tokenizer.eos_token}.")
        else:
            logger.info_rank0(f"Replace eos token: {tokenizer.eos_token}.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Inspect the failing sample (the exception context) and change the message role to one of 'user', 'assistant', 'observation', 'function'.
2. Move system content into the top-level 'system' field of the sample instead of a message with role 'system'.
3. If the role is a tool response, use 'observation' (for tool results fed back) and ensure the template defines format_observation; 'function' is for assistant tool calls rendered by format_function.
4. Validate the whole dataset before training with a small script that asserts allowed roles (see validationCode).

Example fix

// before
{"conversations": [{"from": "system", "value": "You are helpful."}, {"from": "human", "value": "Hi"}]}

// after
{"system": "You are helpful.", "conversations": [{"from": "human", "value": "Hi"}, {"from": "gpt", "value": "Hello!"}]}

ALLOWED = {"user", "assistant", "observation", "function"}
for i, sample in enumerate(dataset):
    for turn in sample["conversations"]:
        role = turn["from"]  # map to 'role' if alpaca-style tools data
        assert role in ALLOWED, f"sample {i}: bad role {role!r}"

def valid_sharegpt_roles(sample) -> bool:
    allowed = {"user", "assistant", "observation", "function"}
    return all(turn["from"] in allowed for turn in sample["conversations"])

try:
    processor._encode_data_example(...)
except NotImplementedError as e:
    if "Unexpected role" in str(e):
        log_bad_sample_and_continue(sample)  # quarantine, do not abort silently

When it happens

Trigger: A dataset sample whose messages contain a role like 'system', 'tool', 'assistant ', 'Bot', or 'function_call' instead of the supported enum values; converting a sharegpt-format dataset where the role field was renamed; using an observation message without the tool template that expects it (role mismatch, e.g. 'observation' vs 'function' depending on template).

Common situations: Preparing sharegpt data with 'from' values like 'system' or 'tool' mapped incorrectly in the dataset_info.json conversion script; datasets generated by other frameworks (axolotl, chatml) using 'tool' role; casing or whitespace differences after JSON editing.

Related errors

• Input must be string, set[str] or dict[str, str], got {type(
• Empty formatter should not contain any placeholder.
• A placeholder is required in the string formatter.
• Expected a string, got {value}
• Invalid JSON format in function message: {str([content])}.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/2a8066aa48852e33. Report an issue: GitHub.