karpathy/nanochat · error · ValueError

Unknown content type: {type(content)}

Error message

Unknown content type: {type(content)}

What it means

In `render_conversation`, an assistant message's `content` must be either a plain str or a list of part dicts. Any other type (int, None, dict, bytes) hits the else on the assistant branch and raises ValueError naming the type. (User messages have a separate strict isinstance(content, str) assert.)

Source

Thrown at nanochat/tokenizer.py:218

                        value_ids = self.encode(part["text"])
                        if part["type"] == "text":
                            # string part => simply add the tokens
                            add_tokens(value_ids, 1)
                        elif part["type"] == "python":
                            # python tool call => add the tokens inside <|python_start|> and <|python_end|>
                            add_tokens(python_start, 1)
                            add_tokens(value_ids, 1)
                            add_tokens(python_end, 1)
                        elif part["type"] == "python_output":
                            # python output => add the tokens inside <|output_start|> and <|output_end|>
                            # none of these tokens are supervised because the tokens come from Python at test time
                            add_tokens(output_start, 0)
                            add_tokens(value_ids, 0)
                            add_tokens(output_end, 0)
                        else:
                            raise ValueError(f"Unknown part type: {part['type']}")
                else:
                    raise ValueError(f"Unknown content type: {type(content)}")
                add_tokens(assistant_end, 1)

        # truncate to max_tokens tokens MAX (helps prevent OOMs)
        ids = ids[:max_tokens]
        mask = mask[:max_tokens]
        return ids, mask

    def visualize_tokenization(self, ids, mask, with_token_id=False):
        """Small helper function useful in debugging: visualize the tokenization of render_conversation"""
        RED = '\033[91m'
        GREEN = '\033[92m'
        RESET = '\033[0m'
        GRAY = '\033[90m'
        tokens = []
        for i, (token_id, mask_val) in enumerate(zip(ids, mask)):
            token_str = self.decode([token_id])
            color = GREEN if mask_val == 1 else RED
            tokens.append(f"{color}{token_str}{RESET}")

View on GitHub (pinned to 92d63d4e8b)

Solutions

1. Normalize the dataset: replace null assistant content with [] (empty part list) or a placeholder 'text' part.
2. Ensure assistant content is str or a list of {type, text} dicts.
3. Add a schema check over conversations before passing them to the tokenizer (see validation code).

Example fix

# before
{"role": "assistant", "content": None, "tool_calls": [...]}

# after
{"role": "assistant", "content": [{"type": "python", "text": "calc(2+2)"}]}

for i, msg in enumerate(conversation["messages"]):
    content = msg.get("content")
    if msg["role"] == "user":
        assert isinstance(content, str), "user content must be str"
    else:
        assert isinstance(content, (str, list)), f"assistant content must be str or list of parts, got {type(content).__name__}"

def is_valid_assistant_content(content) -> bool:
    return isinstance(content, str) or (isinstance(content, list) and all(isinstance(p, dict) for p in content))

When it happens

Trigger: A conversation where an assistant message's content is None (common in OpenAI-format data when tool_calls are used and content is nulled), a single dict instead of a list of dicts, or a number/other non-string scalar.

Common situations: Loading OpenAI-format chat logs where assistant tool-call messages have "content": null; a dataset bug where content fields were dropped; nested dict-of-parts instead of list-of-parts.

Related errors

• Invalid input type: {type(text)}
• Unknown part type: {part['type']}
• Unknown dataset tag: {dataset_tag}
• Unsupported task type: {task_type}
• Unsupported task evaluation type: {task_object.eval_type}

AI-assisted analysis of karpathy/nanochat@92d63d4e8b (2026-08-15). Data as JSON: /api/errors/a794b1df2f180504. Report an issue: GitHub.