hiyouga/LlamaFactory · error · RuntimeError

Invalid JSON format in function message: {str([content])}.

Error message

Invalid JSON format in function message: {str([content])}.

What it means

Raised inside FunctionFormatter.apply -> _parse_functions when the `content` of a function/assistant tool-call message cannot be parsed by json.loads (json.JSONDecodeError). LlamaFactory expects the function_call or tool_calls content in your dataset to be valid JSON (an object or a list of objects with 'name' and 'arguments'). This is a per-sample data validation error: one bad row aborts preprocessing.

Source

Thrown at src/llamafactory/data/formatter.py:110

    def __post_init__(self):
        super().__post_init__()
        self.tool_utils = get_tool_utils(self.tool_format)

    @override
    def apply(self, **kwargs) -> SLOTS:
        content: str = kwargs.pop("content")
        thought_words = kwargs.pop("thought_words", None)
        tool_call_words = kwargs.pop("tool_call_words", None)

        def _parse_functions(json_content: str) -> list["FunctionCall"]:
            try:
                tool_calls = json.loads(json_content)
                if not isinstance(tool_calls, list):  # parallel function call
                    tool_calls = [tool_calls]

                return [FunctionCall(tc["name"], json.dumps(tc["arguments"], ensure_ascii=False)) for tc in tool_calls]
            except json.JSONDecodeError:
                raise RuntimeError(f"Invalid JSON format in function message: {str([content])}.")

        tool_call_match = None
        if tool_call_words and len(tool_call_words) == 2:
            tool_call_regex = re.compile(
                rf"{re.escape(tool_call_words[0])}(.*?){re.escape(tool_call_words[1])}", re.DOTALL
            )
            tool_call_match = re.search(tool_call_regex, content)

        if tool_call_match is None:
            thought_match = None
            if thought_words and len(thought_words) == 2:
                regex = re.compile(rf"{re.escape(thought_words[0])}(.*?){re.escape(thought_words[1])}", re.DOTALL)
                thought_match = re.search(regex, content)

            if thought_match:
                json_part = content.replace(thought_match.group(0), "")
            else:
                json_part = content

View on GitHub (pinned to f28afaf635)

Solutions

1. Locate the offending sample using the content echoed in the error message and fix its JSON so it is an object or list of {"name": ..., "arguments": {...}}.
2. Sanitize the whole dataset with a script that json.loads each tool-call content and drops/repairs failures before training.
3. Strip markdown fences, leading/trailing prose, and ensure double quotes are used; re-save with ensure_ascii=False.
4. If the samples are intentionally non-JSON thought traces, move them to the normal content field and keep only real calls in the function-call field.

Example fix

# before (dataset row)
{"from": "function_call", "value": "```json\n{ 'name': 'search', 'arguments': {'q': 'hf'} }\n```"}

# after
{"from": "function_call", "value": "{\"name\": \"search\", \"arguments\": {\"q\": \"hf\"}}"}

import json

def tool_call_rows_ok(rows: list[dict]) -> list[int]:
    bad = []
    for i, r in enumerate(rows):
        content = r.get("function_call") or r.get("tool_calls")
        if content:
            try:
                parsed = json.loads(content)
                if not isinstance(parsed, (dict, list)):
                    bad.append(i)
            except json.JSONDecodeError:
                bad.append(i)
    return bad

def is_valid_tool_call_json(content: str) -> bool:
    try:
        parsed = json.loads(content)
        return isinstance(parsed, (dict, list)) and all("name" in tc and "arguments" in tc for tc in (parsed if isinstance(parsed, list) else [parsed]))
    except (json.JSONDecodeError, TypeError):
        return False

try:
    json.loads(content)
except json.JSONDecodeError as e:
    logger.warning("sample %d has malformed tool call JSON: %s", idx, e)
    # repair or drop the row instead of aborting the whole run

When it happens

Trigger: A sharegpt-format dataset whose assistant messages with function calls contain malformed JSON (truncated responses, single quotes, trailing commas, raw markdown fences, or plain-text tool output placed in the function content field). Occurs during dataset preprocessing for tool-use SFT/DPO when tool_format is set (e.g. react, llama3, qwen).

Common situations: Log datasets harvested from model outputs where tool calls were not strictly serialized; datasets converted from other frameworks that wrap JSON in ```json fences or prose; encodings/BOM issues; non-ASCII content saved without proper quoting.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Invalid JSON format in tool description: {str([content])}.
• tool_call value is not valid JSON: {content['value']!r}
• tool_call must be a JSON object with 'name' and 'arguments'
• Input must be string, set[str] or dict[str, str], got {type(
• Cannot find valid samples, check `data/README.md` for the da

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/23e2c2eefa97b9b4. Report an issue: GitHub.