hiyouga/LlamaFactory · error · RuntimeError

Invalid JSON format in tool description: {str([content])}.

Error message

Invalid JSON format in tool description: {str([content])}.

What it means

Raised by ToolFormatter.apply when the `tools` column of a sample fails json.loads. LlamaFactory expects the tools description of a sharegpt-style dataset to be a valid JSON list of tool schemas; the formatter then renders it with the model-specific tool_utils.tool_formatter. A flat string or any non-JSON text triggers this RuntimeError during preprocessing.

Source

Thrown at src/llamafactory/data/formatter.py:155

            function_str = self.tool_utils.function_formatter(functions)
            function_str = thought_content + function_str

        return super().apply(content=function_str)


@dataclass
class ToolFormatter(Formatter):
    def __post_init__(self):
        self.tool_utils = get_tool_utils(self.tool_format)

    @override
    def apply(self, **kwargs) -> SLOTS:
        content = kwargs.pop("content")
        try:
            tools = json.loads(content)
            return [self.tool_utils.tool_formatter(tools) if len(tools) != 0 else ""]
        except json.JSONDecodeError:
            raise RuntimeError(f"Invalid JSON format in tool description: {str([content])}.")  # flat string

    @override
    def extract(self, content: str) -> str | list["FunctionCall"]:
        return self.tool_utils.tool_extractor(content)

View on GitHub (pinned to f28afaf635)

Solutions

1. Make every non-empty `tools` field a strict JSON array of tool schemas, e.g. [{"name": ..., "description": ..., "parameters": {...}}].
2. Validate the column offline: json.loads(tools) and isinstance(tools, list) for all rows; repair or drop failures.
3. Remove markdown fences and prose around the JSON; ensure double quotes and no trailing commas.
4. If a sample genuinely has no tools, use an empty string or empty list instead of a textual note.

Example fix

# before
"tools": "[{'name': 'get_weather', 'parameters': {'city': 'str'}}]"

# after
"tools": "[{\"name\": \"get_weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"city\": {\"type\": \"string\"}}}}]"

import json

def tools_column_ok(rows: list[dict]) -> list[int]:
    bad = []
    for i, r in enumerate(rows):
        tools = r.get("tools")
        if tools:
            try:
                if not isinstance(json.loads(tools), list):
                    bad.append(i)
            except json.JSONDecodeError:
                bad.append(i)
    return bad

def is_valid_tools_json(tools: str) -> bool:
    try:
        parsed = json.loads(tools)
        return isinstance(parsed, list) and all(isinstance(t, dict) and "name" in t for t in parsed)
    except (json.JSONDecodeError, TypeError):
        return False

try:
    json.loads(tools_field)
except json.JSONDecodeError:
    # rewrite row with "" (no tools) or drop it, and record the index
    pass

When it happens

Trigger: Running SFT on a sharegpt dataset whose `tools` field is plain text, a JSON object instead of a list, truncated JSON, or JSON wrapped in markdown fences; happens at the first sample containing a non-empty tools field when a tool_format is configured.

Common situations: Hand-written tools descriptions in natural language; datasets exported from OpenAI-format conversations where tools were serialized with single quotes or extra commas; mixing tool schema styles across rows.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Invalid JSON format in function message: {str([content])}.
• Unknown identifier: {node.id}
• tool_call value is not valid JSON: {content['value']!r}
• tools is not valid JSON: {tools!r}
• Invalid tools

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/0391501d34d81a0e. Report an issue: GitHub.