hiyouga/LlamaFactory · error · ValueError

GLM-4 does not support parallel functions.

Error message

GLM-4 does not support parallel functions.

What it means

GLM-4's function formatter renders exactly one function call per assistant message (its chat format has no separator for multiple parallel calls). If the parsed FunctionCall list contains more than one entry, GLM4ToolUtils.function_formatter raises ValueError before the text is serialized into training data.

Source

Thrown at src/llamafactory/data/tool_utils.py:392

    r"""GLM-4 tool using template."""

    @override
    @staticmethod
    def tool_formatter(tools: list[dict[str, Any]]) -> str:
        tool_text = ""
        for tool in tools:
            tool = tool.get("function", "") if tool.get("type") == "function" else tool
            tool_text += "\n\n## {name}\n\n{body}\n在调用上述函数时，请使用 Json 格式表示调用的参数。".format(
                name=tool["name"], body=json.dumps(tool, indent=4, ensure_ascii=False)
            )

        return GLM4_TOOL_PROMPT.format(tool_text=tool_text)

    @override
    @staticmethod
    def function_formatter(functions: list["FunctionCall"]) -> str:
        if len(functions) > 1:
            raise ValueError("GLM-4 does not support parallel functions.")

        return f"{functions[0].name}\n{functions[0].arguments}"

    @override
    @staticmethod
    def tool_extractor(content: str) -> Union[str, list["FunctionCall"]]:
        if "\n" not in content:
            return content

        tool_name, tool_input = content.split("\n", maxsplit=1)
        try:
            arguments = json.loads(tool_input.strip())
        except json.JSONDecodeError:
            return content

        return [FunctionCall(tool_name, json.dumps(arguments, ensure_ascii=False))]

View on GitHub (pinned to f28afaf635)

Solutions

1. Split multi-call assistant turns into separate turns, or keep only the first call when preparing data for GLM-4.
2. Switch tool_format to one that supports parallel calls (e.g. qwen) if your model/format allows it.
3. Pre-scan the dataset for len(functions) > 1 and clean those samples (see validationCode).

Example fix

// before
{"from": "gpt", "value": "[{\"name\": \"search\", ...}, {\"name\": \"weather\", ...}]"}

// after
{"from": "gpt", "value": "[{\"name\": \"search\", ...}]"}

import json

for sample in dataset:
    for turn in sample["conversations"]:
        if turn["from"] == "gpt":
            try:
                calls = json.loads(turn["value"])
            except (json.JSONDecodeError, TypeError):
                continue
            if isinstance(calls, list) and len(calls) > 1:
                raise ValueError(f"parallel tool calls unsupported by glm4: {sample}")

When it happens

Trigger: Using tool_format: glm4 with a dataset whose assistant messages contain multiple tool calls in one turn (e.g. [{'name': 'f1', ...}, {'name': 'f2', ...}] after JSON parsing); data converted from OpenAI parallel-function-call format; model outputs with several calls pasted as training targets.

Common situations: Tool-calling datasets generated from GPT-4 traces that use parallel calls; switching tool_format from qwen (supports multiple calls) to glm4 without cleaning data.

Related errors

• Invalid tools
• Cannot stream function calls.
• Invalid JSON format in tool description: {str([content])}.
• The length of packed example should be identical to the cuto
• Input must be string, set[str] or dict[str, str], got {type(

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/498ac4a7be5545a7. Report an issue: GitHub.