sgl-project/sglang · error · ValueError

expected a JSON array of tool calls, got {type(tool_call_dat

Error message

expected a JSON array of tool calls, got {type(tool_call_data).__name__}

What it means

In the JSON-fallback tool-call path, SGLang parsed the model's text output as JSON but the top-level value was neither a JSON object nor an array (e.g. a string or number). The parser expected [{"name": ..., "parameters": ...}, ...] and raises this ValueError during _process_tool_calls.

Source

Thrown at python/sglang/srt/entrypoints/openai/serving_chat.py:2169

                    "Required tool call missing from %s output (%d chars)",
                    self.tool_call_parser,
                    len(text),
                )
                logger.debug("Unparsed required tool call output: %r", text[:2000])
                return ToolCallProcessingResult(None, text, finish_reason)

        # json_schema constraint → JSON array output for required/named
        if is_required:
            original_finish_type = finish_reason["type"]
            if finish_reason["type"] == "stop":
                finish_reason["type"] = "tool_calls"
                finish_reason["matched"] = None
            try:
                tool_call_data = orjson.loads(text)
                if isinstance(tool_call_data, dict):
                    tool_call_data = [tool_call_data]
                if not isinstance(tool_call_data, list):
                    raise ValueError(
                        "expected a JSON array of tool calls, got "
                        f"{type(tool_call_data).__name__}"
                    )
                if not all(
                    isinstance(tool, dict) and "name" in tool for tool in tool_call_data
                ):
                    raise ValueError(
                        "every tool call must be a JSON object with a 'name'"
                    )
                tool_calls = []
                for i, tool in enumerate(tool_call_data):
                    parameters = json.dumps(
                        tool.get("parameters", {}), ensure_ascii=False
                    )
                    call_info = ToolCallItem(
                        tool_index=i,
                        name=tool["name"],
                        parameters=parameters,

View on GitHub (pinned to 0132848349)

Solutions

1. Use a model + --tool-call-parser pair that reliably emits the expected JSON format
2. If output is fenced/truncated, increase max_tokens or strip markdown fences in the parser's normalizer
3. Catch the error server-side and treat the message as plain content (the surrounding code already logs and drops some malformed tool calls)
4. Validate model output format with a small smoke test before deploying tool use

Example fix

# before
[{"name": "get_weather", ...}]```   # trailing fence -> orjson yields list OK; but '"[{...}]"' string fails
# after
[{"name": "get_weather", "parameters": {"city": "SF"}}]

def parse_tool_json(text):
    data = orjson.loads(text)
    if isinstance(data, dict): data = [data]
    assert isinstance(data, list), 'not a JSON array'
    return data

def is_tool_array(v) -> bool:
    return isinstance(v, list) and all(isinstance(t, dict) and "name" in t for t in v)

try: resp = client.chat.completions.create(tools=[...])
except BadRequestError: retry with stricter tool prompt or no tools

When it happens

Trigger: A tool-calling request where the model emits malformed tool-call JSON (e.g. "`[{...}]`" with markdown fences left after stripping, or a bare scalar), and no native parser handled it, so the JSON fallback path in serving_chat.py:2169 runs.

Common situations: Weak models producing wrapped/fenced JSON; a custom function-call parser whose normalizer doesn't strip code fences; truncated outputs from max_tokens hitting mid-JSON.

Related errors

• every tool call must be a JSON object with a 'name'
• Invalid request body: {e}
• No call message found for {call_id}
• Invalid messages at {index}: {assistant_msg}
• No tool calls but found tool output

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b3858d8c0bda2780. Report an issue: GitHub.