sgl-project/sglang · error · ValueError

tool_choice 'required' or a named tool cannot be combined wi

Error message

tool_choice 'required' or a named tool cannot be combined with response_format, regex, or ebnf: the tool-call constraint and the output constraint cannot both be honored.

What it means

to_sampling_params refuses requests that simultaneously force tool calls (tool_choice='required' or a named ToolChoice object) and impose an output constraint (response_format, regex, or ebnf). The grammar for tool-call structure and the user grammar cannot both be satisfied, so the request is rejected rather than silently misbehaving. With tool_choice='auto' it only logs a warning and drops the constraint.

Source

Thrown at python/sglang/srt/entrypoints/openai/protocol.py:1154

            sampling_params["json_schema"] = '{"type": "object"}'
        elif self.response_format and self.response_format.type == "structural_tag":
            sampling_params["structural_tag"] = convert_json_schema_to_str(
                self.response_format.model_dump(by_alias=True)
            )

        # Check if there are already existing output constraints
        has_existing_constraints = (
            sampling_params.get("regex")
            or sampling_params.get("ebnf")
            or sampling_params.get("structural_tag")
            or sampling_params.get("json_schema")
        )

        if tool_call_constraint and has_existing_constraints:
            if self.tool_choice == "required" or isinstance(
                self.tool_choice, ToolChoice
            ):
                raise ValueError(
                    "tool_choice 'required' or a named tool cannot be combined with "
                    "response_format, regex, or ebnf: the tool-call constraint and the "
                    "output constraint cannot both be honored."
                )
            logger.warning("Constrained decoding is not compatible with tool calls.")
        elif tool_call_constraint:
            constraint_type, constraint_value = tool_call_constraint
            if constraint_type == "structural_tag":
                sampling_params[constraint_type] = convert_json_schema_to_str(
                    constraint_value.model_dump(by_alias=True)
                )
            elif constraint_type == "json_schema":
                sampling_params[constraint_type] = convert_json_schema_to_str(
                    constraint_value  # type: ignore
                )
            else:
                sampling_params[constraint_type] = constraint_value

View on GitHub (pinned to 0132848349)

Solutions

1. Remove response_format/regex/ebnf and rely on the tool's parameters JSON schema to constrain arguments
2. Or switch tool_choice to "auto"/"none" if structured output, not tool calling, is the real goal
3. Express argument constraints inside the tool definition's JSON schema instead

Example fix

// before
{"tools": [...], "tool_choice": "required", "response_format": {"type": "json_object"}}
// after
{"tools": [...], "tool_choice": "required"}

forced = body.get("tool_choice") in ("required",) or isinstance(body.get("tool_choice"), dict)
constrained = any(body.get(k) for k in ("response_format", "regex", "ebnf"))
if forced and constrained:
    del body["response_format"]; body.pop("regex", None); body.pop("ebnf", None)

def has_forced_tool_choice(tc): return tc == "required" or (isinstance(tc, dict) and tc.get("type") == "tool")

try: create(...)
except ValueError as e: if 'cannot be combined' in str(e): retry without response_format

When it happens

Trigger: POST /v1/chat/completions with tools=[...] plus tool_choice="required" (or {"type":"function","function":{"name":"f"}}) and also response_format={"type":"json_schema",...} or regex/ebnf fields.

Common situations: Developers wanting schema-guaranteed tool arguments trying to also pass a json_schema response_format; migrating from clients that set response_format globally on all requests; combining structured output examples with function calling examples.

Related errors

• Cannot combine tool calls with constrained decoding (text.fo
• Assistant tool call function.arguments must be valid JSON.
• Assistant tool call function.arguments must be a JSON object
• invalid reasoning effort: {effort!r}
• Value error, parameter top_n should be larger than 0.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/890e1ef13c942261. Report an issue: GitHub.