zylon-ai/private-gpt · error · ValueError

Tools are not supported when response_format is set to json_

Error message

Tools are not supported when response_format is set to json_schema

What it means

ChatBody's structured-output guard: when response_format.type is json_schema AND tools are present, the request is rejected. Structured output via json_schema and function/tool calling are treated as mutually exclusive — the response must be schema-conformant JSON, which tool calls would break.

Source

Thrown at private_gpt/server/chat/chat_models.py:359

            )

        # Apply global tool context to tools without specific context
        if self.tools is not None:
            global_tool_context = self.tool_context or []
            if global_tool_context:
                for tool in self.tools:
                    if tool.context is None:
                        tool.context = global_tool_context

        has_structured_output = bool(self.output_config and self.output_config.format)
        if self.response_format.type == ResponseFormatType.json_schema:
            has_structured_output = True

        # Check that we don't have tools when structured output is enabled
        if has_structured_output:
            if self.tools:
                if self.response_format.type == ResponseFormatType.json_schema:
                    raise ValueError(
                        "Tools are not supported when response_format is set to json_schema"
                    )
                raise ValueError(
                    "Tools are not supported when structured output is enabled."
                )
            if self.mcp_servers:
                raise ValueError(
                    "MCP servers are not supported when structured output is enabled."
                )
            if system.citations.enabled:
                raise ValueError(
                    "Citations are not supported when structured output is enabled."
                )

        # Check unique tools
        if self.tools:
            tool_names = [tool.name for tool in self.tools]
            if len(tool_names) != len(set(tool_names)):

View on GitHub (pinned to 4a030776a3)

Solutions

Remove tools (and mcp_servers) from the request when response_format.type is json_schema.
Or drop the json_schema response_format and instead define a tool with the desired schema and tool_choice forcing it (the classic function-calling way to get structured data).
Gate the two features on mutually exclusive configuration in the client.

Example fix

# before
{"response_format":{"type":"json_schema",...},"tools":[...]}

# after
{"response_format":{"type":"json_schema",...}}  # tools removed

Defensive patterns

Strategy: validation

Validate before calling

if body.get('response_format', {}).get('type') == 'json_schema':
    body.pop('tools', None); body.pop('mcp_servers', None)

Type guard

def json_schema_clean(body: dict) -> bool:
    rf = body.get('response_format', {}).get('type')
    return rf != 'json_schema' or (not body.get('tools') and not body.get('mcp_servers'))

Try / catch

except ValidationError as e:
    if 'json_schema' in str(e):
        body.pop('tools', None); retry()
    else:
        raise

Prevention

Model structured output and tools as mutually exclusive client modes
If you need schema-constrained tool calls, force a tool whose input schema IS your schema
Assert mode exclusivity in request-builder tests

When it happens

Trigger: tool_choice={"type":"tool","name":"lookup"} while tools only defines e.g. 'search'; renaming a tool in the tools registry without updating the client's tool_choice; dynamic tool filtering that removes the forced tool from the list while tool_choice still names it.

Common situations: Version skew between client tool names and server-registered tools; feature-flagged tool sets where the forced tool is disabled; typos in the tool name string.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/7e12b64842ab6da4. Report an issue: GitHub.