BerriAI/litellm · error · JSONSchemaValidationError

litellm.JSONSchemaValidationError: model={model}, returned a

Error message

litellm.JSONSchemaValidationError: model={model}, returned an invalid response={raw_response}, for schema={schema}.\nAccess raw response with `e.raw_response`

What it means

Raised inside LiteLLM's JSON-validation rule when the model's response cannot even be parsed as JSON (json.JSONDecodeError) while validating against a response_format JSON schema. LiteLLM wraps this into JSONSchemaValidationError with model='', the raw response and schema attached, so callers can inspect e.raw_response. It means the LLM returned non-JSON output (prose, markdown fences, or an empty/Partial response) for a request expecting structured JSON.

Source

Thrown at litellm/litellm_core_utils/json_validation_rule.py:114

    return normalized_tool


def validate_schema(schema: dict, response: str):
    """
    Validate if the returned json response follows the schema.

    Params:
    - schema - dict: JSON schema
    - response - str: Received json response as string.
    """
    from jsonschema import ValidationError, validate

    from litellm import JSONSchemaValidationError

    try:
        response_dict: Final = json.loads(response)
    except json.JSONDecodeError:
        raise JSONSchemaValidationError(model="", llm_provider="", raw_response=response, schema=json.dumps(schema))

    try:
        validate(response_dict, schema=schema)
    except ValidationError:
        raise JSONSchemaValidationError(model="", llm_provider="", raw_response=response, schema=json.dumps(schema))

View on GitHub (pinned to 6c2dcb801b)

Solutions

Catch JSONSchemaValidationError and retry the request (optionally with the error appended as corrective feedback) — transient non-JSON replies are common
Use a model/provider combination with native structured-output support (OpenAI json_schema strict mode) instead of client-side validation only
Strip markdown fences / re-parse e.raw_response yourself before giving up
Increase max_tokens so the JSON is not truncated, and simplify the schema

Example fix

# before
resp = litellm.completion(model='gpt-4o', messages=msgs,
    response_format={'type':'json_schema','json_schema':{'name':'out','schema':S,'strict':True}})

# after: retry once on invalid JSON
from litellm import JSONSchemaValidationError
try:
    resp = litellm.completion(...)
except JSONSchemaValidationError as e:
    raw = e.raw_response.strip().removeprefix('```json').removesuffix('```')
    resp = litellm.completion(model='gpt-4o', messages=msgs + [
        {'role':'assistant','content':e.raw_response},
        {'role':'user','content':'Return ONLY valid JSON matching the schema.'}],
        response_format={'type':'json_schema','json_schema':{'name':'out','schema':S,'strict':True}})

Defensive patterns

Strategy: retry

Validate before calling

import json

def parse_model_json(raw: str) -> dict | None:
    try:
        return json.loads(raw)
    except json.JSONDecodeError:
        stripped = raw.strip().removeprefix('```json').removeprefix('```').removesuffix('```').strip()
        try:
            return json.loads(stripped)
        except json.JSONDecodeError:
            return None

Type guard

def is_parseable_json(raw: str) -> bool:
    try:
        json.loads(raw)
        return True
    except (json.JSONDecodeError, TypeError):
        return False

Try / catch

from litellm import JSONSchemaValidationError

for attempt in range(2):
    try:
        resp = litellm.completion(model=m, messages=msgs, response_format=rf)
        break
    except JSONSchemaValidationError as e:
        if attempt == 1:
            raise
        msgs = msgs + [{'role': 'user', 'content': 'Your previous reply was not valid JSON. Return ONLY valid JSON.'}]

Prevention

Prefer providers with native strict json_schema enforcement
Set max_tokens generously to avoid truncated JSON
Always catch JSONSchemaValidationError and use e.raw_response rather than re-calling blindly

When it happens

Trigger: Calling completion with response_format={'type':'json_schema', ...} (or the json_validation enforce rule) where the model's reply is not parseable by json.loads — e.g. '```json\n{...}\n```' with fences, truncated output hitting max_tokens, or a model that ignores the schema.

Common situations: Using weaker models that wrap JSON in markdown; low max_tokens cutting off the JSON; streaming responses validated before completion; schemas without strict mode on providers that don't enforce it natively.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/ed93c845861ff977. Report an issue: GitHub.