HKUDS/DeepTutor · error · GraphRagStructuredOutputError

graphrag_model_incompatible

graphrag_model_incompatible

Error message

The model did not accept or return the structured output required by GraphRAG.

What it means

GraphRagStructuredOutputError (code graphrag_model_incompatible): the model's response to a GraphRAG structured-output request could not be parsed/validated (json.JSONDecodeError, TypeError, pydantic ValidationError, ValueError) and truncation was not the cause. The model either ignored the schema or returned malformed JSON.

Solutions

1. Switch the active chat model to one with verified structured-output support (recent OpenAI/Anthropic/DeepSeek models).
2. Enable/force JSON mode on the provider endpoint.
3. Update the model or serving stack so response_format is honored.
4. If using a proxy, verify it forwards the response_format field.

Example fix

# before: model ignores response_format
model = "local-mistral-7b"
# after
model = "gpt-4o"  # or another model with native structured output

try:
    await build(root)
except GraphRagStructuredOutputError as e:
    if e.code == "graphrag_model_incompatible":
        switch_to_structured_output_model()
    raise

When it happens

Trigger: A chat model that doesn't support response_format / structured outputs returns prose or schema-violating JSON during GraphRAG indexing or the probe (_probe_completion_model_impl / _fallback_sync / _fallback_async).

Common situations: Using an OpenAI-compatible proxy or local model (llama.cpp, vLLM, older TGI) that ignores json_schema response_format; models that wrap JSON in markdown fences; mismatched pydantic schema expectations across GraphRAG versions.

Related errors

• graphrag_model_incompatible
• graphrag_model_output_truncated
• Anthropic API error
• Anthropic API error: unexpected response payload
• Anthropic API key is missing from the active LLM profile.

AI-assisted analysis of HKUDS/DeepTutor@3e82f13042 (2026-08-27). Data as JSON: /api/errors/d4dcb90c09ffcb4b. Report an issue: GitHub.

Appendix: source

Thrown at deeptutor/services/rag/pipelines/graphrag/completion_adapter.py:128

    fallback["messages"] = _messages_with_schema(fallback["messages"], response_format)
    fallback.pop("response_format", None)
    return fallback


def _format_response(response: Any, response_format: type[BaseModel]) -> Any:
    from graphrag_llm.utils import structure_completion_response

    try:
        response.formatted_response = structure_completion_response(
            response.content,
            response_format,
        )
    except (json.JSONDecodeError, TypeError, ValidationError, ValueError) as error:
        choices = getattr(response, "choices", None)
        finish_reason = getattr(choices[0], "finish_reason", None) if choices else None
        if finish_reason in {"length", "max_tokens"}:
            raise GraphRagStructuredOutputTruncatedError(MODEL_OUTPUT_TRUNCATED_MESSAGE) from error
        raise GraphRagStructuredOutputError(MODEL_INCOMPATIBLE_MESSAGE) from error
    return response


def _native_validation_error(error: BaseException) -> bool:
    return isinstance(error, (json.JSONDecodeError, ValidationError))


def _fallback_sync(instance: Any, kwargs: dict[str, Any], response_format: type[BaseModel]) -> Any:
    fallback = _format_fallback_kwargs(kwargs, response_format)
    if fallback.get("stream"):
        raise ValueError("response_format is not supported for streaming completions.")
    messages = fallback.pop("messages")
    if isinstance(messages, str):
        messages = [{"role": "user", "content": messages}]
    request_metrics = fallback.pop("metrics", None) or {}
    if not instance._track_metrics:
        request_metrics = None
    try:

View on GitHub (pinned to 3e82f13042)