HKUDS/DeepTutor · error · GraphRagStructuredOutputTruncatedError

graphrag_model_output_truncated

graphrag_model_output_truncated

Error message

GraphRAG compatibility could not be verified because the model response reached its output token limit. Try again.

What it means

GraphRagStructuredOutputTruncatedError (code graphrag_model_output_truncated): the model's response to a structured-output request failed JSON/pydantic validation AND finish_reason was 'length' or 'max_tokens', meaning the JSON was cut off by the output token limit. Raised from _format_response in the GraphRAG completion adapter.

Source

Thrown at deeptutor/services/rag/pipelines/graphrag/completion_adapter.py:127

    fallback = dict(kwargs)
    fallback["messages"] = _messages_with_schema(fallback["messages"], response_format)
    fallback.pop("response_format", None)
    return fallback


def _format_response(response: Any, response_format: type[BaseModel]) -> Any:
    from graphrag_llm.utils import structure_completion_response

    try:
        response.formatted_response = structure_completion_response(
            response.content,
            response_format,
        )
    except (json.JSONDecodeError, TypeError, ValidationError, ValueError) as error:
        choices = getattr(response, "choices", None)
        finish_reason = getattr(choices[0], "finish_reason", None) if choices else None
        if finish_reason in {"length", "max_tokens"}:
            raise GraphRagStructuredOutputTruncatedError(MODEL_OUTPUT_TRUNCATED_MESSAGE) from error
        raise GraphRagStructuredOutputError(MODEL_INCOMPATIBLE_MESSAGE) from error
    return response


def _native_validation_error(error: BaseException) -> bool:
    return isinstance(error, (json.JSONDecodeError, ValidationError))


def _fallback_sync(instance: Any, kwargs: dict[str, Any], response_format: type[BaseModel]) -> Any:
    fallback = _format_fallback_kwargs(kwargs, response_format)
    if fallback.get("stream"):
        raise ValueError("response_format is not supported for streaming completions.")
    messages = fallback.pop("messages")
    if isinstance(messages, str):
        messages = [{"role": "user", "content": messages}]
    request_metrics = fallback.pop("metrics", None) or {}
    if not instance._track_metrics:
        request_metrics = None

View on GitHub (pinned to 3e82f13042)

Solutions

Raise max_tokens / output token limit for the chat model profile used by GraphRAG.
Switch to a model with larger output capacity or shorter JSON output (smaller chunk size).
Retry — the message itself says transient truncation can succeed on re-run.
Use a model that supports native response_format/structured outputs so the fallback path isn't used.

Example fix

# before
llm_cfg.max_tokens = 512
# after
llm_cfg.max_tokens = 4096

Defensive patterns

Strategy: retry

Try / catch

try:
    result = await build(root)
except GraphRagStructuredOutputTruncatedError:
    raise RetryableError("raise max_tokens, then retry") from None

Prevention

Set generous max_tokens on models used for GraphRAG.
Run the completion probe before starting a long indexing job.

When it happens

Trigger: A chat model with a small max_tokens setting answers a GraphRAG structured request (entity extraction, probe completion) and the valid JSON gets truncated; long documents producing large extraction payloads; fallback (non-native response_format) parsing path.

Common situations: Provider profiles with low max_output_tokens defaults; reasoning models burning tokens on hidden reasoning before emitting JSON; local/self-hosted models with tiny context output caps.

Related errors

AI-assisted analysis of HKUDS/DeepTutor@3e82f13042 (2026-08-27). Data as JSON: /api/errors/502ed860f9c8c0bd. Report an issue: GitHub.