docling-project/docling · error · RuntimeError

Invalid binary inference response header from {response.url}

Error message

Invalid binary inference response header from {response.url}: {exc}

What it means

_decode_binary_response: the header length was valid, but decoding response.content[:header_len] as UTF-8 JSON and validating it as KserveV2InferResponse failed (json.loads, UnicodeDecodeError, or pydantic ValidationError). The chained exception carries the precise reason. The declared header region does not contain a well-formed infer response.

Source

Thrown at docling/models/inference_engines/common/kserve_v2_http.py:200

            ) from exc

    try:
        header_len = int(header_len_text)
    except ValueError as exc:
        raise RuntimeError(
            f"Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_len_text!r}"
        ) from exc

    if header_len < 0 or header_len > len(response.content):
        raise RuntimeError(
            f"Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_len}"
        )

    try:
        header_json = json.loads(response.content[:header_len].decode("utf-8"))
        return KserveV2InferResponse.model_validate(header_json)
    except Exception as exc:
        raise RuntimeError(
            f"Invalid binary inference response header from {response.url}: {exc}"
        ) from exc


@dataclass(frozen=True)
class KserveV2HttpClient:
    """Minimal client for KServe v2 REST infer requests."""

    base_url: str
    model_name: str
    model_version: Optional[str]
    timeout: float
    headers: Mapping[str, str]
    use_binary_data: bool = True

    def close(self) -> None:
        """No-op close for transport parity with gRPC client."""

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Dump response.content[:header_len] in a repro script and eyeball whether it is complete JSON
2. Verify the server against the KServe v2 binary tensor extension spec (header length must be the exact JSON byte count)
3. Disable transparent gzip at proxies, or the binary mode, while diagnosing
4. Report to the server implementer with the captured header bytes

try:
    outputs = client.infer(inputs=inputs, output_names=[...])
except RuntimeError as e:
    if "Invalid binary inference response header" in str(e):
        raise  # inspect e.__cause__ for the JSON/pydantic error; fix server header
    raise

When it happens

Trigger: Header length off by N bytes (server counting differently), so the slice cuts into/short of the JSON; binary payload preceded by non-JSON bytes; proxy injecting a prefix (BOM, banner) before the header; pydantic validation failing because outputs are missing or mistyped.

Common situations: Custom servers implementing the binary extension slightly wrong; encoding transformations (gzip auto-decompress changing offsets); client pydantic model stricter than what the server emits.

Related errors

• Invalid binary_data_size value: {size!r}
• Invalid binary_data_size value: {parsed_size}
• Binary KServe response from {response.url} did not include {
• KServe v2 HTTP response did not include enough binary output
• KServe v2 HTTP response included trailing binary output data

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/68b1292603ad85dd. Report an issue: GitHub.