docling-project/docling · error · RuntimeError

Binary KServe response from {response.url} did not include {

Error message

Binary KServe response from {response.url} did not include {_INFERENCE_HEADER_CONTENT_LENGTH} and was not valid JSON: {exc}

What it means

In _decode_binary_response: the response lacks the Inference-Header-Content-Length header (which should delimit the JSON header preceding the binary payload), so the client falls back to parsing the whole body as a plain JSON infer response - and that also failed. The chained exception says why the JSON parse/validation failed. Typically the body is an error page/JSON, or a proxy stripped the header.

Source

Thrown at docling/models/inference_engines/common/kserve_v2_http.py:179

    if request_parameters:
        payload["parameters"] = dict(request_parameters)

    header_bytes = json.dumps(payload, separators=(",", ":")).encode("utf-8")
    request_body = header_bytes + b"".join(raw_inputs)
    request_headers = {
        "Content-Type": "application/octet-stream",
        _INFERENCE_HEADER_CONTENT_LENGTH: str(len(header_bytes)),
    }
    return request_headers, request_body


def _decode_binary_response(response: requests.Response) -> KserveV2InferResponse:
    header_len_text = response.headers.get(_INFERENCE_HEADER_CONTENT_LENGTH)
    if header_len_text is None:
        try:
            return KserveV2InferResponse.model_validate(response.json())
        except Exception as exc:
            raise RuntimeError(
                f"Binary KServe response from {response.url} did not include "
                f"{_INFERENCE_HEADER_CONTENT_LENGTH} and was not valid JSON: {exc}"
            ) from exc

    try:
        header_len = int(header_len_text)
    except ValueError as exc:
        raise RuntimeError(
            f"Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_len_text!r}"
        ) from exc

    if header_len < 0 or header_len > len(response.content):
        raise RuntimeError(
            f"Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_len}"
        )

    try:
        header_json = json.loads(response.content[:header_len].decode("utf-8"))

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Replay the request with curl -i and inspect status, headers, and body - almost always the body is an error message, and fixing that error resolves this
2. Ensure the gateway forwards the Inference-Header-Content-Length response header (no underspecified header whitelisting)
3. Confirm base_url/model_name build the correct /v2/models/<model>/infer path
4. If the server genuinely does not support binary responses, set use_binary_data=False

try:
    outputs = client.infer(inputs=inputs, output_names=[...])
except RuntimeError as e:
    if "did not include Inference-Header-Content-Length" in str(e):
        # body is usually a server error - inspect it via a debug proxy and fix root cause
        raise
    raise

When it happens

Trigger: use_binary_data=True but the server returned a JSON error (4xx/5xx body that was still 200 via gateway, or a plain error payload); an intermediary (nginx, API gateway) dropped unknown headers; the endpoint is not actually a KServe v2 infer route.

Common situations: Base URL pointing at the wrong path (e.g. root instead of /v2/models/<m>/infer); auth failures surfaced as JSON error bodies; header-filtering proxies in front of the model server.

Related errors

• Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_le
• Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_le
• Invalid binary inference response header from {response.url}
• Invalid inference response from {self.infer_url}: {exc}
• KServe v2 output tensor {raw_output.name} did not include in

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/a01faf1ea7e72394. Report an issue: GitHub.