docling-project/docling · error · RuntimeError

KServe v2 HTTP response did not include enough binary output

Error message

KServe v2 HTTP response did not include enough binary output data for tensor {output.name}: expected {binary_data_size} bytes at offset {raw_offset}, got {len(raw_body) - raw_offset}

What it means

During binary HTTP response slicing: the header JSON declared a tensor with binary_data_size N at cumulative offset raw_offset, but the remaining body after the JSON header is shorter than N bytes. The client refuses to read past the buffer. Indicates truncated payloads or binary_data_size values inconsistent with what was actually sent.

Source

Thrown at docling/models/inference_engines/common/kserve_v2_http.py:420

            raise RuntimeError(
                f"Invalid inference response from {self.infer_url}: {exc}"
            ) from exc

        decoded_outputs: Dict[str, np.ndarray] = {}
        header_len_text = response.headers.get(_INFERENCE_HEADER_CONTENT_LENGTH)
        raw_body = b""
        if self.use_binary_data and header_len_text is not None:
            raw_body = response.content[int(header_len_text) :]
        raw_offset = 0
        for output in body.outputs:
            binary_data_size = _parse_binary_data_size(output.parameters)
            if binary_data_size is None:
                decoded_outputs[output.name] = _decode_output_tensor(output)
                continue

            raw_end = raw_offset + binary_data_size
            if raw_end > len(raw_body):
                raise RuntimeError(
                    "KServe v2 HTTP response did not include enough binary output data "
                    f"for tensor {output.name}: expected {binary_data_size} bytes at "
                    f"offset {raw_offset}, got {len(raw_body) - raw_offset}"
                )
            decoded_outputs[output.name] = _decode_binary_output_tensor(
                output, raw_body[raw_offset:raw_end]
            )
            raw_offset = raw_end

        if raw_offset != len(raw_body):
            raise RuntimeError(
                "KServe v2 HTTP response included trailing binary output data that was "
                f"not consumed: {len(raw_body) - raw_offset} bytes"
            )

        if _log.isEnabledFor(logging.DEBUG):
            _log.debug(
                "PIPELINE_PROFILING KServe infer deserialization: batch_size=%d start=%.3f end=%.3f duration=%.3fs",

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Log len(response.content), the header length, and each tensor's binary_data_size to find which number is wrong
2. Remove response-size limits and transparent content-encoding on the proxy path
3. If server-side sizes are wrong, fix the predictor; if transport truncates, fix the network layer
4. Temporarily set use_binary_data=False to keep working while diagnosing

try:
    outputs = client.infer(inputs=inputs, output_names=[...])
except RuntimeError as e:
    if "did not include enough binary output data" in str(e):
        client = replace(client, use_binary_data=False)
        outputs = client.infer(inputs=inputs, output_names=[...])
    else:
        raise

When it happens

Trigger: Response truncated in transit (proxy size limit, connection cut); server computing binary_data_size before some transformation (gzip re-encode) changes byte counts; multiple binary tensors where an earlier size is overstated, shifting the budget; Inference-Header-Content-Length undercounting the JSON header so raw_body starts mid-JSON.

Common situations: Large vision/LLM outputs hitting gateway body limits; custom predictors with off-by-N size math; content-encoding transformations applied by CDNs.

Related errors

• Invalid binary_data_size value: {size!r}
• Invalid binary_data_size value: {parsed_size}
• Invalid {_INFERENCE_HEADER_CONTENT_LENGTH} value: {header_le
• Invalid binary inference response header from {response.url}
• KServe v2 HTTP response included trailing binary output data

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/f4c234031241872a. Report an issue: GitHub.