docling-project/docling · error · RuntimeError

KServe v2 HTTP response included trailing binary output data

Error message

KServe v2 HTTP response included trailing binary output data that was not consumed: {len(raw_body) - raw_offset} bytes

What it means

After consuming all tensors that declared binary_data_size, bytes remain unconsumed in the binary body (raw_offset != len(raw_body)). The KServe v2 binary extension requires the binary payloads to be exactly the concatenation of the declared per-tensor sizes; leftover bytes mean the sizes undercount what was sent, or the header-length slicing misaligned the body start.

Source

Thrown at docling/models/inference_engines/common/kserve_v2_http.py:431

            binary_data_size = _parse_binary_data_size(output.parameters)
            if binary_data_size is None:
                decoded_outputs[output.name] = _decode_output_tensor(output)
                continue

            raw_end = raw_offset + binary_data_size
            if raw_end > len(raw_body):
                raise RuntimeError(
                    "KServe v2 HTTP response did not include enough binary output data "
                    f"for tensor {output.name}: expected {binary_data_size} bytes at "
                    f"offset {raw_offset}, got {len(raw_body) - raw_offset}"
                )
            decoded_outputs[output.name] = _decode_binary_output_tensor(
                output, raw_body[raw_offset:raw_end]
            )
            raw_offset = raw_end

        if raw_offset != len(raw_body):
            raise RuntimeError(
                "KServe v2 HTTP response included trailing binary output data that was "
                f"not consumed: {len(raw_body) - raw_offset} bytes"
            )

        if _log.isEnabledFor(logging.DEBUG):
            _log.debug(
                "PIPELINE_PROFILING KServe infer deserialization: batch_size=%d start=%.3f end=%.3f duration=%.3fs",
                _batch_size,
                _t_deser_start,
                time.time(),
                time.monotonic() - _t_deser_mono,
            )

        return decoded_outputs

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Compare the sum of declared binary_data_size values with len(raw_body) to confirm the accounting gap
2. Fix the server-side binary_data_size computation to match the bytes actually written
3. Check the Inference-Header-Content-Length value against the true JSON header length
4. Disable binary transport while the server's binary implementation is non-conformant

try:
    outputs = client.infer(inputs=inputs, output_names=[...])
except RuntimeError as e:
    if "trailing binary output data" in str(e):
        client = replace(client, use_binary_data=False)
        outputs = client.infer(inputs=inputs, output_names=[...])
    else:
        raise

When it happens

Trigger: Server understates one or more binary_data_size values; an extra binary payload appended for a tensor that did not declare one; Inference-Header-Content-Length overcounting so raw_body starts too early; predictor emitting padding bytes.

Common situations: Custom KServe runtimes with sloppy size accounting; padding/alignment added by an intermediary; evolving server code during a rolling deploy.

Related errors

• KServe v2 output tensor {raw_output.name} did not include in
• Invalid binary_data_size value: {size!r}
• Invalid binary_data_size value: {parsed_size}
• Invalid binary inference response header from {response.url}
• KServe v2 HTTP response did not include enough binary output

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/1eb5a0830fe8a303. Report an issue: GitHub.