docling-project/docling · error · RuntimeError

KServe v2 gRPC response did not include binary output payloa

Error message

KServe v2 gRPC response did not include binary output payloads for all tensors. Set use_binary_data=False or ensure server supports binary_data outputs.

What it means

The client sent binary_data=True for outputs, but the gRPC response declared N tensors in response.outputs while response.raw_output_contents has fewer entries. This means the server acknowledged the binary output request but did not attach a binary payload for every tensor, so the zip-based decode loop would silently drop tensors; the client refuses instead.

Source

Thrown at docling/models/inference_engines/common/kserve_v2_grpc.py:366

                f"gRPC infer call failed for model {self.model_name}: {exc}"
            ) from exc

        if _log.isEnabledFor(logging.DEBUG):
            _log.debug(
                "PIPELINE_PROFILING KServe gRPC infer round-trip: batch_size=%d start=%.3f end=%.3f duration=%.3fs",
                _batch_size,
                _t_grpc_start,
                time.time(),
                time.monotonic() - _t_grpc_mono,
            )
            _t_deser_start = time.time()
            _t_deser_mono = time.monotonic()

        decoded_outputs: Dict[str, np.ndarray] = {}

        if self.use_binary_data:
            if len(response.raw_output_contents) != len(response.outputs):
                raise RuntimeError(
                    "KServe v2 gRPC response did not include binary output payloads for all tensors. "
                    "Set use_binary_data=False or ensure server supports binary_data outputs."
                )
            for output_tensor, raw_output in zip(
                response.outputs, response.raw_output_contents
            ):
                np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(output_tensor.datatype)
                if np_dtype is None:
                    raise RuntimeError(
                        f"Unsupported KServe v2 gRPC output datatype: {output_tensor.datatype}. "
                        f"Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}"
                    )
                shape = tuple(int(dim) for dim in output_tensor.shape)

                # Bytes decoding
                # Special handling for BYTES datatype (variable-length strings)
                if output_tensor.datatype == "BYTES":
                    decoded_outputs[output_tensor.name] = decode_bytes_tensor(

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Set use_binary_data=False on the gRPC engine options so outputs are decoded from the inline tensor contents
2. If binary transport is required, verify with a standalone grpcurl/Triton client that the server fills raw_output_contents for every output, and upgrade/reconfigure the server
3. Check whether any intermediary (envoy, custom gRPC gateway) strips unknown fields

Example fix

// before
engine = KserveV2GrpcClient(..., use_binary_data=True)

// after
engine = KserveV2GrpcClient(..., use_binary_data=False)

try:
    outputs = engine.infer(inputs=inputs)
except RuntimeError as e:
    if "binary output payloads" in str(e):
        engine = replace(engine, use_binary_data=False)
        outputs = engine.infer(inputs=inputs)
    else:
        raise

When it happens

Trigger: use_binary_data=True against a KServe/Triton build that ignores or partially honors the binary_data output parameter; a server that only returns binary for some output tensors (e.g. mixed BYTES + FP32 outputs); a proxy/serializer between client and server dropping raw_output_contents fields.

Common situations: Pointing the engine at an older Triton or a custom KServe runtime without full binary-data support; version drift after upgrading the inference server; enabling use_binary_data for latency on a deployment that was never tested with it.

Related errors

• Unsupported KServe v2 gRPC output datatype: {output_tensor.d
• KServe v2 output tensor {raw_output.name} did not include in
• KServe v2 HTTP response included trailing binary output data
• Unsupported KServe request parameter integer range for gRPC:
• Unsupported KServe request parameter type for gRPC: key={key

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/43eaa0e31fa9c88f. Report an issue: GitHub.