{"record":{"id":"43eaa0e31fa9c88f","repo":"docling-project/docling","slug":"kserve-v2-grpc-response-did-not-include-binary-out","errorCode":null,"errorMessage":"KServe v2 gRPC response did not include binary output payloads for all tensors. Set use_binary_data=False or ensure server supports binary_data outputs.","messagePattern":"KServe v2 gRPC response did not include binary output payloads for all tensors\\. Set use_binary_data=False or ensure server supports binary_data outputs\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_grpc.py","lineNumber":366,"sourceCode":"                f\"gRPC infer call failed for model {self.model_name}: {exc}\"\n            ) from exc\n\n        if _log.isEnabledFor(logging.DEBUG):\n            _log.debug(\n                \"PIPELINE_PROFILING KServe gRPC infer round-trip: batch_size=%d start=%.3f end=%.3f duration=%.3fs\",\n                _batch_size,\n                _t_grpc_start,\n                time.time(),\n                time.monotonic() - _t_grpc_mono,\n            )\n            _t_deser_start = time.time()\n            _t_deser_mono = time.monotonic()\n\n        decoded_outputs: Dict[str, np.ndarray] = {}\n\n        if self.use_binary_data:\n            if len(response.raw_output_contents) != len(response.outputs):\n                raise RuntimeError(\n                    \"KServe v2 gRPC response did not include binary output payloads for all tensors. \"\n                    \"Set use_binary_data=False or ensure server supports binary_data outputs.\"\n                )\n            for output_tensor, raw_output in zip(\n                response.outputs, response.raw_output_contents\n            ):\n                np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(output_tensor.datatype)\n                if np_dtype is None:\n                    raise RuntimeError(\n                        f\"Unsupported KServe v2 gRPC output datatype: {output_tensor.datatype}. \"\n                        f\"Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}\"\n                    )\n                shape = tuple(int(dim) for dim in output_tensor.shape)\n\n                # Bytes decoding\n                # Special handling for BYTES datatype (variable-length strings)\n                if output_tensor.datatype == \"BYTES\":\n                    decoded_outputs[output_tensor.name] = decode_bytes_tensor(","sourceCodeStart":348,"sourceCodeEnd":384,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_grpc.py#L348-L384","documentation":"The client sent binary_data=True for outputs, but the gRPC response declared N tensors in response.outputs while response.raw_output_contents has fewer entries. This means the server acknowledged the binary output request but did not attach a binary payload for every tensor, so the zip-based decode loop would silently drop tensors; the client refuses instead.","triggerScenarios":"use_binary_data=True against a KServe/Triton build that ignores or partially honors the binary_data output parameter; a server that only returns binary for some output tensors (e.g. mixed BYTES + FP32 outputs); a proxy/serializer between client and server dropping raw_output_contents fields.","commonSituations":"Pointing the engine at an older Triton or a custom KServe runtime without full binary-data support; version drift after upgrading the inference server; enabling use_binary_data for latency on a deployment that was never tested with it.","solutions":["Set use_binary_data=False on the gRPC engine options so outputs are decoded from the inline tensor contents","If binary transport is required, verify with a standalone grpcurl/Triton client that the server fills raw_output_contents for every output, and upgrade/reconfigure the server","Check whether any intermediary (envoy, custom gRPC gateway) strips unknown fields"],"exampleFix":"// before\nengine = KserveV2GrpcClient(..., use_binary_data=True)\n\n// after\nengine = KserveV2GrpcClient(..., use_binary_data=False)","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    outputs = engine.infer(inputs=inputs)\nexcept RuntimeError as e:\n    if \"binary output payloads\" in str(e):\n        engine = replace(engine, use_binary_data=False)\n        outputs = engine.infer(inputs=inputs)\n    else:\n        raise","preventionTips":["Smoke-test use_binary_data=True against your server build before enabling it in production","Keep a non-binary engine configuration ready as a fallback","Verify the server version supports the KServe binary tensor extension for outputs"],"tags":["grpc","kserve","binary-data","protocol-mismatch"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}