{"record":{"id":"fa0d55e858c8c0c7","repo":"docling-project/docling","slug":"kserve-v2-output-tensor-raw-output-name-did-not","errorCode":null,"errorMessage":"KServe v2 output tensor {raw_output.name} did not include inline data.","messagePattern":"KServe v2 output tensor (.+?) did not include inline data\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_http.py","lineNumber":104,"sourceCode":"    \"\"\"KServe v2 infer response payload.\"\"\"\n\n    outputs: List[KserveV2OutputTensor]\n\n\ndef _decode_output_tensor(raw_output: KserveV2OutputTensor) -> np.ndarray:\n    shape = tuple(int(dim) for dim in raw_output.shape)\n    np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(raw_output.datatype)\n    if np_dtype is None:\n        raise RuntimeError(\n            f\"Unsupported KServe v2 output datatype: {raw_output.datatype}. \"\n            f\"Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}\"\n        )\n\n    if raw_output.data is not None:\n        array = np.asarray(raw_output.data, dtype=np_dtype)\n        return array.reshape(shape)\n\n    raise RuntimeError(\n        f\"KServe v2 output tensor {raw_output.name} did not include inline data.\"\n    )\n\n\ndef _decode_binary_output_tensor(\n    raw_output: KserveV2OutputTensor, raw_payload: bytes\n) -> np.ndarray:\n    np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(raw_output.datatype)\n    if np_dtype is None:\n        raise RuntimeError(\n            f\"Unsupported KServe v2 output datatype: {raw_output.datatype}. \"\n            f\"Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}\"\n        )\n\n    shape = tuple(int(dim) for dim in raw_output.shape)\n    if raw_output.datatype == \"BYTES\":\n        return decode_bytes_tensor(raw_payload, shape)\n","sourceCodeStart":86,"sourceCodeEnd":122,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_http.py#L86-L122","documentation":"On the JSON HTTP path, an output tensor arrived with data=None. The KServe v2 spec says non-binary responses must carry inline data for every output; an omitted data field usually means the server actually answered in binary form (payload in the body, tensor carrying a binary_data_size parameter instead of data) or the response is truncated/malformed.","triggerScenarios":"use_binary_data=False on the client while the server is configured to always respond with binary tensors; a proxy rewrites or trims the JSON body; a buggy predictor that omits data for zero-size tensors.","commonSituations":"Mismatched binary settings between client engine options and server/model config; KServe predictors that force binary outputs regardless of request parameters; content-transforming middlewares.","solutions":["Enable use_binary_data=True on the HTTP engine so binary responses are decoded via the binary_data_size path","If binary must stay off, configure the server/model to return inline JSON data (disable binary output in the predictor or model config)","Capture the raw response body and check whether outputs[].parameters contains binary_data_size - that confirms a binary answer"],"exampleFix":"// before\nclient = KserveV2HttpClient(..., use_binary_data=False)\n\n// after\nclient = KserveV2HttpClient(..., use_binary_data=True)","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    outputs = client.infer(inputs=inputs, output_names=[...])\nexcept RuntimeError as e:\n    if \"did not include inline data\" in str(e):\n        client = replace(client, use_binary_data=True)\n        outputs = client.infer(inputs=inputs, output_names=[...])\n    else:\n        raise","preventionTips":["Match the client's use_binary_data flag with the server's actual response mode, verified by one smoke request","Watch for servers that force binary outputs regardless of request parameters","Log the raw response once when onboarding a new model to see its response mode"],"tags":["http","kserve","binary-data","protocol-mismatch"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}