docling-project/docling · error · RuntimeError

KServe v2 output tensor {raw_output.name} did not include in

Error message

KServe v2 output tensor {raw_output.name} did not include inline data.

What it means

On the JSON HTTP path, an output tensor arrived with data=None. The KServe v2 spec says non-binary responses must carry inline data for every output; an omitted data field usually means the server actually answered in binary form (payload in the body, tensor carrying a binary_data_size parameter instead of data) or the response is truncated/malformed.

Source

Thrown at docling/models/inference_engines/common/kserve_v2_http.py:104

    """KServe v2 infer response payload."""

    outputs: List[KserveV2OutputTensor]


def _decode_output_tensor(raw_output: KserveV2OutputTensor) -> np.ndarray:
    shape = tuple(int(dim) for dim in raw_output.shape)
    np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(raw_output.datatype)
    if np_dtype is None:
        raise RuntimeError(
            f"Unsupported KServe v2 output datatype: {raw_output.datatype}. "
            f"Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}"
        )

    if raw_output.data is not None:
        array = np.asarray(raw_output.data, dtype=np_dtype)
        return array.reshape(shape)

    raise RuntimeError(
        f"KServe v2 output tensor {raw_output.name} did not include inline data."
    )


def _decode_binary_output_tensor(
    raw_output: KserveV2OutputTensor, raw_payload: bytes
) -> np.ndarray:
    np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(raw_output.datatype)
    if np_dtype is None:
        raise RuntimeError(
            f"Unsupported KServe v2 output datatype: {raw_output.datatype}. "
            f"Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}"
        )

    shape = tuple(int(dim) for dim in raw_output.shape)
    if raw_output.datatype == "BYTES":
        return decode_bytes_tensor(raw_payload, shape)

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Enable use_binary_data=True on the HTTP engine so binary responses are decoded via the binary_data_size path
2. If binary must stay off, configure the server/model to return inline JSON data (disable binary output in the predictor or model config)
3. Capture the raw response body and check whether outputs[].parameters contains binary_data_size - that confirms a binary answer

Example fix

// before
client = KserveV2HttpClient(..., use_binary_data=False)

// after
client = KserveV2HttpClient(..., use_binary_data=True)

try:
    outputs = client.infer(inputs=inputs, output_names=[...])
except RuntimeError as e:
    if "did not include inline data" in str(e):
        client = replace(client, use_binary_data=True)
        outputs = client.infer(inputs=inputs, output_names=[...])
    else:
        raise

When it happens

Trigger: use_binary_data=False on the client while the server is configured to always respond with binary tensors; a proxy rewrites or trims the JSON body; a buggy predictor that omits data for zero-size tensors.

Common situations: Mismatched binary settings between client engine options and server/model config; KServe predictors that force binary outputs regardless of request parameters; content-transforming middlewares.

Related errors

• KServe v2 HTTP response included trailing binary output data
• KServe v2 gRPC response did not include binary output payloa
• Invalid binary_data_size value: {size!r}
• Invalid binary_data_size value: {parsed_size}
• Binary KServe response from {response.url} did not include {

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/fa0d55e858c8c0c7. Report an issue: GitHub.