docling-project/docling · error · RuntimeError

Unsupported KServe v2 output datatype: {raw_output.datatype}

Error message

Unsupported KServe v2 output datatype: {raw_output.datatype}. Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}

What it means

Raised in _decode_output_tensor on the JSON (non-binary) HTTP path: an output tensor's datatype string is not in KSERVE_V2_NUMPY_DATATYPES, so the inline data list cannot be converted to a numpy array. Supported: BOOL, UINT8-64, INT8-64, FP16/32/64, BYTES.

Source

Thrown at docling/models/inference_engines/common/kserve_v2_http.py:95

    name: str
    datatype: str
    shape: List[int]
    data: Optional[List[Any]] = None
    parameters: Optional[Dict[str, Any]] = None


class KserveV2InferResponse(BaseModel):
    """KServe v2 infer response payload."""

    outputs: List[KserveV2OutputTensor]


def _decode_output_tensor(raw_output: KserveV2OutputTensor) -> np.ndarray:
    shape = tuple(int(dim) for dim in raw_output.shape)
    np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(raw_output.datatype)
    if np_dtype is None:
        raise RuntimeError(
            f"Unsupported KServe v2 output datatype: {raw_output.datatype}. "
            f"Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}"
        )

    if raw_output.data is not None:
        array = np.asarray(raw_output.data, dtype=np_dtype)
        return array.reshape(shape)

    raise RuntimeError(
        f"KServe v2 output tensor {raw_output.name} did not include inline data."
    )


def _decode_binary_output_tensor(
    raw_output: KserveV2OutputTensor, raw_payload: bytes
) -> np.ndarray:
    np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(raw_output.datatype)
    if np_dtype is None:

View on GitHub (pinned to 61d76f1ff3)

Solutions

Cast model outputs to a supported datatype on the server side (FP32 is the usual choice)
Verify the datatype string via the v2 metadata endpoint and align model config
Upgrade docling or extend KSERVE_V2_NUMPY_DATATYPES for genuinely standard types

Defensive patterns

Strategy: try-catch

Validate before calling

from docling.models.inference_engines.common.kserve_v2_types import KSERVE_V2_NUMPY_DATATYPES

metadata = client.get_model_metadata()
unsupported = [o.datatype for o in metadata.outputs if o.datatype not in KSERVE_V2_NUMPY_DATATYPES]
assert not unsupported, f"Server outputs unsupported datatypes: {unsupported}"

Try / catch

try:
    outputs = client.infer(inputs=inputs, output_names=[...])
except RuntimeError as e:
    if "Unsupported KServe v2 output datatype" in str(e):
        raise  # requires a server-side model/config fix
    raise

Prevention

Assert output datatypes from the metadata endpoint during client startup
Serve FP32 outputs unless a supported narrower type is required
Regression-test the client against the server after every server upgrade

When it happens

Trigger: use_binary_data=False and the model returns a datatype like BF16, FP8, or a custom label in outputs[].datatype; server metadata/model config advertises a type this client cannot map.

Common situations: BF16-serving Triton models; KServe custom predictors with non-Triton datatype names; client older than the server's datatype vocabulary.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/189848a28a79d7cf. Report an issue: GitHub.