docling-project/docling · error · ValueError

Unsupported numpy dtype for KServe v2 gRPC input: {np_tensor

Error message

Unsupported numpy dtype for KServe v2 gRPC input: {np_tensor.dtype!s}. Supported types: {list(NUMPY_KSERVE_V2_DATATYPES.keys())}

What it means

Raised while building a KServe v2 gRPC ModelInferRequest: a numpy tensor passed in the `inputs` mapping has a dtype that has no KServe v2 datatype name. The mapping (NUMPY_KSERVE_V2_DATATYPES in kserve_v2_types.py) only covers BOOL, UINT8/16/32/64, INT8/16/32/64, FP16/32/64 and object (BYTES). Any other numpy dtype (float128, complex, datetime64, '<U' string dtypes, structured dtypes) cannot be serialized onto the wire.

Source

Thrown at docling/models/inference_engines/common/kserve_v2_grpc.py:303

        _batch_size = next(iter(inputs.values())).shape[0] if inputs else 0

        if _log.isEnabledFor(logging.DEBUG):
            _t_ser_start = time.time()
            _t_ser_mono = time.monotonic()

        request = service_pb2.ModelInferRequest(model_name=self.model_name)
        if self.model_version:
            request.model_version = self.model_version

        if request_parameters:
            for key, value in request_parameters.items():
                _set_request_parameter(request.parameters, key=key, value=value)

        for input_name, tensor in inputs.items():
            np_tensor = np.asarray(tensor)
            kserve_dtype = NUMPY_KSERVE_V2_DATATYPES.get(np_tensor.dtype)
            if kserve_dtype is None:
                raise ValueError(
                    f"Unsupported numpy dtype for KServe v2 gRPC input: {np_tensor.dtype!s}. "
                    f"Supported types: {list(NUMPY_KSERVE_V2_DATATYPES.keys())}"
                )

            input_tensor = request.inputs.add()
            input_tensor.name = input_name
            input_tensor.datatype = kserve_dtype
            input_tensor.shape.extend(int(dim) for dim in np_tensor.shape)

            if self.use_binary_data:
                input_tensor.parameters["binary_data"].bool_param = True
                if kserve_dtype == "BYTES":  # Bytes encoding
                    request.raw_input_contents.append(encode_bytes_tensor(np_tensor))
                else:
                    contiguous = np.ascontiguousarray(np_tensor)
                    request.raw_input_contents.append(contiguous.tobytes())
            else:
                _encode_contents(np_tensor, input_tensor.contents)

View on GitHub (pinned to 61d76f1ff3)

Solutions

  1. Cast the offending tensor before calling infer: string tensors to dtype=object (arr.astype(object)), numeric ones to np.float32/np.int64 as the model expects
  2. Check membership first: assert np.asarray(t).dtype in NUMPY_KSERVE_V2_DATATYPES for each input tensor
  3. Inspect the exception message - it names the exact dtype and the full supported list
  4. If a genuinely needed dtype is missing (e.g. BF16), extend NUMPY_KSERVE_V2_DATATYPES in a fork/PR rather than bypassing the check

Example fix

// before
inputs = {"labels": np.array(["foo", "bar"])}  # dtype '<U3' -> ValueError

// after
inputs = {"labels": np.array(["foo", "bar"], dtype=object)}  # maps to BYTES
Defensive patterns

Strategy: validation

Validate before calling

import numpy as np
from docling.models.inference_engines.common.kserve_v2_types import NUMPY_KSERVE_V2_DATATYPES

def validate_inputs(inputs: dict[str, np.ndarray]) -> None:
    for name, tensor in inputs.items():
        dtype = np.asarray(tensor).dtype
        if dtype not in NUMPY_KSERVE_V2_DATATYPES:
            raise TypeError(
                f"Input {name!r} has unsupported dtype {dtype}; "
                f"cast to one of {sorted(map(str, NUMPY_KSERVE_V2_DATATYPES))}"
            )

Type guard

import numpy as np
from docling.models.inference_engines.common.kserve_v2_types import NUMPY_KSERVE_V2_DATATYPES

def is_encodable_tensor(tensor: np.ndarray) -> bool:
    """True when the tensor's dtype can be sent to a KServe v2 endpoint."""
    return np.asarray(tensor).dtype in NUMPY_KSERVE_V2_DATATYPES

Try / catch

try:
    outputs = engine.infer(inputs=inputs)
except ValueError as e:
    if "Unsupported numpy dtype" in str(e):
        inputs = {k: normalize(v) for k, v in inputs.items()}  # cast and retry once
    else:
        raise

Prevention

When it happens

Trigger: Calling the gRPC engine's infer with e.g. np.array(['a','b']) (dtype '<U1'), np.float128 arrays, np.complex128, or a Pandas/canvas-produced array with an unusual dtype. np.asarray(tensor) is applied first, so list-of-str inputs also become '<U' dtype and hit this.

Common situations: Passing raw text/label tensors for BYTES models without encoding to object dtype; mixed Python types producing '<U32' arrays; upgrading numpy where an op returns float128 on some platforms; feeding datetime columns from a dataframe.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/7870a3d8eb2169dd. Report an issue: GitHub.