{"record":{"id":"7870a3d8eb2169dd","repo":"docling-project/docling","slug":"unsupported-numpy-dtype-for-kserve-v2-grpc-input","errorCode":null,"errorMessage":"Unsupported numpy dtype for KServe v2 gRPC input: {np_tensor.dtype!s}. Supported types: {list(NUMPY_KSERVE_V2_DATATYPES.keys())}","messagePattern":"Unsupported numpy dtype for KServe v2 gRPC input: (.+?)\\. Supported types: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_grpc.py","lineNumber":303,"sourceCode":"        _batch_size = next(iter(inputs.values())).shape[0] if inputs else 0\n\n        if _log.isEnabledFor(logging.DEBUG):\n            _t_ser_start = time.time()\n            _t_ser_mono = time.monotonic()\n\n        request = service_pb2.ModelInferRequest(model_name=self.model_name)\n        if self.model_version:\n            request.model_version = self.model_version\n\n        if request_parameters:\n            for key, value in request_parameters.items():\n                _set_request_parameter(request.parameters, key=key, value=value)\n\n        for input_name, tensor in inputs.items():\n            np_tensor = np.asarray(tensor)\n            kserve_dtype = NUMPY_KSERVE_V2_DATATYPES.get(np_tensor.dtype)\n            if kserve_dtype is None:\n                raise ValueError(\n                    f\"Unsupported numpy dtype for KServe v2 gRPC input: {np_tensor.dtype!s}. \"\n                    f\"Supported types: {list(NUMPY_KSERVE_V2_DATATYPES.keys())}\"\n                )\n\n            input_tensor = request.inputs.add()\n            input_tensor.name = input_name\n            input_tensor.datatype = kserve_dtype\n            input_tensor.shape.extend(int(dim) for dim in np_tensor.shape)\n\n            if self.use_binary_data:\n                input_tensor.parameters[\"binary_data\"].bool_param = True\n                if kserve_dtype == \"BYTES\":  # Bytes encoding\n                    request.raw_input_contents.append(encode_bytes_tensor(np_tensor))\n                else:\n                    contiguous = np.ascontiguousarray(np_tensor)\n                    request.raw_input_contents.append(contiguous.tobytes())\n            else:\n                _encode_contents(np_tensor, input_tensor.contents)","sourceCodeStart":285,"sourceCodeEnd":321,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_grpc.py#L285-L321","documentation":"Raised while building a KServe v2 gRPC ModelInferRequest: a numpy tensor passed in the `inputs` mapping has a dtype that has no KServe v2 datatype name. The mapping (NUMPY_KSERVE_V2_DATATYPES in kserve_v2_types.py) only covers BOOL, UINT8/16/32/64, INT8/16/32/64, FP16/32/64 and object (BYTES). Any other numpy dtype (float128, complex, datetime64, '<U' string dtypes, structured dtypes) cannot be serialized onto the wire.","triggerScenarios":"Calling the gRPC engine's infer with e.g. np.array(['a','b']) (dtype '<U1'), np.float128 arrays, np.complex128, or a Pandas/canvas-produced array with an unusual dtype. np.asarray(tensor) is applied first, so list-of-str inputs also become '<U' dtype and hit this.","commonSituations":"Passing raw text/label tensors for BYTES models without encoding to object dtype; mixed Python types producing '<U32' arrays; upgrading numpy where an op returns float128 on some platforms; feeding datetime columns from a dataframe.","solutions":["Cast the offending tensor before calling infer: string tensors to dtype=object (arr.astype(object)), numeric ones to np.float32/np.int64 as the model expects","Check membership first: assert np.asarray(t).dtype in NUMPY_KSERVE_V2_DATATYPES for each input tensor","Inspect the exception message - it names the exact dtype and the full supported list","If a genuinely needed dtype is missing (e.g. BF16), extend NUMPY_KSERVE_V2_DATATYPES in a fork/PR rather than bypassing the check"],"exampleFix":"// before\ninputs = {\"labels\": np.array([\"foo\", \"bar\"])}  # dtype '<U3' -> ValueError\n\n// after\ninputs = {\"labels\": np.array([\"foo\", \"bar\"], dtype=object)}  # maps to BYTES","handlingStrategy":"validation","validationCode":"import numpy as np\nfrom docling.models.inference_engines.common.kserve_v2_types import NUMPY_KSERVE_V2_DATATYPES\n\ndef validate_inputs(inputs: dict[str, np.ndarray]) -> None:\n    for name, tensor in inputs.items():\n        dtype = np.asarray(tensor).dtype\n        if dtype not in NUMPY_KSERVE_V2_DATATYPES:\n            raise TypeError(\n                f\"Input {name!r} has unsupported dtype {dtype}; \"\n                f\"cast to one of {sorted(map(str, NUMPY_KSERVE_V2_DATATYPES))}\"\n            )","typeGuard":"import numpy as np\nfrom docling.models.inference_engines.common.kserve_v2_types import NUMPY_KSERVE_V2_DATATYPES\n\ndef is_encodable_tensor(tensor: np.ndarray) -> bool:\n    \"\"\"True when the tensor's dtype can be sent to a KServe v2 endpoint.\"\"\"\n    return np.asarray(tensor).dtype in NUMPY_KSERVE_V2_DATATYPES","tryCatchPattern":"try:\n    outputs = engine.infer(inputs=inputs)\nexcept ValueError as e:\n    if \"Unsupported numpy dtype\" in str(e):\n        inputs = {k: normalize(v) for k, v in inputs.items()}  # cast and retry once\n    else:\n        raise","preventionTips":["Always build string tensors with dtype=object, never leave them as '<U' dtype","Standardize numeric inputs with .astype(np.float32) right after creation","Wrap infer calls with a helper that pre-validates dtypes against NUMPY_KSERVE_V2_DATATYPES"],"tags":["numpy","dtype","grpc","kserve","validation"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}