{"record":{"id":"189848a28a79d7cf","repo":"docling-project/docling","slug":"unsupported-kserve-v2-output-datatype-raw-output","errorCode":null,"errorMessage":"Unsupported KServe v2 output datatype: {raw_output.datatype}. Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}","messagePattern":"Unsupported KServe v2 output datatype: (.+?)\\. Supported types: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/inference_engines/common/kserve_v2_http.py","lineNumber":95,"sourceCode":"\n    name: str\n    datatype: str\n    shape: List[int]\n    data: Optional[List[Any]] = None\n    parameters: Optional[Dict[str, Any]] = None\n\n\nclass KserveV2InferResponse(BaseModel):\n    \"\"\"KServe v2 infer response payload.\"\"\"\n\n    outputs: List[KserveV2OutputTensor]\n\n\ndef _decode_output_tensor(raw_output: KserveV2OutputTensor) -> np.ndarray:\n    shape = tuple(int(dim) for dim in raw_output.shape)\n    np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(raw_output.datatype)\n    if np_dtype is None:\n        raise RuntimeError(\n            f\"Unsupported KServe v2 output datatype: {raw_output.datatype}. \"\n            f\"Supported types: {list(KSERVE_V2_NUMPY_DATATYPES.keys())}\"\n        )\n\n    if raw_output.data is not None:\n        array = np.asarray(raw_output.data, dtype=np_dtype)\n        return array.reshape(shape)\n\n    raise RuntimeError(\n        f\"KServe v2 output tensor {raw_output.name} did not include inline data.\"\n    )\n\n\ndef _decode_binary_output_tensor(\n    raw_output: KserveV2OutputTensor, raw_payload: bytes\n) -> np.ndarray:\n    np_dtype = KSERVE_V2_NUMPY_DATATYPES.get(raw_output.datatype)\n    if np_dtype is None:","sourceCodeStart":77,"sourceCodeEnd":113,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/inference_engines/common/kserve_v2_http.py#L77-L113","documentation":"Raised in _decode_output_tensor on the JSON (non-binary) HTTP path: an output tensor's datatype string is not in KSERVE_V2_NUMPY_DATATYPES, so the inline data list cannot be converted to a numpy array. Supported: BOOL, UINT8-64, INT8-64, FP16/32/64, BYTES.","triggerScenarios":"use_binary_data=False and the model returns a datatype like BF16, FP8, or a custom label in outputs[].datatype; server metadata/model config advertises a type this client cannot map.","commonSituations":"BF16-serving Triton models; KServe custom predictors with non-Triton datatype names; client older than the server's datatype vocabulary.","solutions":["Cast model outputs to a supported datatype on the server side (FP32 is the usual choice)","Verify the datatype string via the v2 metadata endpoint and align model config","Upgrade docling or extend KSERVE_V2_NUMPY_DATATYPES for genuinely standard types"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"from docling.models.inference_engines.common.kserve_v2_types import KSERVE_V2_NUMPY_DATATYPES\n\nmetadata = client.get_model_metadata()\nunsupported = [o.datatype for o in metadata.outputs if o.datatype not in KSERVE_V2_NUMPY_DATATYPES]\nassert not unsupported, f\"Server outputs unsupported datatypes: {unsupported}\"","typeGuard":null,"tryCatchPattern":"try:\n    outputs = client.infer(inputs=inputs, output_names=[...])\nexcept RuntimeError as e:\n    if \"Unsupported KServe v2 output datatype\" in str(e):\n        raise  # requires a server-side model/config fix\n    raise","preventionTips":["Assert output datatypes from the metadata endpoint during client startup","Serve FP32 outputs unless a supported narrower type is required","Regression-test the client against the server after every server upgrade"],"tags":["http","kserve","dtype","model-config"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}