docling-project/docling · error · RuntimeError

Unsupported numpy dtype for gRPC inline (non-binary) decodin

Error message

Unsupported numpy dtype for gRPC inline (non-binary) decoding: {canonical_dtype!s}. Supported non-binary dtypes: bool, uint8/uint16/uint32/uint64, int8/int16/int32/int64, float32/float64, BYTES.

What it means

Raised as RuntimeError by _decode_contents when a response tensor's declared datatype maps to a numpy dtype outside the supported inline decoding set (bool, u8-u64, i8-i64, f32/f64, object/BYTES). The server sent a tensor whose datatype this client cannot materialize into a numpy array.

Source

Thrown at docling/models/inference_engines/common/kserve_v2_grpc.py:169

        np.dtype(np.int32),
    ):
        data = list(contents.int_contents)
    elif canonical_dtype == np.dtype(np.int64):
        data = list(contents.int64_contents)
    elif canonical_dtype in (
        np.dtype(np.uint8),
        np.dtype(np.uint16),
        np.dtype(np.uint32),
    ):
        data = list(contents.uint_contents)
    elif canonical_dtype == np.dtype(np.uint64):
        data = list(contents.uint64_contents)
    elif canonical_dtype == np.dtype(np.bool_):
        data = list(contents.bool_contents)
    elif canonical_dtype == np.dtype(object):
        data = list(contents.bytes_contents)
    else:
        raise RuntimeError(
            f"Unsupported numpy dtype for gRPC inline (non-binary) decoding: {canonical_dtype!s}. "
            "Supported non-binary dtypes: bool, uint8/uint16/uint32/uint64, "
            "int8/int16/int32/int64, float32/float64, BYTES."
        )
    return np.asarray(data, dtype=canonical_dtype).reshape(shape)


@dataclass
class KserveV2GrpcClient:
    """Minimal client for KServe v2 gRPC infer requests."""

    base_url: str
    model_name: str
    model_version: str | None
    timeout: float
    metadata: Mapping[str, str]
    use_tls: bool
    max_message_bytes: int

View on GitHub (pinned to 61d76f1ff3)

Solutions

Configure the server (or model output signature) to emit FP32/FP64 tensors instead of FP16/BF16.
Upgrade docling so the client knows the datatype, if support was added.
Enable binary tensor contents on the server side so raw bytes + explicit datatype are used rather than inline decoding.

Example fix

# before: server outputs fp16 -> RuntimeError on decode
# after: configure the model server to cast outputs to fp32
# e.g. in the model's inference fn: return {'outputs': out.astype(np.float32)}

Defensive patterns

Strategy: try-catch

Validate before calling

meta = client.get_model_metadata()
unsupported = {t.datatype for t in meta.outputs} - {'BOOL','UINT8','UINT16','UINT32','UINT64','INT8','INT16','INT32','INT64','FP32','FP64','BYTES'}
if unsupported:
    raise RuntimeError(f'server emits unsupported datatypes: {unsupported}')

Try / catch

try:
    result = client.infer(...)
except RuntimeError as e:
    if 'decoding' in str(e):
        raise RuntimeError('server datatype unsupported by client; emit FP32 or enable binary contents') from e
    raise

Prevention

Probe get_model_metadata() at client startup to learn output datatypes.
Configure servers to output fp32 tensors for gRPC clients.
Keep server and docling client versions aligned on the datatype vocabulary.

When it happens

Trigger: The KServe server responds with a tensor datatype such as FP16, BF16, FP8 or a string datatype that canonicalizes to an unsupported numpy dtype, and the response was not binary-encoded.

Common situations: Remote model served with half-precision outputs; server and client disagree on the allowed datatype vocabulary; newer KServe datatypes not known to this client version.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/0cb5549ea1be7c71. Report an issue: GitHub.