sgl-project/sglang · error · TypeError

Cannot msgpack encode object of type {type(obj)} with enc_ho

Error message

Cannot msgpack encode object of type {type(obj)} with enc_hook. Use an explicit PickleWrapper field via wrap_as_pickle(...) for arbitrary payloads, or add a dedicated enc_hook/dec_hook branch for this transport type.

What it means

msgpack_utils' enc_hook serializes a fixed set of types (torch.Tensor, np.ndarray, shared-memory pointers, CUDA IPC proxies). Any other non-msgpack-native object reaches the final raise: arbitrary payloads must be explicitly wrapped in PickleWrapper via wrap_as_pickle(...), or a dedicated enc_hook branch must be added.

Source

Thrown at python/sglang/srt/utils/msgpack_utils.py:191

            raw_data,
        )
    if isinstance(obj, np.floating):
        return float(obj)
    if isinstance(obj, np.integer):
        return int(obj)
    if isinstance(obj, np.bool_):
        return bool(obj)
    if isinstance(obj, CudaIpcTensorTransportProxy):
        return _pack_ext(
            _MSGPACK_EXT_CUDA_IPC_TENSOR_PROXY,
            _encode_cuda_ipc_tensor_proxy(obj),
        )
    if _is_shm_pointer_mm_data(obj):
        return _pack_ext(
            _MSGPACK_EXT_SHM_POINTER_MM_DATA,
            _encode_shm_pointer_mm_data(obj),
        )
    raise TypeError(
        f"Cannot msgpack encode object of type {type(obj)} with enc_hook. "
        "Use an explicit PickleWrapper field via wrap_as_pickle(...) for "
        "arbitrary payloads, or add a dedicated enc_hook/dec_hook branch "
        "for this transport type."
    )


def dec_hook(tp: type, obj: object) -> object:
    if isinstance(obj, tp):
        return obj
    if tp is array:
        typecode, raw_data = obj
        res = array(typecode)
        res.frombytes(raw_data)
        return res
    if tp is torch.Tensor:
        shape, dtype, data, *device = obj
        return _restore_torch_tensor(shape, dtype, data, device[0] if device else "cpu")

View on GitHub (pinned to 0132848349)

Solutions

Wrap the arbitrary field with wrap_as_pickle(obj) when constructing the message, and unwrap_from_pickle(...) on receipt.
Prefer converting to primitives (lists/dicts/bytes) before packing if the payload crosses trust boundaries.
If the type is common and performance-sensitive, add an explicit enc_hook/dec_hook + ext-code branch in msgpack_utils.

Example fix

# before
msg = {"req": some_custom_object}
packed = msgpack.packb(msg, default=enc_hook)

# after
msg = {"req": wrap_as_pickle(some_custom_object)}
packed = msgpack.packb(msg, default=enc_hook)
# on the receiver:
obj = unwrap_from_pickle(msgpack.unpackb(packed, ext_hook=ext_hook)["req"])

Defensive patterns

Strategy: type-guard

Type guard

def is_msgpack_safe(obj) -> bool:
    import torch
    return obj is None or isinstance(obj, (bool, int, float, str, bytes, list, tuple, dict, torch.Tensor))

Try / catch

try:
    packed = msgpack.packb(msg, default=enc_hook)
except TypeError as e:
    raise ValueError(f"payload has unserializable field: {e}") from e

Prevention

Wrap every non-primitive field with wrap_as_pickle at construction time.
Keep wire messages primitive; document the schema.

When it happens

Trigger: Calling msgpack pack/encode on a structure containing an object of an unsupported type — e.g., a dataclass, custom class, or torch.dtype — without first passing it through wrap_as_pickle().

Common situations: Adding new fields to IPC/ZMQ message types (scheduler↔detokenizer, tokenizer manager) that carry non-primitive objects; upgrading a payload that previously held plain dicts to hold rich Python objects.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2eb71420a03f79f6. Report an issue: GitHub.