cocoindex-io/cocoindex · error · DeserializationError

Cannot deserialize msgspec payload ({_error_context()})

Error message

Cannot deserialize msgspec payload ({_error_context()})

What it means

Raised by `_deserialize` when a msgpack payload with msgspec routing byte 0x01 arrives but no msgspec Decoder could be built for the declared type hint. The Decoder construction (in `make_deserialize_fn`) failed eagerly — typically because the type hint uses constructs msgspec cannot support, such as unions mixing custom classes with non-None types. The original cause is chained on the raised DeserializationError.

Source

Thrown at python/cocoindex/_internal/serde.py:396

        decoder_error.__cause__ = e
    pydantic_adapter: Any = None
    pydantic_lock = threading.Lock()

    def _error_context() -> str:
        parts = [f"type_hint={type_hint!r}"]
        if source_label is not None:
            parts.append(f"source={source_label}")
        return ", ".join(parts)

    def _deserialize(data: bytes | memoryview) -> Any:
        nonlocal pydantic_adapter
        mv = memoryview(data) if not isinstance(data, memoryview) else data
        routing_byte = mv[0]

        # A: Msgspec (most common)
        if routing_byte == 0x01:
            if decoder is None:
                raise DeserializationError(
                    f"Cannot deserialize msgspec payload ({_error_context()})"
                ) from decoder_error
            try:
                return decoder.decode(mv[1:])
            except Exception as e:
                raise DeserializationError(
                    f"Failed to deserialize msgspec payload ({_error_context()})"
                ) from e

        # B: Pydantic
        if routing_byte == 0x02:
            try:
                if pydantic_adapter is None:
                    with pydantic_lock:
                        if pydantic_adapter is None:
                            import pydantic

                            pydantic_adapter = pydantic.TypeAdapter(type_hint)

View on GitHub (pinned to e84aa99b32)

Solutions

1. Read the chained cause ('Cannot build msgspec Decoder for ...' with the Hint about unions) and restructure the type hint — replace `A | B` unions of custom classes with tagged msgspec.Struct subclasses or a single dataclass
2. Resolve forward-reference annotations before serialization (e.g. call the function after its types are defined, or use eval_str-resolvable annotations)
3. If the value truly needs multiple custom variants, mark variants with @serialize_by_pickle or restructure so the top-level type is msgspec-compatible
4. Restore the previous type hint that originally produced the 0x01 payload, or delete/rebuild stale persisted state

Example fix

// before
def fn(x: MyDataclass | int) -> None: ...  # union of custom + non-None -> no msgspec Decoder
// after
import msgspec
@msgspec.tagged
class A(msgspec.Struct): ...
class B(msgspec.Struct): ...
def fn(x: A | B) -> None: ...  # tagged Struct unions are supported

from cocoindex._internal.serde import make_deserialize_fn
try:
    make_deserialize_fn(type_hint, source_label="precheck")
except Exception as e:
    raise TypeError(f"Unsupported type hint before serialization: {type_hint!r}") from e

def msgspec_supported_hint(tp) -> bool:
    import types, typing
    origin = typing.get_origin(tp)
    if origin is typing.Union or origin is types.UnionType:
        args = [a for a in typing.get_args(tp) if a is not type(None)]
        return len(args) == 1  # Optional[T] OK; mixed custom unions are not
    return True

try:
    value = serde.deserialize(data, type_hint=MyType)
except serde.DeserializationError as e:
    log.error("msgspec decoder unavailable for hint", cause=e.__cause__)
    value = fallback_reconstruct(data)

When it happens

Trigger: Calling `deserialize(data, type_hint)` or `get_deserialize_fn(type_hint)(data)` where: (1) the data's first byte is 0x01, and (2) `msgspec.msgpack.Decoder(type=type_hint)` previously raised (e.g. `int | MyDataclass`-style unions, unsupported forward references, unresolvable type hints).

Common situations: Annotating a @coco.fn parameter or return type as `X | Y` with two custom classes; using unresolvable string forward references (module-level annotations not resolved); payloads persisted by a previous run whose stored type hint was msgspec-compatible but the code has since changed the annotation to an unsupported union.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Cannot serialize {type(obj).__name__}
• Unknown extension code: {code}
• Failed to deserialize msgspec payload ({_error_context()})
• Failed to deserialize pydantic payload ({_error_context()})
• Failed to deserialize pickle payload ({_error_context()})

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/8cad4c52af52f4d1. Report an issue: GitHub.