cocoindex-io/cocoindex · error · DeserializationError

Cannot deserialize msgspec payload

Error message

Cannot deserialize msgspec payload ({_error_context()})

What it means

Raised by `_deserialize` when a msgpack payload with msgspec routing byte 0x01 arrives but no msgspec Decoder could be built for the declared type hint. The Decoder construction (in `make_deserialize_fn`) failed eagerly — typically because the type hint uses constructs msgspec cannot support, such as unions mixing custom classes with non-None types. The original cause is chained on the raised DeserializationError.

Solutions

1. Read the chained cause ('Cannot build msgspec Decoder for ...' with the Hint about unions) and restructure the type hint — replace `A | B` unions of custom classes with tagged msgspec.Struct subclasses or a single dataclass
2. Resolve forward-reference annotations before serialization (e.g. call the function after its types are defined, or use eval_str-resolvable annotations)
3. If the value truly needs multiple custom variants, mark variants with @serialize_by_pickle or restructure so the top-level type is msgspec-compatible
4. Restore the previous type hint that originally produced the 0x01 payload, or delete/rebuild stale persisted state

Example fix

// before
def fn(x: MyDataclass | int) -> None: ...  # union of custom + non-None -> no msgspec Decoder
// after
import msgspec
@msgspec.tagged
class A(msgspec.Struct): ...
class B(msgspec.Struct): ...
def fn(x: A | B) -> None: ...  # tagged Struct unions are supported

from cocoindex._internal.serde import make_deserialize_fn
try:
    make_deserialize_fn(type_hint, source_label="precheck")
except Exception as e:
    raise TypeError(f"Unsupported type hint before serialization: {type_hint!r}") from e

def msgspec_supported_hint(tp) -> bool:
    import types, typing
    origin = typing.get_origin(tp)
    if origin is typing.Union or origin is types.UnionType:
        args = [a for a in typing.get_args(tp) if a is not type(None)]
        return len(args) == 1  # Optional[T] OK; mixed custom unions are not
    return True

try:
    value = serde.deserialize(data, type_hint=MyType)
except serde.DeserializationError as e:
    log.error("msgspec decoder unavailable for hint", cause=e.__cause__)
    value = fallback_reconstruct(data)

When it happens

Trigger: Calling `deserialize(data, type_hint)` or `get_deserialize_fn(type_hint)(data)` where: (1) the data's first byte is 0x01, and (2) `msgspec.msgpack.Decoder(type=type_hint)` previously raised (e.g. `int | MyDataclass`-style unions, unsupported forward references, unresolvable type hints).

Common situations: Annotating a @coco.fn parameter or return type as `X | Y` with two custom classes; using unresolvable string forward references (module-level annotations not resolved); payloads persisted by a previous run whose stored type hint was msgspec-compatible but the code has since changed the annotation to an unsupported union.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Cannot serialize
• Failed to deserialize msgspec payload
• Unknown extension code
• Failed to deserialize pickle payload
• Failed to deserialize pydantic payload

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/8cad4c52af52f4d1. Report an issue: GitHub.

Appendix: source

Thrown at python/cocoindex/_internal/serde.py:396

        decoder_error.__cause__ = e
    pydantic_adapter: Any = None
    pydantic_lock = threading.Lock()

    def _error_context() -> str:
        parts = [f"type_hint={type_hint!r}"]
        if source_label is not None:
            parts.append(f"source={source_label}")
        return ", ".join(parts)

    def _deserialize(data: bytes | memoryview) -> Any:
        nonlocal pydantic_adapter
        mv = memoryview(data) if not isinstance(data, memoryview) else data
        routing_byte = mv[0]

        # A: Msgspec (most common)
        if routing_byte == 0x01:
            if decoder is None:
                raise DeserializationError(
                    f"Cannot deserialize msgspec payload ({_error_context()})"
                ) from decoder_error
            try:
                return decoder.decode(mv[1:])
            except Exception as e:
                raise DeserializationError(
                    f"Failed to deserialize msgspec payload ({_error_context()})"
                ) from e

        # B: Pydantic
        if routing_byte == 0x02:
            try:
                if pydantic_adapter is None:
                    with pydantic_lock:
                        if pydantic_adapter is None:
                            import pydantic

                            pydantic_adapter = pydantic.TypeAdapter(type_hint)

View on GitHub (pinned to e84aa99b32)