{"record":{"id":"8cad4c52af52f4d1","repo":"cocoindex-io/cocoindex","slug":"cannot-deserialize-msgspec-payload-error-contex","errorCode":null,"errorMessage":"Cannot deserialize msgspec payload ({_error_context()})","messagePattern":"Cannot deserialize msgspec payload \\((.+?)\\)","errorType":"exception","errorClass":"DeserializationError","httpStatus":null,"severity":"error","filePath":"python/cocoindex/_internal/serde.py","lineNumber":396,"sourceCode":"        decoder_error.__cause__ = e\n    pydantic_adapter: Any = None\n    pydantic_lock = threading.Lock()\n\n    def _error_context() -> str:\n        parts = [f\"type_hint={type_hint!r}\"]\n        if source_label is not None:\n            parts.append(f\"source={source_label}\")\n        return \", \".join(parts)\n\n    def _deserialize(data: bytes | memoryview) -> Any:\n        nonlocal pydantic_adapter\n        mv = memoryview(data) if not isinstance(data, memoryview) else data\n        routing_byte = mv[0]\n\n        # A: Msgspec (most common)\n        if routing_byte == 0x01:\n            if decoder is None:\n                raise DeserializationError(\n                    f\"Cannot deserialize msgspec payload ({_error_context()})\"\n                ) from decoder_error\n            try:\n                return decoder.decode(mv[1:])\n            except Exception as e:\n                raise DeserializationError(\n                    f\"Failed to deserialize msgspec payload ({_error_context()})\"\n                ) from e\n\n        # B: Pydantic\n        if routing_byte == 0x02:\n            try:\n                if pydantic_adapter is None:\n                    with pydantic_lock:\n                        if pydantic_adapter is None:\n                            import pydantic\n\n                            pydantic_adapter = pydantic.TypeAdapter(type_hint)","sourceCodeStart":378,"sourceCodeEnd":414,"githubUrl":"https://github.com/cocoindex-io/cocoindex/blob/e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b/python/cocoindex/_internal/serde.py#L378-L414","documentation":"Raised by `_deserialize` when a msgpack payload with msgspec routing byte 0x01 arrives but no msgspec Decoder could be built for the declared type hint. The Decoder construction (in `make_deserialize_fn`) failed eagerly — typically because the type hint uses constructs msgspec cannot support, such as unions mixing custom classes with non-None types. The original cause is chained on the raised DeserializationError.","triggerScenarios":"Calling `deserialize(data, type_hint)` or `get_deserialize_fn(type_hint)(data)` where: (1) the data's first byte is 0x01, and (2) `msgspec.msgpack.Decoder(type=type_hint)` previously raised (e.g. `int | MyDataclass`-style unions, unsupported forward references, unresolvable type hints).","commonSituations":"Annotating a @coco.fn parameter or return type as `X | Y` with two custom classes; using unresolvable string forward references (module-level annotations not resolved); payloads persisted by a previous run whose stored type hint was msgspec-compatible but the code has since changed the annotation to an unsupported union.","solutions":["Read the chained cause ('Cannot build msgspec Decoder for ...' with the Hint about unions) and restructure the type hint — replace `A | B` unions of custom classes with tagged msgspec.Struct subclasses or a single dataclass","Resolve forward-reference annotations before serialization (e.g. call the function after its types are defined, or use eval_str-resolvable annotations)","If the value truly needs multiple custom variants, mark variants with @serialize_by_pickle or restructure so the top-level type is msgspec-compatible","Restore the previous type hint that originally produced the 0x01 payload, or delete/rebuild stale persisted state"],"exampleFix":"// before\ndef fn(x: MyDataclass | int) -> None: ...  # union of custom + non-None -> no msgspec Decoder\n// after\nimport msgspec\n@msgspec.tagged\nclass A(msgspec.Struct): ...\nclass B(msgspec.Struct): ...\ndef fn(x: A | B) -> None: ...  # tagged Struct unions are supported","handlingStrategy":"type-guard","validationCode":"from cocoindex._internal.serde import make_deserialize_fn\ntry:\n    make_deserialize_fn(type_hint, source_label=\"precheck\")\nexcept Exception as e:\n    raise TypeError(f\"Unsupported type hint before serialization: {type_hint!r}\") from e","typeGuard":"def msgspec_supported_hint(tp) -> bool:\n    import types, typing\n    origin = typing.get_origin(tp)\n    if origin is typing.Union or origin is types.UnionType:\n        args = [a for a in typing.get_args(tp) if a is not type(None)]\n        return len(args) == 1  # Optional[T] OK; mixed custom unions are not\n    return True","tryCatchPattern":"try:\n    value = serde.deserialize(data, type_hint=MyType)\nexcept serde.DeserializationError as e:\n    log.error(\"msgspec decoder unavailable for hint\", cause=e.__cause__)\n    value = fallback_reconstruct(data)","preventionTips":["Avoid unions mixing custom classes with non-None types in @coco.fn annotations","Use tagged msgspec.Struct subclasses for multi-variant types","Resolve string forward references at module import time","Pre-check the type hint with make_deserialize_fn in unit tests"],"tags":["python","serialization","msgpack","type-hints"],"backgroundTag":"type-mismatch","analyzedSha":"e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b","analyzedAt":"2026-09-08T15:59:19.997Z","contentChangedAt":"2026-09-08T15:59:19.997Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}