{"record":{"id":"42a2b96609df4002","repo":"cocoindex-io/cocoindex","slug":"failed-to-deserialize-msgspec-payload-error-con","errorCode":null,"errorMessage":"Failed to deserialize msgspec payload ({_error_context()})","messagePattern":"Failed to deserialize msgspec payload \\((.+?)\\)","errorType":"exception","errorClass":"DeserializationError","httpStatus":null,"severity":"error","filePath":"python/cocoindex/_internal/serde.py","lineNumber":402,"sourceCode":"        if source_label is not None:\n            parts.append(f\"source={source_label}\")\n        return \", \".join(parts)\n\n    def _deserialize(data: bytes | memoryview) -> Any:\n        nonlocal pydantic_adapter\n        mv = memoryview(data) if not isinstance(data, memoryview) else data\n        routing_byte = mv[0]\n\n        # A: Msgspec (most common)\n        if routing_byte == 0x01:\n            if decoder is None:\n                raise DeserializationError(\n                    f\"Cannot deserialize msgspec payload ({_error_context()})\"\n                ) from decoder_error\n            try:\n                return decoder.decode(mv[1:])\n            except Exception as e:\n                raise DeserializationError(\n                    f\"Failed to deserialize msgspec payload ({_error_context()})\"\n                ) from e\n\n        # B: Pydantic\n        if routing_byte == 0x02:\n            try:\n                if pydantic_adapter is None:\n                    with pydantic_lock:\n                        if pydantic_adapter is None:\n                            import pydantic\n\n                            pydantic_adapter = pydantic.TypeAdapter(type_hint)\n                raw = msgspec.msgpack.decode(mv[1:], ext_hook=_ext_hook)\n                if type_hint is Any:\n                    return raw\n                return pydantic_adapter.validate_python(raw)\n            except Exception as e:\n                raise DeserializationError(","sourceCodeStart":384,"sourceCodeEnd":420,"githubUrl":"https://github.com/cocoindex-io/cocoindex/blob/e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b/python/cocoindex/_internal/serde.py#L384-L420","documentation":"Raised when a msgpack payload with msgspec routing byte 0x01 is decoded but `decoder.decode()` raises — the bytes do not conform to what the declared type hint expects. CocoIndex wraps any underlying msgspec exception (DecodeError, ValidationError, UnpicklingError from the ext hook, etc.) in a DeserializationError that includes the type hint and source label, chaining the original exception as cause.","triggerScenarios":"`deserialize(data, type_hint)` where the 0x01-routed bytes: contain a different structure than type_hint (e.g. saved when the type was `int`, now decoded as `MyDataclass`); nest a custom object whose restricted unpickle is forbidden (find_class not in the allow-list); have truncated/corrupted msgpack bytes; or the stored data predates a type change.","commonSituations":"Changing a @coco.fn's parameter/return annotation after memoized results were persisted; decoding state files from a different cocoindex version; dataclasses whose fields changed shape; a class renamed/moved so the pickle quarantine can no longer resolve it; truncated files from interrupted writes.","solutions":["Inspect the chained cause (`e.__cause__`) for the exact msgspec/validation error and fix the mismatch between payload and type hint","Clear stale memoization/persisted state (delete the environment's db/state directory) so payloads are regenerated with the current type annotations","If you changed a type's fields, migrate old payloads or use a new component path / new function name so old entries are not reused","If the underlying error is 'Forbidden global during unpickling', register the class with @cocoindex unpickle_safe (or @serialize_by_pickle) and re-serialize"],"exampleFix":"// before: annotation changed after memoized payloads were saved\n@coco.fn(memo=True)\ndef process(x: list[int]) -> int: ...  # old payload saved as tuple shape\n// after: rebuild state\ncoco_env.drop_blocking()  # or delete state dir\napp.update_blocking()     # regenerate payloads with current annotations","handlingStrategy":"try-catch","validationCode":"# verify the stored payload's routing byte matches expectations before decode\nassert data[0] == 0x01, \"payload is not msgspec-routed\"","typeGuard":null,"tryCatchPattern":"try:\n    value = serde.deserialize(data, type_hint=MyType)\nexcept serde.DeserializationError as e:\n    print(e.__cause__)  # exact msgspec/validation error\n    value = serde.deserialize(regenerate_payload(), type_hint=MyType)","preventionTips":["When changing a @coco.fn's type annotations, drop stale memoized/persisted state first","Use memo=True only with stable type annotations","Keep cocoindex versions consistent across writer/reader","Wrap risky decodes and inspect e.__cause__ before retrying","Avoid truncating payload files; write atomically (tmp file + rename)"],"tags":["python","serialization","msgpack","schema-change"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b","analyzedAt":"2026-09-08T15:59:19.997Z","contentChangedAt":"2026-09-08T15:59:19.997Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}