{"record":{"id":"b8b4edeb55764c61","repo":"cocoindex-io/cocoindex","slug":"failed-to-deserialize-pickle-payload-error-cont","errorCode":null,"errorMessage":"Failed to deserialize pickle payload ({_error_context()})","messagePattern":"Failed to deserialize pickle payload \\((.+?)\\)","errorType":"exception","errorClass":"DeserializationError","httpStatus":null,"severity":"error","filePath":"python/cocoindex/_internal/serde.py","lineNumber":429,"sourceCode":"                        if pydantic_adapter is None:\n                            import pydantic\n\n                            pydantic_adapter = pydantic.TypeAdapter(type_hint)\n                raw = msgspec.msgpack.decode(mv[1:], ext_hook=_ext_hook)\n                if type_hint is Any:\n                    return raw\n                return pydantic_adapter.validate_python(raw)\n            except Exception as e:\n                raise DeserializationError(\n                    f\"Failed to deserialize pydantic payload ({_error_context()})\"\n                ) from e\n\n        # C: Pickle (legacy and @serialize_by_pickle)\n        if routing_byte == 0x80:\n            try:\n                return _RestrictedUnpickler(io.BytesIO(bytes(mv))).load()\n            except Exception as e:\n                raise DeserializationError(\n                    f\"Failed to deserialize pickle payload ({_error_context()})\"\n                ) from e\n\n        raise DeserializationError(\n            f\"Unknown routing byte: {routing_byte:#x} ({_error_context()})\"\n        )\n\n    return _deserialize\n\n\n# ---------------------------------------------------------------------------\n# Top-level serialize / deserialize\n# ---------------------------------------------------------------------------\n\n\ndef serialize(value: Any) -> bytes:\n    \"\"\"Serialize a value using the routing-byte protocol (C → B → A priority).\"\"\"\n    # C: Explicit pickle (user opted in — highest priority)","sourceCodeStart":411,"sourceCodeEnd":447,"githubUrl":"https://github.com/cocoindex-io/cocoindex/blob/e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b/python/cocoindex/_internal/serde.py#L411-L447","documentation":"Raised when a pickle-routed payload (routing byte 0x80) cannot be unpickled through cocoindex's restricted unpickler. Serialization deliberately limits unpickling to an allow-list of builtin/stdlib types plus classes registered via `unpickle_safe` / `serialize_by_pickle`; any other global reference, or any malformed/truncated pickle bytes, causes the load to fail and is wrapped in this DeserializationError with the type hint and source label as context.","triggerScenarios":"`deserialize(data, type_hint)` with 0x80-routed bytes where: the pickled object references a class not in the unpickle allow-list ('Forbidden global during unpickling'); the class was renamed/moved between serialize and deserialize; the data is truncated or corrupted; or the pickle was produced with objects from a module not importable in the current environment.","commonSituations":"Serializing a custom class then refactoring its module/qualname before the memoized value is read; deserializing on a machine where the pickled class's package is not installed; numpy version changes altering internal ndarray reconstruction globals; payloads saved under a virtualenv where the class differs.","solutions":["Read `e.__cause__`: if 'Forbidden global during unpickling: module.Name', decorate the class with @cocoindex.unpickle_safe (or @serialize_by_pickle) so it is allow-listed, then re-serialize","Restore the original module path/qualname of the pickled class, or re-generate the payload after the refactor","Install the missing package that defines the pickled class in the current environment","If bytes are truncated/corrupted, delete the stale persisted state and regenerate it"],"exampleFix":"// before\nclass MyMatrix: ...  # not registered -> Forbidden global during unpickling\n// after\nimport cocoindex as coco\nfrom cocoindex._internal.serde import serialize_by_pickle\n@serialize_by_pickle\nclass MyMatrix: ...  # registered as unpickle-safe and pickled on serialize","handlingStrategy":"validation","validationCode":"from cocoindex._internal.serde import _UNPICKLE_SAFE_GLOBALS\ndef type_is_unpickle_safe(cls: type) -> bool:\n    return (cls.__module__, cls.__qualname__) in _UNPICKLE_SAFE_GLOBALS\n# call before serializing: assert type_is_unpickle_safe(MyClass)","typeGuard":null,"tryCatchPattern":"try:\n    value = serde.deserialize(data, type_hint=MyType)\nexcept serde.DeserializationError as e:\n    if isinstance(e.__cause__, pickle.UnpicklingError):\n        value = rebuild_payload_manually()\n    else:\n        raise","preventionTips":["Decorate custom classes with @serialize_by_pickle (or @unpickle_safe) before serializing them","Avoid renaming/moving pickled classes between serialize and deserialize; if needed, clear persisted state","Ensure all packages defining pickled classes are installed in the reading environment","Never unpickle bytes from untrusted sources — the restricted unpickler exists for a reason","Pin numpy/stdlib versions where ndarray payloads are persisted"],"tags":["python","serialization","pickle","security"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b","analyzedAt":"2026-09-08T15:59:19.997Z","contentChangedAt":"2026-09-08T15:59:19.997Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}