{"record":{"id":"75c341c9c8b859cc","repo":"cocoindex-io/cocoindex","slug":"failed-to-deserialize-pydantic-payload-error-co","errorCode":null,"errorMessage":"Failed to deserialize pydantic payload ({_error_context()})","messagePattern":"Failed to deserialize pydantic payload \\((.+?)\\)","errorType":"exception","errorClass":"DeserializationError","httpStatus":null,"severity":"error","filePath":"python/cocoindex/_internal/serde.py","lineNumber":420,"sourceCode":"                raise DeserializationError(\n                    f\"Failed to deserialize msgspec payload ({_error_context()})\"\n                ) from e\n\n        # B: Pydantic\n        if routing_byte == 0x02:\n            try:\n                if pydantic_adapter is None:\n                    with pydantic_lock:\n                        if pydantic_adapter is None:\n                            import pydantic\n\n                            pydantic_adapter = pydantic.TypeAdapter(type_hint)\n                raw = msgspec.msgpack.decode(mv[1:], ext_hook=_ext_hook)\n                if type_hint is Any:\n                    return raw\n                return pydantic_adapter.validate_python(raw)\n            except Exception as e:\n                raise DeserializationError(\n                    f\"Failed to deserialize pydantic payload ({_error_context()})\"\n                ) from e\n\n        # C: Pickle (legacy and @serialize_by_pickle)\n        if routing_byte == 0x80:\n            try:\n                return _RestrictedUnpickler(io.BytesIO(bytes(mv))).load()\n            except Exception as e:\n                raise DeserializationError(\n                    f\"Failed to deserialize pickle payload ({_error_context()})\"\n                ) from e\n\n        raise DeserializationError(\n            f\"Unknown routing byte: {routing_byte:#x} ({_error_context()})\"\n        )\n\n    return _deserialize\n","sourceCodeStart":402,"sourceCodeEnd":438,"githubUrl":"https://github.com/cocoindex-io/cocoindex/blob/e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b/python/cocoindex/_internal/serde.py#L402-L438","documentation":"Raised when a pydantic-routed payload (routing byte 0x02) fails during the raw msgpack decode or the subsequent `pydantic.TypeAdapter.validate_python` step. CocoIndex encodes Pydantic models as msgpack dicts of their JSON dump; this error means those bytes could not be decoded into raw Python, or the resulting dict/list did not match the declared type hint per pydantic validation. The original exception is chained as cause.","triggerScenarios":"`deserialize(data, type_hint)` with 0x02-routed bytes where: the model fields changed since serialization (missing/extra fields failing validation); type_hint is incompatible with the decoded raw data; pydantic import or TypeAdapter construction fails inside the lazy adapter; or the bytes are corrupted/truncated msgpack.","commonSituations":"Evolving a Pydantic BaseModel used in a @coco.fn (removing/renaming fields with old memoized payloads persisted); storing payloads with an older pydantic v1-style model; passing a type_hint like `list[str]` where the payload contains a model dict; unpickling errors inside nested ext-quarantined values during raw decode.","solutions":["Read `e.__cause__` for the exact pydantic ValidationError and align the type hint or the model fields with what was serialized","Clear stale persisted state / memoization entries so payloads are regenerated with the current model definition","Give removed/renamed pydantic fields defaults or `Optional` types so old payloads still validate","Verify pydantic v2 is installed — the adapter path uses TypeAdapter/model_dump(mode='json') and fails with incompatible pydantic versions"],"exampleFix":"// before: renamed field breaks old payloads\nclass Row(BaseModel):\n    user_name: str\n// after: keep old payloads validating\nclass Row(BaseModel):\n    user_name: str = \"\"\n    # or migrate: user_name: str = Field(validation_alias=AliasChoices('user_name','name'))","handlingStrategy":"try-catch","validationCode":"import pydantic\ntry:\n    pydantic.TypeAdapter(type_hint)\nexcept Exception as e:\n    raise TypeError(f\"hint not pydantic-compatible: {type_hint!r}\") from e","typeGuard":"def looks_like_pydantic_payload(data: bytes) -> bool:\n    return bool(data) and data[0] == 0x02","tryCatchPattern":"try:\n    value = serde.deserialize(data, type_hint=MyModel)\nexcept serde.DeserializationError as e:\n    # e.__cause__ is the pydantic ValidationError with per-field details\n    for err in getattr(e.__cause__, 'errors', lambda: [])():\n        print(err['loc'], err['type'])\n    value = MyModel.model_validate(new_raw_dict)","preventionTips":["Give pydantic fields defaults or Optional types so old payloads keep validating","Use AliasChoices to keep old field names valid after renames","Keep pydantic v2 installed (the serde path relies on TypeAdapter/model_dump)","Clear memoized state after breaking model changes","Inspect e.__cause__.errors() to pinpoint failing fields"],"tags":["python","serialization","pydantic","validation"],"backgroundTag":"schema-validation-failed","analyzedSha":"e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b","analyzedAt":"2026-09-08T15:59:19.997Z","contentChangedAt":"2026-09-08T15:59:19.997Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}