{"record":{"id":"fd8a6d1fe2f01558","repo":"cocoindex-io/cocoindex","slug":"unknown-extension-code-code","errorCode":null,"errorMessage":"Unknown extension code: {code}","messagePattern":"Unknown extension code: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/cocoindex/_internal/serde.py","lineNumber":296,"sourceCode":"    # B: Bridge Pydantic into msgspec\n    if _is_pydantic_instance(obj):\n        return obj.model_dump(mode=\"json\")\n    raise NotImplementedError(f\"Cannot serialize {type(obj).__name__}\")\n\n\n_msgspec_encoder = msgspec.msgpack.Encoder(enc_hook=_enc_hook)\n\n\n# ---------------------------------------------------------------------------\n# Deserialization hooks\n# ---------------------------------------------------------------------------\n\n\ndef _ext_hook(code: int, data: memoryview) -> Any:  # type: ignore[type-arg]\n    \"\"\"Un-quarantine pickle inside msgspec payloads.\"\"\"\n    if code == 100:\n        return _RestrictedUnpickler(io.BytesIO(bytes(data))).load()\n    raise ValueError(f\"Unknown extension code: {code}\")\n\n\ndef _dec_hook(type_hint: Any, obj: Any) -> Any:\n    \"\"\"Handle custom types during msgspec decoding.\n\n    Called when msgspec encounters a type it doesn't natively support.\n    Only two cases reach here:\n\n    1. Pydantic models — enc_hook serialized via model_dump(mode=\"json\"),\n       so *obj* is a dict that needs model_validate to reconstruct.\n    2. Pickle-quarantined values — ext_hook already reconstructed the\n       correct object; just pass through.\n    \"\"\"\n    if _is_pydantic_model_type(type_hint):\n        return type_hint.model_validate(obj)\n    return obj\n\n","sourceCodeStart":278,"sourceCodeEnd":314,"githubUrl":"https://github.com/cocoindex-io/cocoindex/blob/e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b/python/cocoindex/_internal/serde.py#L278-L314","documentation":"Raised by the msgspec extension hook `_ext_hook` in cocoindex's serde layer. CocoIndex encodes custom types inside msgpack payloads as msgspec extension blobs, all of which must use extension code 100 (its pickle quarantine channel). If a msgpack payload contains an extension record with any other code, the decoder cannot interpret it and raises this ValueError, indicating the payload was not produced by cocoindex's serializer or was corrupted/hand-crafted.","triggerScenarios":"Decoding a msgpack buffer (via `deserialize`, `make_deserialize_fn`, or a msgspec Decoder built with ext_hook=_ext_hook) whose bytes contain a msgpack ext extension with a code other than 100 — e.g. data serialized by another library's msgpack encoder that emitted ext records, or a payload hand-modified/corrupted.","commonSituations":"Feeding payloads produced by a different msgpack encoder into cocoindex's deserializer; mixing versions where an older/experimental cocoindex build used a different ext code; manual binary editing of persisted state (memoization/state files); tests crafting raw msgpack bytes with custom ext codes.","solutions":["Regenerate the payload with cocoindex's own `serialize()` so extension values are encoded with code 100","Verify the bytes being deserialized actually came from this library/version (check for stale or foreign persisted state files and delete/rebuild them)","Ensure you are deserializing the correct field — a corrupted offset can make non-ext bytes be read as an ext record","If you craft msgpack in tests, wrap custom objects with msgspec.msgpack.Ext(100, pickle_bytes) instead of other codes"],"exampleFix":"// before (hand-crafted payload with wrong ext code)\nmsgpack.encode({\"x\": Ext(7, b\"...\")})  # -> Unknown extension code: 7\n// after\nfrom cocoindex._internal import serde\npayload = serde.serialize(my_value)  # ext values encoded with code 100\nserde.deserialize(payload, type_hint=MyType)","handlingStrategy":"try-catch","validationCode":"# only safe if you know the producer\nif data[0] != 0x01:\n    raise ValueError(\"not a msgspec-routed payload; ext codes not applicable\")","typeGuard":"def is_cocoindex_payload(data: bytes) -> bool:\n    return bool(data) and data[0] in (0x01, 0x02, 0x80)","tryCatchPattern":"try:\n    value = serde.deserialize(data, type_hint=MyType)\nexcept serde.DeserializationError as e:\n    if \"Unknown extension code\" in str(e.__cause__):\n        data = regenerate_payload()  # re-serialize with cocoindex.serialize\n        value = serde.deserialize(data, type_hint=MyType)\n    else:\n        raise","preventionTips":["Only deserialize bytes produced by cocoindex's own serialize()","Never hand-edit persisted msgpack state files","When crafting test msgpack, always use ext code 100 for custom objects","Keep the cocoindex version consistent between writer and reader of payloads"],"tags":["python","serialization","msgpack","corrupt-data"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"e84aa99b3292c5270a4b313b2a7137ad9ce8ab3b","analyzedAt":"2026-09-08T15:59:19.997Z","contentChangedAt":"2026-09-08T15:59:19.997Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}