cocoindex-io/cocoindex · error · DeserializationError

Unknown routing byte: {routing_byte:#x} ({_error_context()})

Error message

Unknown routing byte: {routing_byte:#x} ({_error_context()})

What it means

This DeserializationError is thrown when a serialized payload's leading routing byte does not match any format the deserializer knows how to handle. The routing byte tells _deserialize which codec (e.g. pickle, native, JSON) produced the payload; an unknown value means the bytes were not produced by this library's serializer or were corrupted/truncated.

Source

Thrown at python/cocoindex/_internal/serde.py:433

                raw = msgspec.msgpack.decode(mv[1:], ext_hook=_ext_hook)
                if type_hint is Any:
                    return raw
                return pydantic_adapter.validate_python(raw)
            except Exception as e:
                raise DeserializationError(
                    f"Failed to deserialize pydantic payload ({_error_context()})"
                ) from e

        # C: Pickle (legacy and @serialize_by_pickle)
        if routing_byte == 0x80:
            try:
                return _RestrictedUnpickler(io.BytesIO(bytes(mv))).load()
            except Exception as e:
                raise DeserializationError(
                    f"Failed to deserialize pickle payload ({_error_context()})"
                ) from e

        raise DeserializationError(
            f"Unknown routing byte: {routing_byte:#x} ({_error_context()})"
        )

    return _deserialize


# ---------------------------------------------------------------------------
# Top-level serialize / deserialize
# ---------------------------------------------------------------------------


def serialize(value: Any) -> bytes:
    """Serialize a value using the routing-byte protocol (C → B → A priority)."""
    # C: Explicit pickle (user opted in — highest priority)
    if type(value) in _SERIALIZE_BY_PICKLE_TYPES:
        return _strict_pickle_dumps(value)

    # B: Pydantic BaseModel

View on GitHub (pinned to e84aa99b32)

Solutions

1. Re-serialize the data with the same cocoindex version's serializer instead of feeding foreign bytes to _deserialize
2. Check for a cocoindex version mismatch between the process writing the payloads and the one reading them, and align versions
3. Verify the payload bytes are intact (not truncated/shifted); inspect the first byte against the codec constants in serde.py
4. If migrating from an old version, rebuild the stored data (re-run the pipeline) rather than decoding old payloads directly

Example fix

// before
obj = deserialize(raw_pickle_bytes)  # unknown routing byte
// after
obj = deserialize(serialize(original_obj))  # round-trip through the library's serializer

def is_library_serialized(payload: bytes) -> bool:
    return len(payload) > 0 and payload[0] in KNOWN_ROUTING_BYTES  # constants from serde.py

def has_known_routing_byte(payload: bytes) -> bool:
    return bool(payload) and payload[0] in {b'\x01', b'\x02'}  # adjust to serde constants

try:
    obj = deserialize(payload)
except DeserializationError as e:
    log.error("unrecognized payload: %s", e)
    obj = recompute_and_serialize()

When it happens

Trigger: Calling a deserialization function (the closure returned by the serializer factory) on bytes whose first byte is not one of the recognized routing-byte constants — e.g. passing raw pickle bytes that were never wrapped by the library's serializer, hand-crafted payloads, or data written by a different cocoindex version with a different byte scheme.

Common situations: Reading cached/indexed payloads written by an older or newer cocoindex version after upgrading; manually loading data stored in an LMDB/DB column assuming it is plain pickle; corrupted or truncated serialized blobs.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• Cannot serialize {type(obj).__name__}
• Unknown extension code: {code}
• Cannot deserialize msgspec payload ({_error_context()})
• Failed to deserialize msgspec payload ({_error_context()})
• Failed to deserialize pydantic payload ({_error_context()})

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/dd6b32a501509889. Report an issue: GitHub.