cocoindex-io/cocoindex · error · DeserializationError

Failed to deserialize msgspec payload ({_error_context()})

Error message

Failed to deserialize msgspec payload ({_error_context()})

What it means

Raised when a msgpack payload with msgspec routing byte 0x01 is decoded but `decoder.decode()` raises — the bytes do not conform to what the declared type hint expects. CocoIndex wraps any underlying msgspec exception (DecodeError, ValidationError, UnpicklingError from the ext hook, etc.) in a DeserializationError that includes the type hint and source label, chaining the original exception as cause.

Source

Thrown at python/cocoindex/_internal/serde.py:402

        if source_label is not None:
            parts.append(f"source={source_label}")
        return ", ".join(parts)

    def _deserialize(data: bytes | memoryview) -> Any:
        nonlocal pydantic_adapter
        mv = memoryview(data) if not isinstance(data, memoryview) else data
        routing_byte = mv[0]

        # A: Msgspec (most common)
        if routing_byte == 0x01:
            if decoder is None:
                raise DeserializationError(
                    f"Cannot deserialize msgspec payload ({_error_context()})"
                ) from decoder_error
            try:
                return decoder.decode(mv[1:])
            except Exception as e:
                raise DeserializationError(
                    f"Failed to deserialize msgspec payload ({_error_context()})"
                ) from e

        # B: Pydantic
        if routing_byte == 0x02:
            try:
                if pydantic_adapter is None:
                    with pydantic_lock:
                        if pydantic_adapter is None:
                            import pydantic

                            pydantic_adapter = pydantic.TypeAdapter(type_hint)
                raw = msgspec.msgpack.decode(mv[1:], ext_hook=_ext_hook)
                if type_hint is Any:
                    return raw
                return pydantic_adapter.validate_python(raw)
            except Exception as e:
                raise DeserializationError(

View on GitHub (pinned to e84aa99b32)

Solutions

1. Inspect the chained cause (`e.__cause__`) for the exact msgspec/validation error and fix the mismatch between payload and type hint
2. Clear stale memoization/persisted state (delete the environment's db/state directory) so payloads are regenerated with the current type annotations
3. If you changed a type's fields, migrate old payloads or use a new component path / new function name so old entries are not reused
4. If the underlying error is 'Forbidden global during unpickling', register the class with @cocoindex unpickle_safe (or @serialize_by_pickle) and re-serialize

Example fix

// before: annotation changed after memoized payloads were saved
@coco.fn(memo=True)
def process(x: list[int]) -> int: ...  # old payload saved as tuple shape
// after: rebuild state
coco_env.drop_blocking()  # or delete state dir
app.update_blocking()     # regenerate payloads with current annotations

# verify the stored payload's routing byte matches expectations before decode
assert data[0] == 0x01, "payload is not msgspec-routed"

try:
    value = serde.deserialize(data, type_hint=MyType)
except serde.DeserializationError as e:
    print(e.__cause__)  # exact msgspec/validation error
    value = serde.deserialize(regenerate_payload(), type_hint=MyType)

When it happens

Trigger: `deserialize(data, type_hint)` where the 0x01-routed bytes: contain a different structure than type_hint (e.g. saved when the type was `int`, now decoded as `MyDataclass`); nest a custom object whose restricted unpickle is forbidden (find_class not in the allow-list); have truncated/corrupted msgpack bytes; or the stored data predates a type change.

Common situations: Changing a @coco.fn's parameter/return annotation after memoized results were persisted; decoding state files from a different cocoindex version; dataclasses whose fields changed shape; a class renamed/moved so the pickle quarantine can no longer resolve it; truncated files from interrupted writes.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• Cannot serialize {type(obj).__name__}
• Unknown extension code: {code}
• Cannot deserialize msgspec payload ({_error_context()})
• Failed to deserialize pydantic payload ({_error_context()})
• Failed to deserialize pickle payload ({_error_context()})

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/42a2b96609df4002. Report an issue: GitHub.