cocoindex-io/cocoindex · warning

@serialize_by_pickle on

Error message

@serialize_by_pickle on {cls.__qualname__} (a {'dataclass' if dataclasses.is_dataclass(cls) else 'NamedTuple' if issubclass(cls, tuple) else 'msgspec.Struct'}) has no effect when nested inside another msgspec-compatible type, because msgspec encodes these types natively and bypasses the pickle hook. Consider restructuring the type to be fully msgspec-compatible.

What it means

Applying the @serialize_by_pickle decorator to a type that msgspec already encodes natively (dataclass, NamedTuple, or msgspec.Struct) has no effect when that type is nested inside another msgspec-compatible container: msgspec serializes it directly and never consults the pickle fallback hook. The decorator detects this at decoration time and warns so you don't falsely believe pickle is being used.

Solutions

  1. Remove @serialize_by_pickle and make the type fully msgspec-compatible (ensure all its fields are msgspec-encodable) so native encoding works correctly.
  2. If the type truly cannot be msgspec-encoded, change it so it is no longer a dataclass/NamedTuple/Struct (e.g. wrap the payload in a plain class or bytes) so the pickle hook actually applies.
  3. Restructure the parent container so the field is not nested inside a msgspec-encoded type if pickle semantics are required.

Example fix

// before
@serialize_by_pickle
@dataclass
class Embedding:
    vec: Any  # not msgspec-encodable
// after
@dataclass
class Embedding:
    vec: list[float]  # fully msgspec-compatible; drop the pickle decorator
Defensive patterns

Strategy: validation

Validate before calling

import dataclasses
from cocoindex._internal.serde import _is_msgspec_native_type
def pickle_hook_effective(cls: type) -> bool:
    return not _is_msgspec_native_type(cls)  # warn-free only if True

Type guard

def is_msgspec_native(cls: type) -> bool:
    return dataclasses.is_dataclass(cls) or issubclass(cls, tuple) or issubclass(cls, msgspec.Struct)

Try / catch

import warnings
with warnings.catch_warnings():
    warnings.simplefilter("error", UserWarning)
    try:
        decorated = serialize_by_pickle(MyType)
    except UserWarning:
        ...  # type is msgspec-native; make it msgspec-compatible instead

Prevention

When it happens

Trigger: Calling @serialize_by_pickle on (or applying it to) a class where _is_msgspec_native_type(cls) is True — i.e. a dataclass, a tuple subclass (NamedTuple), or msgspec.Struct — and later embedding it inside another msgspec-encoded type.

Common situations: Migrating a field type from a plain class to a dataclass/NamedTuple/Struct for typing reasons while keeping the old pickle decorator; the pickle hook silently stops firing for nested positions.

Understand the failure class

Background: "is deprecated and will be removed" — deprecation warnings for old API names, keywords, and options, and how to migrate before the removal release — this error's family across 29 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/3c448f3e5182e37d. Report an issue: GitHub.

Appendix: source

Thrown at python/cocoindex/_internal/serde.py:165

    _UNPICKLE_SAFE_GLOBALS[(module, qualname)] = obj


def _is_msgspec_native_type(cls: type) -> bool:
    """Check if a type is natively handled by msgspec (and thus bypasses enc_hook)."""
    if dataclasses.is_dataclass(cls):
        return True
    if isinstance(cls, type) and issubclass(cls, tuple) and hasattr(cls, "_fields"):
        # NamedTuple
        return True
    if isinstance(cls, type) and issubclass(cls, msgspec.Struct):
        return True
    return False


def serialize_by_pickle(cls: type) -> type:
    """Decorator: serialize this type with pickle. Auto-registers as unpickle-safe."""
    if _is_msgspec_native_type(cls):
        warnings.warn(
            f"@serialize_by_pickle on {cls.__qualname__} (a "
            f"{'dataclass' if dataclasses.is_dataclass(cls) else 'NamedTuple' if issubclass(cls, tuple) else 'msgspec.Struct'}"
            f") has no effect when nested inside another msgspec-compatible type, "
            f"because msgspec encodes these types natively and bypasses the pickle "
            f"hook. Consider restructuring the type to be fully msgspec-compatible.",
            stacklevel=2,
        )
    _SERIALIZE_BY_PICKLE_TYPES.add(cls)
    unpickle_safe(cls)
    return cls


# ---------------------------------------------------------------------------
# Restricted unpickler
# ---------------------------------------------------------------------------


class _RestrictedUnpickler(pickle.Unpickler):

View on GitHub (pinned to e84aa99b32)