huggingface/transformers · error · TypeError

Cannot flatten a bound method for pytree context

Error message

Cannot flatten a bound method for pytree context

What it means

During serialization of a traced program's pytree context, _flatten_to_context refuses to serialize bound methods (types.MethodType). This happens when a model's forward attaches methods onto objects it returns at trace time — the comment cites recurrent_gemma binding get_seq_length/get_mask_sizes onto its DynamicCache. Bound methods cannot be captured in an ExportedProgram's constants, so the pattern is unexportable by design; the model is skipped until the binding is refactored.

Source

Thrown at src/transformers/exporters/exporter_dynamo.py:513

    cls = type(obj)
    if isinstance(obj, dict):  # dict subclasses (OrderedDict, etc.)
        return {
            "_t": "map",
            "p": _class_to_path(cls),
            "v": {k: _flatten_to_context(v, tensors) for k, v in obj.items()},
        }
    if isinstance(obj, (tuple, list, set, frozenset)):  # sequences/sets incl. NamedTuple
        return {
            "_t": "seq",
            "p": _class_to_path(cls),
            "v": [_flatten_to_context(i, tensors) for i in obj],
        }
    if isinstance(obj, types.MethodType):
        # A bound method can't be flattened into pytree context. Models shouldn't bind methods onto
        # objects they return at forward time (e.g. recurrent_gemma binds `get_seq_length`/
        # `get_mask_sizes` onto its `DynamicCache`) — that pattern isn't exportable; such a model is
        # skipped until the binding is refactored away (e.g. into a `Cache` subclass).
        raise TypeError("Cannot flatten a bound method for pytree context")
    if hasattr(obj, "__dict__"):
        state = {k: _flatten_to_context(v, tensors) for k, v in vars(obj).items()}
        return {"_t": "obj", "p": _class_to_path(cls), "s": state}

    raise TypeError(f"Cannot flatten {type(obj).__name__} for pytree context")


def _unflatten_from_context(ctx: Any, tensors: list) -> Any:
    """Reconstruct an object from its JSON-native context, substituting tensor index markers."""
    # --- Pure Python / JSON-native ---
    if ctx is None or type(ctx) in (bool, int, float, str):
        return ctx
    if type(ctx) is list:
        return [_unflatten_from_context(i, tensors) for i in ctx]
    if type(ctx) is dict and "_t" not in ctx:
        return {k: _unflatten_from_context(v, tensors) for k, v in ctx.items()}

    # --- Torch objects ---

View on GitHub (pinned to a597f97485)

Solutions

1. Refactor the model: move get_seq_length/get_mask_sizes into a proper Cache subclass instead of binding them at forward time.
2. Use a model revision where the binding was already refactored away (check the model repo for updates).
3. As a maintainer of a custom model, never attach methods to returned objects; subclass the cache class instead.

Example fix

# before (model code, unexportable)
cache.get_seq_length = lambda: cache.seen_tokens  # bound at forward time

# after (model code)
class MyCache(DynamicCache):
    def get_seq_length(self):  # real method on a Cache subclass
        return self.seen_tokens

# Detect the anti-pattern before exporting (model-side check):
import types

def binds_methods_at_forward(model) -> bool:
    # smoke-trace a single step and inspect returned cache for bound methods
    out = model(**sample_inputs)
    cache = getattr(out, "past_key_values", None)
    return any(isinstance(v, types.MethodType) for v in (vars(cache).values() if cache else []))

def is_exportable_cache(cache) -> bool:
    import types
    return not any(isinstance(v, types.MethodType) for v in vars(cache).values())

try:
    DynamoExporter().export(model, inputs, cfg)
except TypeError as e:
    if "bound method" in str(e):
        logger.warning("%s binds methods onto its cache; skip or refactor model", model.config.name_or_path)
        raise

When it happens

Trigger: Exporting recurrent_gemma (or any model whose Cache gets methods bound at forward time) with DynamoExporter; the flattening walks the model's outputs/state and hits the MethodType branch.

Common situations: Exporting new recurrent models that dynamically extend DynamicCache with helper methods; a fine-tuned or community model that copied the bind-methods-onto-cache pattern.

Related errors

• Cannot flatten {type(obj).__name__} for pytree context
• Expected config to be a DynamoConfig or dict, got {type(conf
• `num_head` was provided as a list of length {len(num_heads)}
• `head_dim` was provided as a list of length {len(num_heads)}
• export_config_dict must contain key 'export_format' set to e

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/f19d782d1c81f056. Report an issue: GitHub.