huggingface/transformers · error · ValueError

0 in strides is not supported for ExecuTorch.

Error message

0 in strides is not supported for ExecuTorch.

What it means

A patched version of ExecuTorch's stride-sorting logic (_spec_prop_pass) that checks each stride element with guard_or_false and raises ValueError when a stride contains 0 — ExecuTorch does not support 0-stride (broadcast/expanded) tensors. The patch exists because the upstream comparison throws GuardOnDataDependentSymNode on unbacked SymInts; here a definite 0 stride is instead rejected outright.

Source

Thrown at src/transformers/exporters/exporter_executorch.py:785

    "executorch.exir.emit._emitter.dim_order_from_stride",
    "executorch.exir.passes.replace_view_copy_with_view_pass.dim_order_from_stride",
)
def _patch_dim_order_from_stride(_original):
    """Replacement for ``executorch.exir.tensor.dim_order_from_stride``.

    The upstream version compares strides with ``guard_size_oblivious`` to sort
    them. When the strides are unbacked SymInts (e.g. ``splinter`` slicing on a
    data-dependent index), the comparison raises ``GuardOnDataDependentSymNode``
    deep inside ``spec_prop_pass``. Use ``guard_or_true`` / ``guard_or_false``
    so the sort still produces *a* dim order when the comparison is unbacked —
    the exact order on unbacked dims doesn't affect correctness, just memory layout.
    """
    from torch.fx.experimental.symbolic_shapes import guard_or_false, guard_or_true

    def patch(stride):
        for s in stride:
            if guard_or_false(s == 0):
                raise ValueError("0 in strides is not supported for ExecuTorch.")

        class K:
            __slots__ = ("stride",)

            def __init__(self, stride):
                self.stride = stride

            def __lt__(self, other):
                return guard_or_true(self.stride < other.stride)

        sorted_dims = [i[0] for i in sorted(enumerate(stride), key=lambda x: K(x[1]), reverse=True)]
        return tuple(sorted_dims)

    return patch


@register_patch("executorch", "executorch.exir.passes.spec_prop_pass.SpecPropPass.update_placeholder_tensor_specs")
def _patch_update_placeholder_tensor_specs(_original):

View on GitHub (pinned to a597f97485)

Solutions

1. Make the offending tensors contiguous: call .contiguous() (or .reshape(...)) on expanded tensors, or use arithmetic broadcasting instead of expand.
2. Use the exporters' _make_contiguous helper semantics: pass contiguous sample inputs (the shipped prepare hooks already do this for inputs — check model-internal expands).
3. Find the culprit by scanning the traced graph for expand/as_strided nodes with 0 stride before lowering.

Example fix

# before (model code)
mask = torch.ones(1, 1, S, S).expand(B, H, S, S)  # stride 0 on B,H

# after (model code)
mask = torch.ones(1, 1, S, S).expand(B, H, S, S).contiguous()

def inputs_have_zero_stride(inputs) -> list[str]:
    return [k for k, v in inputs.items() if isinstance(v, torch.Tensor) and 0 in v.stride()]

bad = inputs_have_zero_stride(sample_inputs)
if bad:
    sample_inputs.update({k: sample_inputs[k].contiguous() for k in bad})

def is_executorch_safe(tensor: torch.Tensor) -> bool:
    return 0 not in tensor.stride()

try:
    ExecutorchExporter().export(model, inputs, cfg)
except ValueError as e:
    if "0 in strides" in str(e):
        # zero-stride tensor is model-internal; locate expand()/as_strided() calls and .contiguous() them
        raise
    raise

When it happens

Trigger: Exporting to ExecuTorch with sample inputs or intermediate tensors produced by torch.expand/as_strided with stride 0 (e.g. an expanded bias or mask broadcast along a dimension), so a 0 stride reaches stride propagation.

Common situations: Models that call tensor.expand(...) on weights/masks instead of broadcasting; expanded attention masks or position embeddings; inputs passed non-contiguous after a broadcast.

Related errors

• Expected config to be an ExecutorchConfig or dict, got {type
• Unsupported backend {config.backend} for ExecuTorch export
• CUDA is not available in this environment; cannot export to
• `num_head` was provided as a list of length {len(num_heads)}
• `head_dim` was provided as a list of length {len(num_heads)}

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/d236fcacb01c1667. Report an issue: GitHub.