vllm-project/vllm · error · ValueError

Input {arg} to maybe_inplace node {node} is used again after

Error message

Input {arg} to maybe_inplace node {node} is used again after the node. This is not allowed; activation inputs to maybe_inplace ops are donated to the op, meaning their memory may be recycled for outputs.

To preserve the inputs, use the default overload or clone them manually beforehand.

What it means

During in-place functionalization of vLLM's IR, ops with a maybe_inplace overload donate their activation input buffers: the input's memory may be recycled as the output. If any later node still reads that input, it would observe silently-corrupted data, so the pass raises ValueError when an input to a maybe_inplace node has a user scheduled after the node.

Source

Thrown at vllm/compilation/passes/ir/inplace_functionalization.py:73

            op_overload = overload_or_default(node.target)
            overload_name = op_overload._overloadname
            if overload_name != "maybe_inplace":
                assert overload_name == "default", (
                    f"Found overload {overload_name} for op {ir_op.name}, "
                    f"expected maybe_inplace or default"
                )
                continue

            # must have maybe_inplace overload and allow_inplace
            assert ir_op.allow_inplace and hasattr(ir_op, "maybe_inplace")

            # Check that activation inputs are not used after this op
            for arg_idx in ir_op.activation_indices:
                arg = node.args[arg_idx]
                assert isinstance(arg, fx.Node), "Activation inputs must be fx.Node"
                for user in arg.users:
                    if node_to_idx[user] > node_to_idx[node]:
                        raise ValueError(
                            f"Input {arg} to maybe_inplace node {node} "
                            f"is used again after the node. "
                            f"This is not allowed; activation inputs to maybe_inplace "
                            f"ops are donated to the op, meaning their memory may be "
                            f"recycled for outputs.\n\n"
                            f"To preserve the inputs, use the default overload or "
                            f"clone them manually beforehand."
                        )

                if arg.op == "placeholder":
                    # Graph input that maybe_inplace might modify.
                    # Mark it so downstream passes know it's donated.
                    # TODO(luka) store in placeholder node meta once supported
                    pass_context.donated_input_ids.add(node_to_idx[arg])

            # Same signature, just replace the overload that's called.
            node.target = ir_op.torch_op
            self.functionalized_ops[ir_op.name] += 1

View on GitHub (pinned to c794754062)

Solutions

Use the default (out-of-place) overload of the op so the input is preserved.
Or explicitly clone the input before it reaches the maybe_inplace op: x = x.clone().
If you own the op, re-check that allow_inplace is only set when the graph truly has no later consumers of the donated input.

Example fix

# before
h = maybe_inplace_rmsnorm(h, w)
out = h + residual  # residual reads h's donated buffer later -> ValueError
# after
h = maybe_inplace_rmsnorm(h.clone(), w)
out = h + residual

Defensive patterns

Strategy: validation

Validate before calling

def donation_safe(nodes_order, node, arg) -> bool:
    return all(user in nodes_order[:nodes_order.index(node)] for user in arg.users)
# verify no later users of the activation before routing it to a maybe_inplace op

Try / catch

try:
    compile_with_inplace_pass(model)
except ValueError as e:
    if 'donated' in str(e):
        switch_op_to_default_overload(); compile_with_inplace_pass(model)
    else:
        raise

Prevention

Prefer default out-of-place overloads unless donation is proven safe
Clone activations with multiple consumers
Run the inplace functionalization pass in unit tests for custom ops

When it happens

Trigger: Authoring a custom op with allow_inplace/maybe_inplace overload and registering it in a graph where the donated activation tensor is consumed again by a subsequent op; happens during vLLM graph fixing (compilation with piecewise/inplace passes enabled).

Common situations: Adding a fused custom kernel that takes hidden_states by donation (e.g. RMSNorm-style in-place variants) while downstream code (residual adds, logging) still uses the original tensor; reordering passes so a donation happens before other consumers.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/3b587cc513404f2f. Report an issue: GitHub.