{"record":{"id":"3b587cc513404f2f","repo":"vllm-project/vllm","slug":"input-arg-to-maybe-inplace-node-node-is-used-a","errorCode":null,"errorMessage":"Input {arg} to maybe_inplace node {node} is used again after the node. This is not allowed; activation inputs to maybe_inplace ops are donated to the op, meaning their memory may be recycled for outputs.\n\nTo preserve the inputs, use the default overload or clone them manually beforehand.","messagePattern":"Input (.+?) to maybe_inplace node (.+?) is used again after the node\\. This is not allowed; activation inputs to maybe_inplace ops are donated to the op, meaning their memory may be recycled for outputs\\.\n\nTo preserve the inputs, use the default overload or clone them manually beforehand\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/compilation/passes/ir/inplace_functionalization.py","lineNumber":73,"sourceCode":"            op_overload = overload_or_default(node.target)\n            overload_name = op_overload._overloadname\n            if overload_name != \"maybe_inplace\":\n                assert overload_name == \"default\", (\n                    f\"Found overload {overload_name} for op {ir_op.name}, \"\n                    f\"expected maybe_inplace or default\"\n                )\n                continue\n\n            # must have maybe_inplace overload and allow_inplace\n            assert ir_op.allow_inplace and hasattr(ir_op, \"maybe_inplace\")\n\n            # Check that activation inputs are not used after this op\n            for arg_idx in ir_op.activation_indices:\n                arg = node.args[arg_idx]\n                assert isinstance(arg, fx.Node), \"Activation inputs must be fx.Node\"\n                for user in arg.users:\n                    if node_to_idx[user] > node_to_idx[node]:\n                        raise ValueError(\n                            f\"Input {arg} to maybe_inplace node {node} \"\n                            f\"is used again after the node. \"\n                            f\"This is not allowed; activation inputs to maybe_inplace \"\n                            f\"ops are donated to the op, meaning their memory may be \"\n                            f\"recycled for outputs.\\n\\n\"\n                            f\"To preserve the inputs, use the default overload or \"\n                            f\"clone them manually beforehand.\"\n                        )\n\n                if arg.op == \"placeholder\":\n                    # Graph input that maybe_inplace might modify.\n                    # Mark it so downstream passes know it's donated.\n                    # TODO(luka) store in placeholder node meta once supported\n                    pass_context.donated_input_ids.add(node_to_idx[arg])\n\n            # Same signature, just replace the overload that's called.\n            node.target = ir_op.torch_op\n            self.functionalized_ops[ir_op.name] += 1","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/compilation/passes/ir/inplace_functionalization.py#L55-L91","documentation":"During in-place functionalization of vLLM's IR, ops with a maybe_inplace overload donate their activation input buffers: the input's memory may be recycled as the output. If any later node still reads that input, it would observe silently-corrupted data, so the pass raises ValueError when an input to a maybe_inplace node has a user scheduled after the node.","triggerScenarios":"Authoring a custom op with allow_inplace/maybe_inplace overload and registering it in a graph where the donated activation tensor is consumed again by a subsequent op; happens during vLLM graph fixing (compilation with piecewise/inplace passes enabled).","commonSituations":"Adding a fused custom kernel that takes hidden_states by donation (e.g. RMSNorm-style in-place variants) while downstream code (residual adds, logging) still uses the original tensor; reordering passes so a donation happens before other consumers.","solutions":["Use the default (out-of-place) overload of the op so the input is preserved.","Or explicitly clone the input before it reaches the maybe_inplace op: x = x.clone().","If you own the op, re-check that allow_inplace is only set when the graph truly has no later consumers of the donated input."],"exampleFix":"# before\nh = maybe_inplace_rmsnorm(h, w)\nout = h + residual  # residual reads h's donated buffer later -> ValueError\n# after\nh = maybe_inplace_rmsnorm(h.clone(), w)\nout = h + residual","handlingStrategy":"validation","validationCode":"def donation_safe(nodes_order, node, arg) -> bool:\n    return all(user in nodes_order[:nodes_order.index(node)] for user in arg.users)\n# verify no later users of the activation before routing it to a maybe_inplace op","typeGuard":null,"tryCatchPattern":"try:\n    compile_with_inplace_pass(model)\nexcept ValueError as e:\n    if 'donated' in str(e):\n        switch_op_to_default_overload(); compile_with_inplace_pass(model)\n    else:\n        raise","preventionTips":["Prefer default out-of-place overloads unless donation is proven safe","Clone activations with multiple consumers","Run the inplace functionalization pass in unit tests for custom ops"],"tags":["compilation","inplace-ops","custom-ops","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}