tracel-ai/burn · error

graph replay should succeed

Error message

graph replay should succeed

What it means

GraphHandle::replay() records and executes the captured CUDA/WGPU graph on the device; the dispatch call returns a Result and any backend error (invalid graph, device reset, unsupported op captured) is turned into a panic by this expect. The unsafe method's contract requires the graph to remain valid and replayable.

Solutions

1. Verify the graph was captured successfully and the device has not been reset before replaying
2. Re-capture the graph if the device/adapter changed since capture
3. Check backend support for graph capture/replay (some CUDA/WGPU versions or adapters don't support it)
4. Fall back to the non-graph closure execution path if replay is unreliable

Example fix

// before
unsafe { graph.replay() };
// after
if graph_captures_valid && device_healthy {
    unsafe { graph.replay() };
} else {
    output = closure(); // fall back to eager execution
}

if graph.hardware.is_none() || device_was_reset() {
    // fall back to eager closure execution instead of replay
    output = closure();
}

// replay() itself is unsafe and panics via expect; guard the device state first
if graph_valid && !device_lost {
    unsafe { graph.replay() };
} else {
    output = (closure)();
}

When it happens

Trigger: Calling `graph.replay()` after the underlying device graph was invalidated or the device was reset; a backend whose graph capture includes unsupported operations that fail at replay time; calling replay on a closed/already-released handle.

Common situations: CUDA graph capture misuse (capturing streams that performed illegal ops); hot-reload or device-loss recovery paths replaying stale graphs; experimental graph capture on WGPU adapters lacking support.

Related errors

• Failed to read tensor data synchronously. Try using…
• Failed to read tensor data synchronously. Try using…
• rfft kernel launch failed
• kernel failed (device= , dtype= )
• BlackmanWindow size doesn't fit in i64 range.

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/41c90a7a5a8046fb. Report an issue: GitHub.

Appendix: source

Thrown at crates/burn-tensor/src/tensor/api/graph.rs:133

    ///   Tensors owned by the closure (or by `self`, like the output) are kept
    ///   alive automatically; tensors the closure only borrowed must outlive
    ///   the replays.
    /// - **No concurrent use** — no other stream or thread reads or writes a
    ///   tensor shared with the graph while the replay executes; the replay is
    ///   only ordered against work on its own capture stream.
    /// - **Same-stream refreshes** — input refreshes and output reads are
    ///   issued on the stream the graph was captured on (the same device
    ///   thread/client), so they order correctly against the replay rather
    ///   than racing it with stale or torn data.
    ///
    /// On the fallback path (no hardware graph) this simply re-runs the closure
    /// and is trivially safe.
    pub unsafe fn replay(&mut self) -> &T {
        match &self.hardware {
            Some(graph) => {
                // Safety: forwarded verbatim from this method's own contract.
                unsafe { Dispatch::graph_replay(self.device.as_dispatch(), graph) }
                    .expect("graph replay should succeed");
            }
            None => {
                self.output = (self.closure)();
            }
        }
        &self.output
    }

    /// The output tensor(s) the graph writes to — stable across replays on the
    /// hardware path (the same buffer is overwritten each time).
    pub fn output(&self) -> &T {
        &self.output
    }

    /// Whether this graph replays as a hardware dispatch (`true`) or by
    /// re-running the closure (`false`).
    pub fn is_hardware(&self) -> bool {
        self.hardware.is_some()

View on GitHub (pinned to d16f7ba2ed)