tracel-ai/burn · error

graph replay should succeed

Error message

graph replay should succeed

What it means

GraphHandle::replay() records and executes the captured CUDA/WGPU graph on the device; the dispatch call returns a Result and any backend error (invalid graph, device reset, unsupported op captured) is turned into a panic by this expect. The unsafe method's contract requires the graph to remain valid and replayable.

Source

Thrown at crates/burn-tensor/src/tensor/api/graph.rs:133

    ///   Tensors owned by the closure (or by `self`, like the output) are kept
    ///   alive automatically; tensors the closure only borrowed must outlive
    ///   the replays.
    /// - **No concurrent use** — no other stream or thread reads or writes a
    ///   tensor shared with the graph while the replay executes; the replay is
    ///   only ordered against work on its own capture stream.
    /// - **Same-stream refreshes** — input refreshes and output reads are
    ///   issued on the stream the graph was captured on (the same device
    ///   thread/client), so they order correctly against the replay rather
    ///   than racing it with stale or torn data.
    ///
    /// On the fallback path (no hardware graph) this simply re-runs the closure
    /// and is trivially safe.
    pub unsafe fn replay(&mut self) -> &T {
        match &self.hardware {
            Some(graph) => {
                // Safety: forwarded verbatim from this method's own contract.
                unsafe { Dispatch::graph_replay(self.device.as_dispatch(), graph) }
                    .expect("graph replay should succeed");
            }
            None => {
                self.output = (self.closure)();
            }
        }
        &self.output
    }

    /// The output tensor(s) the graph writes to — stable across replays on the
    /// hardware path (the same buffer is overwritten each time).
    pub fn output(&self) -> &T {
        &self.output
    }

    /// Whether this graph replays as a hardware dispatch (`true`) or by
    /// re-running the closure (`false`).
    pub fn is_hardware(&self) -> bool {
        self.hardware.is_some()

View on GitHub (pinned to d16f7ba2ed)

Solutions

1. Verify the graph was captured successfully and the device has not been reset before replaying
2. Re-capture the graph if the device/adapter changed since capture
3. Check backend support for graph capture/replay (some CUDA/WGPU versions or adapters don't support it)
4. Fall back to the non-graph closure execution path if replay is unreliable

Example fix

// before
unsafe { graph.replay() };
// after
if graph_captures_valid && device_healthy {
    unsafe { graph.replay() };
} else {
    output = closure(); // fall back to eager execution
}

if graph.hardware.is_none() || device_was_reset() {
    // fall back to eager closure execution instead of replay
    output = closure();
}

// replay() itself is unsafe and panics via expect; guard the device state first
if graph_valid && !device_lost {
    unsafe { graph.replay() };
} else {
    output = (closure)();
}

When it happens

Trigger: Calling `graph.replay()` after the underlying device graph was invalidated or the device was reset; a backend whose graph capture includes unsupported operations that fail at replay time; calling replay on a closed/already-released handle.

Common situations: CUDA graph capture misuse (capturing streams that performed illegal ops); hot-reload or device-loss recovery paths replaying stale graphs; experimental graph capture on WGPU adapters lacking support.

Related errors

• rfft kernel launch failed (device={input_device:?}, dtype={i
• Failed to read tensor data synchronously. Try using nonzero_
• Failed to read tensor data synchronously. Try using argwhere
• Not a valid DType for tensors.
• capture tensor operations must run inside CaptureDevice::cap

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/41c90a7a5a8046fb. Report an issue: GitHub.