{"record":{"id":"cace1e2353f52330","repo":"tracel-ai/burn","slug":"failed-to-read-a-tensor-off-from-on-the-way-to","errorCode":null,"errorMessage":"Failed to read a tensor off {from:?} on the way to {to:?}: {err}","messagePattern":"Failed to read a tensor off (.+?) on the way to (.+?): (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/burn-cubecl/src/ops/base.rs","lineNumber":129,"sourceCode":"        .client\n        .read_one(tensor.handle.clone())\n        .unwrap_or_else(|err| transfer_failed(&tensor.device, device, err));\n\n    let client = device.client();\n    let handle = client.create(bytes);\n\n    CubeTensor {\n        client,\n        handle,\n        meta: tensor.meta,\n        device: device.clone(),\n        dtype: tensor.dtype,\n        qparams: tensor.qparams,\n    }\n}\n\nfn transfer_failed(from: &CubeDevice, to: &CubeDevice, err: impl core::fmt::Display) -> ! {\n    panic!(\"Failed to read a tensor off {from:?} on the way to {to:?}: {err}\")\n}\n\npub(crate) fn empty(shape: Shape, device: &CubeDevice, dtype: DType) -> CubeTensor {\n    let client = device.client();\n    let alloc = client.empty_tensor(shape.clone(), dtype.size());\n\n    CubeTensor::new(\n        client,\n        alloc.memory,\n        Metadata::new(shape, alloc.strides),\n        device.clone(),\n        dtype,\n    )\n}\n\npub(crate) fn swap_dims(mut tensor: CubeTensor, dim1: usize, dim2: usize) -> CubeTensor {\n    tensor.meta.swap(dim1, dim2);\n","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/tracel-ai/burn/blob/d16f7ba2ed0d41408189384044cc886fb4c8f957/crates/burn-cubecl/src/ops/base.rs#L111-L147","documentation":"transfer_failed panics when reading a tensor's data off one CubeCL device fails while moving it to another runtime/device (to_device_across_runtimes). The message names the source device, the destination device, and the underlying read error. Cross-runtime transfer requires reading the tensor to host memory, so a failed read blocks the whole migration.","triggerScenarios":"Calling to_device_across_runtimes (e.g. moving a tensor from a CUDA client to a Vulkan/Metal/wgpu client) when the source runtime errors while reading tensor data — invalid handle, runtime driver error, or the tensor was freed.","commonSituations":"Mixed-backend setups (cuda + wgpu) where the source runtime crashed or the driver misbehaves, transferring tensors after the source client was dropped, or dtype/shape mismatch causing an invalid read size.","solutions":["Read the inner `{err}` for the concrete read failure from the source runtime","Verify the source device/client is still alive and the runtime is healthy","Re-create the tensor on the source device if the handle became invalid, then transfer","As a workaround, read the tensor to host with an explicit to_data() and re-send it to the target device"],"exampleFix":"// before\nlet on_wgpu = tensor.to_device_across_runtimes(&wgpu_device);\n// after: host round-trip fallback\nlet data = tensor.into_data().unwrap();\nlet on_wgpu = Tensor::from_data(data, &wgpu_device);","handlingStrategy":"try-catch","validationCode":"// verify both runtimes are alive before transfer\nassert!(source_client.is_valid(), \"source runtime unavailable\");","typeGuard":null,"tryCatchPattern":"match std::panic::catch_unwind(|| tensor.to_device_across_runtimes(&target)) {\n    Ok(t) => t,\n    Err(_) => {\n        let data = tensor.into_data().expect(\"host read\");\n        Tensor::from_data(data, &target)\n    }\n}","preventionTips":["Prefer host round-trip (into_data + from_data) when crossing runtimes","Avoid holding tensors on a client you're about to drop","Test mixed-backend transfers once at startup, not mid-training"],"tags":["gpu","device-transfer","memory","burn"],"backgroundTag":"device-transfer-failed","analyzedSha":"d16f7ba2ed0d41408189384044cc886fb4c8f957","analyzedAt":"2026-09-05T13:19:14.260Z","contentChangedAt":"2026-09-05T13:19:14.260Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}