{"record":{"id":"ca748032b89df1c1","repo":"xai-org/x-algorithm","slug":"all-block-sparse-tensors-must-be-on-the-same-devic","errorCode":null,"errorMessage":"All block sparse tensors must be on the same device","messagePattern":"All block sparse tensors must be on the same device","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py","lineNumber":423,"sourceCode":"        expected_count_shape,\n        expected_index_shape,\n        context,\n        hint,\n    )\n    if mask_cnt is None or mask_idx is None:\n        raise ValueError(\"mask_block_cnt and mask_block_idx must be provided for block sparsity.\")\n\n    full_cnt, full_idx = _check_and_expand_block(\n        \"full\",\n        tensors.full_block_cnt,\n        tensors.full_block_idx,\n        expected_count_shape,\n        expected_index_shape,\n        context,\n        hint,\n    )\n    if full_cnt is not None and mask_cnt.device != full_cnt.device:\n        raise ValueError(\"All block sparse tensors must be on the same device\")\n\n    diag_cnt, diag_idx = _check_and_expand_block(\n        \"diag\",\n        tensors.diag_block_cnt,\n        tensors.diag_block_idx,\n        expected_count_shape,\n        expected_index_shape,\n        context,\n        hint,\n    )\n    if diag_cnt is not None and mask_cnt.device != diag_cnt.device:\n        raise ValueError(\"All block sparse tensors must be on the same device\")\n\n    dq_write_order = _check_and_expand_metadata_tensor(\n        \"dq_write_order\",\n        tensors.dq_write_order,\n        tuple(mask_idx.shape),\n        context,","sourceCodeStart":405,"sourceCodeEnd":441,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py#L405-L441","documentation":"All block-sparse tensors in one config must live on the same CUDA device. Here the optional full-block count tensor's device differs from mask_block_cnt's device, which would cause cross-device copies or kernel faults later.","triggerScenarios":"Loading full_block_cnt from disk (defaults to CPU or cuda:0) while the mask tensors live on cuda:1; mixing tensors created in different device contexts in multi-GPU training.","commonSituations":"Multi-GPU models with device_map sharding; masks precomputed and cached on CPU then partially moved to GPU; tensor created inside a torch.cuda.device(idx) context differing from the rest.","solutions":["Move all block-sparse tensors to the same device: tensors.to(device) or per-tensor .to(mask_block_cnt.device)","Standardize on creating/loading all masks inside the module's device context","Add a device consistency assert in your data pipeline"],"exampleFix":"# before\nfull_cnt = torch.load('full_cnt.pt')          # cpu / cuda:0\nmask_cnt = make_mask(device='cuda:1')\n\n# after\ndevice = mask_cnt.device\nfull_cnt = torch.load('full_cnt.pt', map_location=device)\n# or: tensors = tensors.to(device) before normalize_*","handlingStrategy":"validation","validationCode":"dev = tensors.mask_block_cnt.device\nfor t in (tensors.full_block_cnt, tensors.diag_block_cnt):\n    if t is not None:\n        assert t.device == dev, (t.device, dev)","typeGuard":"def all_same_device(t: BlockSparseTensorsTorch) -> bool:\n    dev = t.mask_block_cnt.device\n    return all(x is None or x.device == dev for x in\n               (t.full_block_cnt, t.full_block_idx, t.diag_block_cnt, t.diag_block_idx))","tryCatchPattern":null,"preventionTips":["torch.load(..., map_location=current_device) for cached masks","Move all mask tensors with one .to(device) call per device switch"],"tags":["block-sparsity","multi-gpu","device-mismatch"],"backgroundTag":"cuda-device-mismatch","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}