huggingface/candle · error

block_table must be u32 or i32, got {dtype:?}

Error message

block_table must be u32 or i32, got {dtype:?}

What it means

In the paged-attention/varlen path, block_table entries are read as 32-bit integers via raw pointers; only u32 and i32 storages are accepted. Any other dtype for the block table is rejected with this error.

Source

Thrown at candle-flash-attn/src/lib.rs:750

            let (seqlens_q_ptr, _guard) = seqlens_q.device_ptr(&stream);
            let (seqlens_k_ptr, _guard) = seqlens_k.device_ptr(&stream);
            let (block_table_ptr, block_table_batch_stride) =
                if let Some((block_table, offset, stride)) = block_table.as_ref() {
                    match (&**block_table, self.block_table.as_ref().unwrap().dtype()) {
                        (candle::Storage::Cuda(block_table), DType::U32) => {
                            let block_table = block_table.as_cuda_slice::<u32>()?;
                            let block_table = block_table.slice(*offset..);
                            let (ptr, _guard) = block_table.device_ptr(&stream);
                            (ptr as *const i32, *stride as u32)
                        }
                        (candle::Storage::Cuda(block_table), DType::I32) => {
                            let block_table = block_table.as_cuda_slice::<i32>()?;
                            let block_table = block_table.slice(*offset..);
                            let (ptr, _guard) = block_table.device_ptr(&stream);
                            (ptr as *const i32, *stride as u32)
                        }
                        (_, dtype) => {
                            candle::bail!("block_table must be u32 or i32, got {dtype:?}")
                        }
                    }
                } else {
                    (std::ptr::null(), 0)
                };
            let (mm_prefix_ranges_ptr, mm_prefix_range_batch_stride, max_mm_prefix_ranges) =
                if let Some((storage, offset, stride, max_ranges)) = mm_prefix_ranges.as_ref() {
                    match &**storage {
                        candle::Storage::Cuda(mm_prefix_ranges) => {
                            let mm_prefix_ranges = mm_prefix_ranges.as_cuda_slice::<i32>()?;
                            let mm_prefix_ranges = mm_prefix_ranges.slice(*offset..);
                            let (ptr, _guard) = mm_prefix_ranges.device_ptr(&stream);
                            (ptr as *const i32, *stride as u32, *max_ranges as i32)
                        }
                        _ => unreachable!("mm_prefix_ranges must be a cuda tensor"),
                    }
                } else {
                    (std::ptr::null(), 0, 0)

View on GitHub (pinned to d5fee525bf)

Solutions

1. Convert the block table with .to_dtype(DType::I32)? (or U32) before the call
2. Build the table as i32 from the start (from_vec of i32 values)
3. Check the dtype at construction with a debug_assert

Example fix

// before
let block_table = Tensor::from_vec(pages_i64, (b, max_blocks), &dev)?; // i64
// after
let block_table = Tensor::from_vec(pages_i32, (b, max_blocks), &dev)?; // i32

if !matches!(block_table.dtype(), candle::DType::I32 | candle::DType::U32) {
    let block_table = block_table.to_dtype(candle::DType::I32)?;
}

fn is_i32_or_u32(t: &Tensor) -> bool {
    matches!(t.dtype(), candle::DType::I32 | candle::DType::U32)
}

match result {
    Err(e) if e.to_string().contains("block_table must be u32 or i32") => {
        let bt = block_table.to_dtype(candle::DType::I32)?;
        // retry
    }
    other => other?,
}

When it happens

Trigger: Passing a block_table tensor of dtype f32, i64, u16, etc. to flash-attn with a block table (e.g. page table built as i64 like PyTorch defaults).

Common situations: Porting paged-attention code from PyTorch where page tables are int64; building block tables with candle's default i64 from_from_vec of i64 values.

Related errors

• only f32 can be quantized
• flash-attn is only supported for f16/bf16 ({dt:?})
• block_table must be a cuda tensor
• block_table last dimension must be contiguous
• flash-attn-varlen paged expects k/v tensors of rank 4 (k: {k

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/9c98ac1c9a267e7b. Report an issue: GitHub.