What causes "quantized embedding requires contiguous ids" in huggingface/candle?

Make the ids tensor contiguous before the call: let ids = ids.contiguous()?;

quantized embedding requires contiguous ids

Error message

quantized embedding requires contiguous ids

What it means

The quantized CUDA embedding lookup reads token ids directly from device memory assuming a dense, contiguous layout. If the ids Layout has strides/non-standard offsets, the kernel cannot index them, so candle bails before launching. Only contiguous u32 id tensors are accepted.

Source

Thrown at candle-core/src/quantized/cuda.rs:820

            inner,
            len: data.len(),
        };
        Ok(())
    }

    pub fn storage_size_in_bytes(&self) -> usize {
        self.data.len
    }

    pub fn embedding(
        &self,
        rows: usize,
        hidden: usize,
        ids: &CudaStorage,
        ids_l: &crate::Layout,
    ) -> Result<CudaStorage> {
        if !ids_l.is_contiguous() {
            crate::bail!("quantized embedding requires contiguous ids")
        }
        if !hidden.is_multiple_of(self.dtype.block_size()) {
            crate::bail!(
                "quantized embedding hidden size {hidden} is not divisible by block size {}",
                self.dtype.block_size()
            )
        }
        let expected_size = rows * hidden * self.dtype.type_size() / self.dtype.block_size();
        if self.storage_size_in_bytes() != expected_size {
            crate::bail!(
                "quantized tensor has {} bytes, expected {expected_size}",
                self.storage_size_in_bytes()
            )
        }
        let ids = ids.as_cuda_slice::<u32>()?;
        let ids = match ids_l.contiguous_offsets() {
            Some((o1, o2)) => ids.slice(o1..o2),
            None => Err(crate::Error::RequiresContiguous {

View on GitHub (pinned to d5fee525bf)

Solutions

Make the ids tensor contiguous before the call: let ids = ids.contiguous()?;
Rebuild the ids tensor freshly (e.g. via Tensor::new / cat) so it has a contiguous layout.
Avoid slicing/striding the ids on device; gather ids on CPU into a new contiguous tensor.
Check ids.layout().is_contiguous() before invoking embedding.

Example fix

// before
let out = qweight.embedding(h, &ids_view)?;
// after
let out = qweight.embedding(h, &ids_view.contiguous()?)?;

Defensive patterns

Strategy: validation

Validate before calling

let ids = ids.contiguous()?; // ensure contiguous layout before quantized embedding
// candle: assert ids is u32 on the same device
let ids = ids.to_dtype(candle_core::DType::U32)?.to_device(&device)?;

Try / catch

match qweight.embedding(h, &ids) {
    Err(e) if e.to_string().contains("requires contiguous ids") => {
        let out = qweight.embedding(h, &ids.contiguous()?)?;
        out
    }
    r => r?,
}

Prevention

Call .contiguous()? on ids after any slicing/narrow
Keep id tensors freshly created per batch, not as views of padded buffers
Avoid strided/transposed id layouts in tokenizers feeding quantized embedding

When it happens

Trigger: Calling QTensor::embedding (or forward on an embedding whose weight is a quantized QTensor) with an ids tensor that is a non-contiguous slice/view — e.g. the result of slicing, striding, or transposing without a contiguous copy.

Common situations: Passing token ids sliced out of a padded batch; building ids via indexing ops that keep a stride layout; reusing an ids layout transformed by broadcasting or narrow.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/ebe7cfb00163ce27. Report an issue: GitHub.