Image embedding length {} does not match placeholder tokens

Error message

Image embedding length {} does not match placeholder tokens {}

What it means

Thrown by qwen3_vl's forward when the number of image embedding rows produced by the vision tower does not equal the number of image placeholder token positions (continuous_img_pad span lengths) found in input_ids. Each placeholder token must be replaced by exactly one image embedding; a mismatch would corrupt the merged sequence.

Source

Thrown at candle-transformers/src/models/qwen3_vl/mod.rs:116

                pixel_values = pixel_values.reshape((dims[0] * dims[1], dims[2]))?;
            }
            let (image_embeds, deepstack_image_embeds) =
                self.vision.forward(&pixel_values, image_grid_thw_ref)?;
            let image_embeds = image_embeds.to_device(&device)?.to_dtype(self.text.dtype)?;
            let mut deepstack_image_embeds = deepstack_image_embeds
                .into_iter()
                .map(|t| t.to_device(&device)?.to_dtype(self.text.dtype))
                .collect::<Result<Vec<_>>>()?;

            let mut offset = 0usize;
            let mut image_mask =
                Tensor::zeros((batch_size, seq_len), DType::F32, input_ids.device())?;
            let total_expected: usize = continuous_img_pad
                .iter()
                .flat_map(|spans| spans.iter().map(|(s, e)| e - s))
                .sum();
            if image_embeds.dim(0)? != total_expected {
                candle::bail!(
                    "Image embedding length {} does not match placeholder tokens {}",
                    image_embeds.dim(0)?,
                    total_expected
                );
            }

            for (batch, spans) in continuous_img_pad.iter().enumerate() {
                for &(start, end) in spans {
                    let len = end - start;
                    let chunk = image_embeds.narrow(0, offset, len)?;
                    offset += len;
                    input_embeds = input_embeds.slice_assign(
                        &[batch..batch + 1, start..end, 0..hidden_dim],
                        &chunk.unsqueeze(0)?,
                    )?;
                    let ones = Tensor::ones((1, len), DType::F32, input_ids.device())?;
                    image_mask = image_mask.slice_assign(&[batch..batch + 1, start..end], &ones)?;
                }

View on GitHub (pinned to d5fee525bf)

Solutions

Regenerate pixel_values and image_grid_thw with the same processor call that produced input_ids so patch count and placeholder count match
Verify grid_thw values divide evenly into patch tokens matching the number of <|image_pad|> tokens in input_ids
Ensure pixel_values and input_ids belong to the same example/batch

Example fix

// before (stale placeholders from an old processor run)
let input_ids = old_tokenizer_output; // 200 image pads
let (pixel_values, grid_thw) = new_processor(image)?; // 400 patches
model.forward(&input_ids, Some(&pixel_values), Some(&grid_thw), None, None)?;
// after
let inputs = processor(image, text)?; // consistent pair
model.forward(&inputs.input_ids, Some(&inputs.pixel_values), Some(&inputs.image_grid_thw), None, None)?;

Defensive patterns

Strategy: validation

Validate before calling

let total_pads: usize = image_span_spans.iter().flat_map(|s| s.iter().map(|(a,b)| b-a)).sum();
if image_embeds.dim(0)? != total_pads {
    anyhow::bail!("re-run the processor so patch count matches image placeholders");
}

Try / catch

match model.forward(&input_ids, Some(&pixel_values), Some(&grid_thw), None, None) {
    Ok(l) => l,
    Err(e) if e.to_string().contains("does not match placeholder tokens") => {
        anyhow::bail!("pixel_values and input_ids are out of sync; regenerate both with one processor call")
    }
    Err(e) => return Err(e.into()),
}

Prevention

Produce input_ids, pixel_values and grid_thw in a single processor invocation
Never reuse tokenized input_ids after changing image preprocessing
Count <|image_pad|> tokens and compare to grid_thw-derived patch count before forward

When it happens

Trigger: Calling forward where vision_encoder(image pixel values) yields a different row count than sum of (end-start) over the continuous image placeholder spans — e.g. wrong grid_thw, wrong resize/crop settings, or stale placeholder spans in input_ids.

Common situations: Mismatched preprocessing (image resized differently than the grid implies); tokenization that emits a different number of image placeholders than patches; mixing pixel_values from one image with input_ids from another; batch size mismatch between modalities.

Understand the failure class

Background: Tensor shape mismatch errors ("must have shape", "expected shape ... got ..."): when tensor dimensions disagree with what an op or layer was told to expect — this error's family across 6 libraries.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/accc8076ad45f558. Report an issue: GitHub.