huggingface/candle · error

pixel_values require image_grid_thw

Error message

pixel_values require image_grid_thw

What it means

Thrown by qwen3_vl's forward when pixel_values (image tensors) are supplied but image_grid_thw is None. The vision encoder needs the (temporal, height, width) grid per image to reshape the flattened pixel values and compute positional information; providing pixel values without the grid is an inconsistent input.

Source

Thrown at candle-transformers/src/models/qwen3_vl/mod.rs:93

                self.text.dtype,
                input_ids.device(),
            )?)
        } else {
            None
        };

        let mut input_embeds = self.text.embed_tokens(input_ids)?;
        let (batch_size, seq_len, hidden_dim) = input_embeds.dims3()?;
        let device = input_embeds.device().clone();

        let mut image_mask_opt: Option<Tensor> = None;
        let mut video_mask_opt: Option<Tensor> = None;
        let mut deepstack_image_opt: Option<Vec<Tensor>> = None;
        let mut deepstack_video_opt: Option<Vec<Tensor>> = None;

        if let Some(pixel_values) = &pixel_values {
            let Some(image_grid_thw_ref) = image_grid_thw.as_ref() else {
                candle::bail!("pixel_values require image_grid_thw");
            };
            let mut pixel_values = pixel_values.clone();
            let dims = pixel_values.dims();
            if dims.len() == 3 {
                pixel_values = pixel_values.reshape((dims[0] * dims[1], dims[2]))?;
            }
            let (image_embeds, deepstack_image_embeds) =
                self.vision.forward(&pixel_values, image_grid_thw_ref)?;
            let image_embeds = image_embeds.to_device(&device)?.to_dtype(self.text.dtype)?;
            let mut deepstack_image_embeds = deepstack_image_embeds
                .into_iter()
                .map(|t| t.to_device(&device)?.to_dtype(self.text.dtype))
                .collect::<Result<Vec<_>>>()?;

            let mut offset = 0usize;
            let mut image_mask =
                Tensor::zeros((batch_size, seq_len), DType::F32, input_ids.device())?;
            let total_expected: usize = continuous_img_pad

View on GitHub (pinned to d5fee525bf)

Solutions

1. Pass the image_grid_thw tensor produced alongside pixel_values by the Qwen3-VL processor
2. Check that your preprocessing step actually returns both pixel_values and image_grid_thw and both are forwarded
3. If sending only text, pass pixel_values as None instead of a tensor with no grid

Example fix

// before
model.forward(&input_ids, Some(&pixel_values), None, None, None)?;
// after
model.forward(&input_ids, Some(&pixel_values), Some(&image_grid_thw), None, None)?;

if pixel_values.is_some() && image_grid_thw.is_none() {
    anyhow::bail!("pixel_values provided without image_grid_thw");
}

fn valid_image_inputs(pixel_values: Option<&Tensor>, grid: Option<&Tensor>) -> bool {
    match (pixel_values, grid) {
        (Some(_), Some(_)) | (None, _) => true,
        (Some(_), None) => false,
    }
}

match model.forward(&input_ids, pixel_values.as_ref(), image_grid_thw.as_ref(), None, None) {
    Ok(l) => l,
    Err(e) if e.to_string().contains("pixel_values require image_grid_thw") => {
        anyhow::bail!("preprocessing dropped image_grid_thw; fix the processor call")
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling Model::forward with Some(pixel_values) and image_grid_thw: None, e.g. when the caller preprocesses images but drops or forgets to pass the grid tensor returned by the processor.

Common situations: Hand-rolling the image preprocessing pipeline and forgetting the grid_thw output; using a processor version that returns the grid separately from pixel values; wiring up multimodal inputs for the first time.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Image embedding length {} does not match placeholder tokens
• pixel_values_videos require video_grid_thw
• Video embedding length {} does not match placeholder tokens
• {} is a dummy type and cannot be constructed
• {} is a dummy type and cannot be converted

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/3d023a9cf8abc893. Report an issue: GitHub.