huggingface/candle · error

pixel_values_videos require video_grid_thw

Error message

pixel_values_videos require video_grid_thw

What it means

Thrown by qwen3_vl's forward when pixel_values_videos (video tensors) are supplied but video_grid_thw is None. Like the image path, the vision encoder needs the per-video (temporal, height, width) grid to reshape and position the video patches.

Source

Thrown at candle-transformers/src/models/qwen3_vl/mod.rs:142

                for &(start, end) in spans {
                    let len = end - start;
                    let chunk = image_embeds.narrow(0, offset, len)?;
                    offset += len;
                    input_embeds = input_embeds.slice_assign(
                        &[batch..batch + 1, start..end, 0..hidden_dim],
                        &chunk.unsqueeze(0)?,
                    )?;
                    let ones = Tensor::ones((1, len), DType::F32, input_ids.device())?;
                    image_mask = image_mask.slice_assign(&[batch..batch + 1, start..end], &ones)?;
                }
            }
            image_mask_opt = Some(image_mask.to_dtype(DType::U8)?);
            deepstack_image_opt = Some(std::mem::take(&mut deepstack_image_embeds));
        }

        if let Some(pixel_values_videos) = &pixel_values_videos {
            let Some(video_grid_thw_ref) = video_grid_thw.as_ref() else {
                candle::bail!("pixel_values_videos require video_grid_thw");
            };
            let mut pixel_values = pixel_values_videos.clone();
            let dims = pixel_values.dims();
            if dims.len() == 3 {
                pixel_values = pixel_values.reshape((dims[0] * dims[1], dims[2]))?;
            }
            let (video_embeds, deepstack_video_embeds) =
                self.vision.forward(&pixel_values, video_grid_thw_ref)?;
            let video_embeds = video_embeds.to_device(&device)?.to_dtype(self.text.dtype)?;
            let mut deepstack_video_embeds = deepstack_video_embeds
                .into_iter()
                .map(|t| t.to_device(&device)?.to_dtype(self.text.dtype))
                .collect::<Result<Vec<_>>>()?;

            let mut offset = 0usize;
            let mut video_mask =
                Tensor::zeros((batch_size, seq_len), DType::F32, input_ids.device())?;
            let total_expected: usize = continuous_vid_pad

View on GitHub (pinned to d5fee525bf)

Solutions

1. Pass the video_grid_thw tensor returned by the video preprocessing/processor step
2. Audit your call to forward to ensure video tensors and video_grid_thw are both Some or both None
3. For text-only or image-only inputs, pass pixel_values_videos: None

Example fix

// before
model.forward(&input_ids, Some(&pixel_values), Some(&image_grid_thw), Some(&video_pixels), None)?;
// after
model.forward(&input_ids, Some(&pixel_values), Some(&image_grid_thw), Some(&video_pixels), Some(&video_grid_thw))?;

if pixel_values_videos.is_some() && video_grid_thw.is_none() {
    anyhow::bail!("pixel_values_videos provided without video_grid_thw");
}

fn valid_video_inputs(video: Option<&Tensor>, grid: Option<&Tensor>) -> bool {
    match (video, grid) {
        (Some(_), Some(_)) | (None, _) => true,
        (Some(_), None) => false,
    }
}

match model.forward(&input_ids, None, None, pixel_values_videos.as_ref(), video_grid_thw.as_ref()) {
    Ok(l) => l,
    Err(e) if e.to_string().contains("pixel_values_videos require video_grid_thw") => {
        anyhow::bail!("video preprocessing must return and forward video_grid_thw")
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling Model::forward with Some(pixel_values_videos) and video_grid_thw: None — typically the video preprocessing output grid tensor was discarded or not threaded through the call.

Common situations: Custom video preprocessing that drops the grid_thw output; assuming the image grid argument covers videos; first-time integration of video inputs with the VL model.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• pixel_values require image_grid_thw
• Video embedding length {} does not match placeholder tokens
• Image embedding length {} does not match placeholder tokens
• {} is a dummy type and cannot be constructed
• {} is a dummy type and cannot be converted

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/6d6d16b4e45398ef. Report an issue: GitHub.