{"record":{"id":"22e030cfdc27b967","repo":"huggingface/candle","slug":"video-embedding-length-does-not-match-placehold","errorCode":null,"errorMessage":"Video embedding length {} does not match placeholder tokens {}","messagePattern":"Video embedding length (.+?) does not match placeholder tokens (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/qwen3_vl/mod.rs","lineNumber":165,"sourceCode":"                pixel_values = pixel_values.reshape((dims[0] * dims[1], dims[2]))?;\n            }\n            let (video_embeds, deepstack_video_embeds) =\n                self.vision.forward(&pixel_values, video_grid_thw_ref)?;\n            let video_embeds = video_embeds.to_device(&device)?.to_dtype(self.text.dtype)?;\n            let mut deepstack_video_embeds = deepstack_video_embeds\n                .into_iter()\n                .map(|t| t.to_device(&device)?.to_dtype(self.text.dtype))\n                .collect::<Result<Vec<_>>>()?;\n\n            let mut offset = 0usize;\n            let mut video_mask =\n                Tensor::zeros((batch_size, seq_len), DType::F32, input_ids.device())?;\n            let total_expected: usize = continuous_vid_pad\n                .iter()\n                .flat_map(|spans| spans.iter().map(|(s, e)| e - s))\n                .sum();\n            if video_embeds.dim(0)? != total_expected {\n                candle::bail!(\n                    \"Video embedding length {} does not match placeholder tokens {}\",\n                    video_embeds.dim(0)?,\n                    total_expected\n                );\n            }\n\n            for (batch, spans) in continuous_vid_pad.iter().enumerate() {\n                for &(start, end) in spans {\n                    let len = end - start;\n                    let chunk = video_embeds.narrow(0, offset, len)?;\n                    offset += len;\n                    input_embeds = input_embeds.slice_assign(\n                        &[batch..batch + 1, start..end, 0..hidden_dim],\n                        &chunk.unsqueeze(0)?,\n                    )?;\n                    let ones = Tensor::ones((1, len), DType::F32, input_ids.device())?;\n                    video_mask = video_mask.slice_assign(&[batch..batch + 1, start..end], &ones)?;\n                }","sourceCodeStart":147,"sourceCodeEnd":183,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/qwen3_vl/mod.rs#L147-L183","documentation":"Thrown by qwen3_vl's forward when the video embedding row count from the vision tower differs from the total number of video placeholder token positions (continuous_vid_pad span lengths) in input_ids. Every video placeholder must map to exactly one video embedding row.","triggerScenarios":"Calling forward where vision_encoder output rows != sum of (end-start) across continuous video placeholder spans — wrong video_grid_thw, temporal/patch preprocessing mismatch, or input_ids containing a stale number of <|video_pad|> tokens.","commonSituations":"Frame sampling count differing between tokenization time and pixel-value generation; grid_thw temporal dim not matching sampled frames; mixing videos between examples in a batch; processor version drift.","solutions":["Generate input_ids, pixel_values_videos and video_grid_thw in one processor call so frame/patch counts and placeholder counts agree","Verify video_grid_thw's temporal dimension matches the number of sampled frames encoded in the placeholders","Ensure pixel_values_videos and input_ids come from the same example/batch"],"exampleFix":"// before (frames re-sampled after tokenization)\nlet input_ids = tokenize(text_with_16_frames);\nlet (video_pixels, video_grid_thw) = encode_video(video, /*frames=*/32)?;\nmodel.forward(&input_ids, None, None, Some(&video_pixels), Some(&video_grid_thw))?;\n// after\nlet inputs = processor(text, video)?; // consistent placeholder & patch counts\nmodel.forward(&inputs.input_ids, None, None, Some(&inputs.pixel_values_videos), Some(&inputs.video_grid_thw))?;","handlingStrategy":"validation","validationCode":"let total_vid_pads: usize = vid_spans.iter().flat_map(|s| s.iter().map(|(a,b)| b-a)).sum();\nif video_embeds.dim(0)? != total_vid_pads {\n    anyhow::bail!(\"video patch count {} != placeholder count {}\", video_embeds.dim(0)?, total_vid_pads);\n}","typeGuard":null,"tryCatchPattern":"match model.forward(&input_ids, None, None, Some(&video_pixels), Some(&video_grid_thw)) {\n    Ok(l) => l,\n    Err(e) if e.to_string().contains(\"does not match placeholder tokens\") => {\n        anyhow::bail!(\"video frames/grid out of sync with tokenization; re-run processor\")\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Fix frame sampling before tokenization; never re-sample after","Verify video_grid_thw temporal dim equals sampled frame count","Group all video inputs with their tokenization in one processor pass"],"tags":["rust","candle","qwen3-vl","video","shape-mismatch"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}