{"record":{"id":"a32a69fa5aed2432","repo":"huggingface/candle","slug":"not-enough-audio-embeddings-need-got-inpu","errorCode":null,"errorMessage":"Not enough audio embeddings: need {}, got {}. Input sequence should have {} audio tokens.","messagePattern":"Not enough audio embeddings: need (.+?), got (.+?)\\. Input sequence should have (.+?) audio tokens\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/voxtral/model.rs","lineNumber":182,"sourceCode":"\n    let (batch_size, seq_len, hidden_size) = inputs_embeds.dims3()?;\n    let num_audio_tokens = audio_positions.len();\n\n    // HF-style: audio_embeds shape is (total_audio_seq_len, hidden_size)\n    let audio_embeds_dims = audio_embeds.dims2()?;\n    let total_audio_embeds = audio_embeds_dims.0;\n\n    // HF-style: Use audio embeddings one-to-one with audio tokens\n    // We should now have the right number of audio tokens in the input sequence\n    let audio_embeds = if total_audio_embeds >= num_audio_tokens {\n        // Take the first num_audio_tokens embeddings to match the audio tokens\n        if num_audio_tokens == total_audio_embeds {\n            audio_embeds.clone()\n        } else {\n            audio_embeds.i(0..num_audio_tokens)?\n        }\n    } else {\n        candle::bail!(\n            \"Not enough audio embeddings: need {}, got {}. Input sequence should have {} audio tokens.\",\n            num_audio_tokens,\n            total_audio_embeds,\n            total_audio_embeds\n        );\n    };\n\n    // Create result tensor starting with text embeddings\n    let mut result = inputs_embeds.clone();\n\n    // Replace audio tokens with audio embeddings\n    // Since we don't have scatter operations, we'll do this manually\n    for (idx, &(batch_idx, seq_idx)) in audio_positions.iter().enumerate() {\n        if batch_idx >= batch_size || seq_idx >= seq_len {\n            candle::bail!(\n                \"Invalid audio position: ({}, {}) for tensor shape ({}, {}, {})\",\n                batch_idx,\n                seq_idx,","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/voxtral/model.rs#L164-L200","documentation":"Voxtral's replace_audio_tokens slices num_audio_tokens audio embeddings out of the available audio_embeds. When the model produced fewer embeddings (total_audio_embeds) than the input sequence declares (num_audio_tokens), it bails with this message including both counts. It prevents reading out of bounds when the audio-encoder output and token count disagree.","triggerScenarios":"Calling forward (which invokes replace_audio_tokens) with an inputs sequence whose number of audio placeholder tokens exceeds the audio encoder's output length, e.g. mismatched audio feature sizing or a corrupted/miscounted audio position map.","commonSituations":"Passing a longer audio segment than the encoder produced embeddings for; mismatch between the audio tokenizer config and the encoder (pooling/window settings); batching audio of unexpected duration so feature counts shrink.","solutions":["Verify the audio input length/config so the encoder produces at least num_audio_tokens embeddings","Ensure the token count derived from the processor matches the audio feature frame count (recompute positions)","Update/downgrade candle so the Voxtral audio encoder config matches your checkpoint"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if num_audio_tokens > total_audio_embeds {\n    return Err(anyhow::anyhow!(\"need {} audio embeds, encoder produced {}\", num_audio_tokens, total_audio_embeds));\n}","typeGuard":null,"tryCatchPattern":"match model.forward(&inputs) {\n    Err(e) if e.to_string().contains(\"Not enough audio embeddings\") => {\n        anyhow::bail!(\"recompute audio token count to match encoder output\")\n    }\n    r => r?,\n}","preventionTips":["Derive num_audio_tokens from the audio feature length the encoder will produce","Keep audio duration and processor config consistent with checkpoint expectations","Log both counts before forward to catch drift early"],"tags":["rust","candle","voxtral","audio","shape-mismatch"],"backgroundTag":"audio-embedding-count-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}