Not enough audio embeddings: need {}, got {}. Input sequence

Error message

Not enough audio embeddings: need {}, got {}. Input sequence should have {} audio tokens.

What it means

Voxtral's replace_audio_tokens slices num_audio_tokens audio embeddings out of the available audio_embeds. When the model produced fewer embeddings (total_audio_embeds) than the input sequence declares (num_audio_tokens), it bails with this message including both counts. It prevents reading out of bounds when the audio-encoder output and token count disagree.

Source

Thrown at candle-transformers/src/models/voxtral/model.rs:182

    let (batch_size, seq_len, hidden_size) = inputs_embeds.dims3()?;
    let num_audio_tokens = audio_positions.len();

    // HF-style: audio_embeds shape is (total_audio_seq_len, hidden_size)
    let audio_embeds_dims = audio_embeds.dims2()?;
    let total_audio_embeds = audio_embeds_dims.0;

    // HF-style: Use audio embeddings one-to-one with audio tokens
    // We should now have the right number of audio tokens in the input sequence
    let audio_embeds = if total_audio_embeds >= num_audio_tokens {
        // Take the first num_audio_tokens embeddings to match the audio tokens
        if num_audio_tokens == total_audio_embeds {
            audio_embeds.clone()
        } else {
            audio_embeds.i(0..num_audio_tokens)?
        }
    } else {
        candle::bail!(
            "Not enough audio embeddings: need {}, got {}. Input sequence should have {} audio tokens.",
            num_audio_tokens,
            total_audio_embeds,
            total_audio_embeds
        );
    };

    // Create result tensor starting with text embeddings
    let mut result = inputs_embeds.clone();

    // Replace audio tokens with audio embeddings
    // Since we don't have scatter operations, we'll do this manually
    for (idx, &(batch_idx, seq_idx)) in audio_positions.iter().enumerate() {
        if batch_idx >= batch_size || seq_idx >= seq_len {
            candle::bail!(
                "Invalid audio position: ({}, {}) for tensor shape ({}, {}, {})",
                batch_idx,
                seq_idx,

View on GitHub (pinned to d5fee525bf)

Solutions

Verify the audio input length/config so the encoder produces at least num_audio_tokens embeddings
Ensure the token count derived from the processor matches the audio feature frame count (recompute positions)
Update/downgrade candle so the Voxtral audio encoder config matches your checkpoint

Defensive patterns

Strategy: validation

Validate before calling

if num_audio_tokens > total_audio_embeds {
    return Err(anyhow::anyhow!("need {} audio embeds, encoder produced {}", num_audio_tokens, total_audio_embeds));
}

Try / catch

match model.forward(&inputs) {
    Err(e) if e.to_string().contains("Not enough audio embeddings") => {
        anyhow::bail!("recompute audio token count to match encoder output")
    }
    r => r?,
}

Prevention

Derive num_audio_tokens from the audio feature length the encoder will produce
Keep audio duration and processor config consistent with checkpoint expectations
Log both counts before forward to catch drift early

When it happens

Trigger: Calling forward (which invokes replace_audio_tokens) with an inputs sequence whose number of audio placeholder tokens exceeds the audio encoder's output length, e.g. mismatched audio feature sizing or a corrupted/miscounted audio position map.

Common situations: Passing a longer audio segment than the encoder produced embeddings for; mismatch between the audio tokenizer config and the encoder (pooling/window settings); batching audio of unexpected duration so feature counts shrink.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/a32a69fa5aed2432. Report an issue: GitHub.