huggingface/candle · error

Invalid audio position: ({}, {}) for tensor shape ({}, {}, {

Error message

Invalid audio position: ({}, {}) for tensor shape ({}, {}, {})

What it means

While replacing audio placeholder tokens with embeddings, each (batch_idx, seq_idx) position is checked against the (batch_size, seq_len) of inputs_embeds. An out-of-range position bails with the offending coordinates and tensor shape (batch, seq, hidden). This guards against indexing a tensor out of bounds due to a bad audio position mapping.

Source

Thrown at candle-transformers/src/models/voxtral/model.rs:197

            audio_embeds.i(0..num_audio_tokens)?
        }
    } else {
        candle::bail!(
            "Not enough audio embeddings: need {}, got {}. Input sequence should have {} audio tokens.",
            num_audio_tokens,
            total_audio_embeds,
            total_audio_embeds
        );
    };

    // Create result tensor starting with text embeddings
    let mut result = inputs_embeds.clone();

    // Replace audio tokens with audio embeddings
    // Since we don't have scatter operations, we'll do this manually
    for (idx, &(batch_idx, seq_idx)) in audio_positions.iter().enumerate() {
        if batch_idx >= batch_size || seq_idx >= seq_len {
            candle::bail!(
                "Invalid audio position: ({}, {}) for tensor shape ({}, {}, {})",
                batch_idx,
                seq_idx,
                batch_size,
                seq_len,
                hidden_size
            );
        }

        // Get the audio embedding for this position
        let audio_embed = audio_embeds.i(idx)?;

        // Create a mask for this specific position
        let mut position_mask = vec![0f32; batch_size * seq_len];
        position_mask[batch_idx * seq_len + seq_idx] = 1.0;
        let position_mask = Tensor::new(position_mask.as_slice(), device)?
            .reshape((batch_size, seq_len, 1))?
            .to_dtype(inputs_embeds.dtype())?;

View on GitHub (pinned to d5fee525bf)

Solutions

  1. Regenerate audio_positions from the exact same inputs_embeds tensor passed to forward
  2. Check padding/truncation in preprocessing so positions and embeddings stay aligned
  3. Add an assertion on max(audio_positions) vs (batch_size, seq_len) before calling forward

Example fix

// before
let positions = positions_from_tokens(&input_ids);
model.forward(&inputs_embeds_truncated, &positions, ...)?;
// after
assert!(positions.iter().all(|&(b, s)| b < batch && s < seq));
model.forward(&inputs_embeds, &positions, ...)?;
Defensive patterns

Strategy: validation

Validate before calling

let (b, s) = inputs_embeds.dims2()?; // or dims3
if let Some(&(bi, si)) = audio_positions.iter().max_by_key(|&&(b, s)| (b, s)) {
    if bi >= b || si >= s { return Err(anyhow::anyhow!("audio position ({bi},{si}) out of shape ({b},{s})")); }
}

Try / catch

match model.forward_with_audio(&embeds, &positions, &audio) {
    Err(e) if e.to_string().contains("Invalid audio position") => {
        anyhow::bail!("audio_positions misaligned with inputs_embeds; regenerate them")
    }
    r => r?,
}

Prevention

When it happens

Trigger: Calling forward with audio_positions containing coordinates beyond the inputs_embeds batch or sequence dimension — e.g. positions computed from a tokenized sequence longer/shorter than the embedding tensor passed in.

Common situations: Building audio_positions from input_ids but passing a truncated/padded inputs_embeds; batching mismatch (positions computed for a different batch item); pre/post processing bug inserting fewer or more tokens than positions reference.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/4a25706ac57abdf3. Report an issue: GitHub.