huggingface/candle · error · candle::Error

batch size mismatch between inp ({inp_b_sz}) and target ({b_

Error message

batch size mismatch between inp ({inp_b_sz}) and target ({b_sz})

What it means

Inside nll, after confirming the target is 1-D of length b_sz, the input's first dimension must equal that batch size. This bail fires when the number of rows in the (log-probability) input differs from the number of target labels, since gather cannot align per-sample losses.

Source

Thrown at candle-nn/src/loss.rs:22

/// The negative log likelihood loss.
///
/// Arguments
///
/// * [inp]: The input tensor of dimensions `N, C` where `N` is the batch size and `C` the number
///   of categories. This is expected to contain log probabilities.
/// * [target]: The ground truth labels as a tensor of u32 of dimension `N`.
///
/// The resulting tensor is a scalar containing the average value over the batch.
pub fn nll(inp: &Tensor, target: &Tensor) -> Result<Tensor> {
    let b_sz = match target.dims() {
        &[b_sz] => b_sz,
        dims => candle::bail!("the target tensor should have a single dimension ({dims:?})"),
    };
    match inp.dims() {
        &[inp_b_sz, _] => {
            if inp_b_sz != b_sz {
                candle::bail!("batch size mismatch between inp ({inp_b_sz}) and target ({b_sz})")
            }
        }
        dims => candle::bail!("the target tensor should have two dimensions ({dims:?})"),
    }
    inp.gather(&target.unsqueeze(1)?, 1)?
        .sum_all()?
        .affine(-1f64 / b_sz as f64, 0.)
}

/// The cross-entropy loss.
///
/// Arguments
///
/// * [inp]: The input tensor of dimensions `N, C` where `N` is the batch size and `C` the number
///   of categories. This is expected to raw logits.
/// * [target]: The ground truth labels as a tensor of u32 of dimension `N`.
///
/// The resulting tensor is a scalar containing the average value over the batch.

View on GitHub (pinned to d5fee525bf)

Solutions

1. Slice both tensors to the same batch size before the loss: inp = inp.narrow(0, 0, min_b)?; target = target.narrow(0, 0, min_b)?;
2. Fix the dataloader so inputs and targets use identical batching settings (same batch_size and drop_last).
3. Verify you are not pairing per-epoch logits with per-batch targets (or vice versa).
4. Print inp.dims() and target.dims() at the call site to find which pipeline produced the extra/missing samples.

Example fix

// before
let loss = cross_entropy(&logits, &labels)?; // logits [63, C], labels [64]
// after
let b = logits.dim(0)?;
let loss = cross_entropy(&logits, &labels.narrow(0, 0, b)?)?;

let (bi, bt) = (inp.dim(0)?, target.dim(0)?);
if bi != bt {
    return Err(anyhow!("batch mismatch: inp {bi} vs target {bt}"));
}

let b = inp.dim(0)?.min(target.dim(0)?);
let loss = cross_entropy(
    &inp.narrow(0, 0, b)?,
    &target.narrow(0, 0, b)?
)?;

When it happens

Trigger: Calling nll or cross_entropy where inp.dims()[0] != target.dims()[0] — e.g. logits for a truncated final batch paired with targets from a dropped-last batch, or targets mis-split across micro-batches.

Common situations: DataLoader drop_last differing between inputs and labels pipelines; off-by-one in manual batch slicing; targets accumulated over the epoch while logits are per-batch; combining teacher-forcing sequences of different lengths.

Related errors

• the target tensor should have a single dimension ({dims:?})
• the target tensor should have two dimensions ({dims:?})
• cross_entropy expects an input tensor of rank 2
• input and target must have the same shape, got inp: {:?}, ta
• axis {axis} is too large, tensor rank {rank}

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/484ef125d601d83a. Report an issue: GitHub.