What causes "unexpected shape for txt {:?}" in huggingface/candle?

Reshape txt to rank 3: (batch, seq_len, 4096) for Flux T5 embeddings — use `.unsqueeze(0)` if the batch dim is missing.

unexpected shape for txt {:?}

Error message

unexpected shape for txt {:?}

What it means

Flux::forward validates that the text-token tensor `txt` has rank 3 (batch, seq_len, hidden) before embedding and running the transformer. This bail fires when txt has any other rank, e.g. a 2D (seq_len, hidden) tensor with no batch dimension or unpooled embeddings passed without reshaping. It is an early input-shape sanity check before position embedding and attention are computed.

Source

Thrown at candle-transformers/src/models/flux/model.rs:593

            final_layer,
        })
    }
}

impl super::WithForward for Flux {
    #[allow(clippy::too_many_arguments)]
    fn forward(
        &self,
        img: &Tensor,
        img_ids: &Tensor,
        txt: &Tensor,
        txt_ids: &Tensor,
        timesteps: &Tensor,
        y: &Tensor,
        guidance: Option<&Tensor>,
    ) -> Result<Tensor> {
        if txt.rank() != 3 {
            candle::bail!("unexpected shape for txt {:?}", txt.shape())
        }
        if img.rank() != 3 {
            candle::bail!("unexpected shape for img {:?}", img.shape())
        }
        let dtype = img.dtype();
        let pe = {
            let ids = Tensor::cat(&[txt_ids, img_ids], 1)?;
            ids.apply(&self.pe_embedder)?
        };
        let mut txt = txt.apply(&self.txt_in)?;
        let mut img = img.apply(&self.img_in)?;
        let vec_ = timestep_embedding(timesteps, 256, dtype)?.apply(&self.time_in)?;
        let vec_ = match (self.guidance_in.as_ref(), guidance) {
            (Some(g_in), Some(guidance)) => {
                (vec_ + timestep_embedding(guidance, 256, dtype)?.apply(g_in))?
            }
            _ => vec_,
        };

View on GitHub (pinned to d5fee525bf)

Solutions

Reshape txt to rank 3: (batch, seq_len, 4096) for Flux T5 embeddings — use `.unsqueeze(0)` if the batch dim is missing.
Check dtype/device-preserving reshape: `let txt = txt.reshape((b, seq, hidden))?;`
Ensure the tokenizer/pipeline produces per-batch embeddings; compare with candle's flux example (examples/flux-main.rs) txt preparation.
Log txt.shape() before calling forward to confirm rank and dims.

Example fix

// before
let txt = t5_embeddings; // rank 2: (seq_len, 4096)
model.forward(&img, &img_ids, &txt, &txt_ids, &t, &y, None)?;

// after
let txt = t5_embeddings.unsqueeze(0)?; // rank 3: (1, seq_len, 4096)
model.forward(&img, &img_ids, &txt, &txt_ids, &t, &y, None)?;

Defensive patterns

Strategy: validation

Validate before calling

// Rust: guard before calling Flux::forward
if txt.rank() != 3 {
    return Err(candle_core::Error::Msg(format!(
        "txt must be (batch, seq, hidden); got shape {:?}", txt.shape()
    )));
}

Type guard

fn is_rank3(t: &candle_core::Tensor) -> bool { t.rank() == 3 }

Try / catch

match flux.forward(&img, &img_ids, &txt, &txt_ids, &ts, &y, guidance) {
    Ok(t) => t,
    Err(e) if e.to_string().contains("unexpected shape for txt") => {
        let txt = txt.unsqueeze(0)?; // recover by adding batch dim once
        flux.forward(&img, &img_ids, &txt, &txt_ids, &ts, &y, guidance)?
    }
    Err(e) => return Err(e),
}

Prevention

Keep text embeddings batched end-to-end; do not squeeze the batch dimension after T5 encoding.
Use candle's flux example pipeline (examples/flux-main.rs) as the reference for tensor prep.
Print shapes of all five input tensors before forward during development.
Wrap shape-sensitive calls in helpers that assert rank/dims and fail with actionable messages.

When it happens

Trigger: Calling Flux forward (via generate or WithForward) passing txt built from T5 token embeddings without a batch dimension, e.g. shape (seq, 4096) instead of (1, seq, 4096); passing pooled CLIP embeddings or squeezed tensors as txt; batching errors that flatten the tensor.

Common situations: Writing custom sampling code around candle's Flux instead of using the included generate function and forgetting .unsqueeze(0); porting code from diffusers where tensor shapes are handled internally; concatenating batches incorrectly so the tensor gets flattened.

Understand the failure class

Background: Tensor shape mismatch errors ("must have shape", "expected shape ... got ..."): when tensor dimensions disagree with what an op or layer was told to expect — this error's family across 6 libraries.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/3d24e16a53bb1394. Report an issue: GitHub.