What causes "expected some cross-attn" in huggingface/candle?

Load a BLIP text checkpoint/config that includes cross-attention weights

expected some cross-attn

Error message

expected some cross-attn

What it means

In blip_text's BertEncoderLayer forward, cross-attention is optional; this error is thrown when a layer configured without cross_attention is asked to process encoder_hidden_states, which requires cross-attention. It indicates the BLIP text model was built from a config without cross-attention layers while the forward path needs them.

Source

Thrown at candle-transformers/src/models/blip_text.rs:295

    }

    fn reset_kv_cache(&mut self) {
        self.attention.reset_kv_cache();
        if let Some(ca) = &mut self.cross_attention {
            ca.reset_kv_cache()
        }
    }

    fn forward(
        &mut self,
        xs: &Tensor,
        encoder_hidden_states: &Tensor,
        attention_mask: &Tensor,
    ) -> Result<Tensor> {
        let attention_output = self.attention.forward(xs, None, Some(attention_mask))?;
        let attention_output = match &mut self.cross_attention {
            Some(ca) => ca.forward(&attention_output, Some(encoder_hidden_states), None)?,
            None => candle::bail!("expected some cross-attn"),
        };
        let intermediate_output = self.intermediate.forward(&attention_output)?;
        self.output.forward(&intermediate_output, &attention_output)
    }
}

#[derive(Debug, Clone)]
struct TextEncoder {
    layers: Vec<TextLayer>,
}

impl TextEncoder {
    fn new(cfg: &Config, vb: VarBuilder) -> Result<Self> {
        let vb = vb.pp("layer");
        let mut layers = Vec::with_capacity(cfg.num_hidden_layers);
        for i in 0..cfg.num_hidden_layers {
            let layer = TextLayer::new(cfg, vb.pp(i))?;
            layers.push(layer)

View on GitHub (pinned to d5fee525bf)

Solutions

Load a BLIP text checkpoint/config that includes cross-attention weights
Ensure the model is constructed with cross_attention: Some(...) for the matching task
Use the correct task API: retrieval-only models don't take encoder_hidden_states
Verify config.json / weight names contain cross-attention layers

Example fix

// before
let text_model = BertTextModel::load(vb, config)?; // config without x-attention
let out = layer.forward(&xs, &encoder_hidden_states, &mask)?;
// after
let mut config = config.clone();
config.add_cross_attention = true; // then reload weights
let text_model = BertTextModel::load(vb, &config)?;

Defensive patterns

Strategy: validation

Validate before calling

assert!(config.add_cross_attention, "task requires cross-attention; config lacks it");

Type guard

fn has_cross_attention(layer: &BertEncoderLayer) -> bool {
    layer.cross_attention.is_some()
}

Try / catch

match layer.forward(&xs, &encoder_hidden_states, &mask) {
    Ok(v) => v,
    Err(e) if e.to_string().contains("expected some cross-attn") => {
        return Err(anyhow!("reload model with cross-attention enabled"));
    }
    Err(e) => return Err(e),
}

Prevention

Match the checkpoint variant to the task: matching tasks need cross-attention
Verify config.json sets add_cross_attention for image-text matching models
Check checkpoint weight names include cross-attention layers before running
Use retrieval-only APIs for retrieval-only checkpoints

When it happens

Trigger: Calling forward with encoder_hidden_states on a BertEncoderLayer whose cross_attention field is None, typically when the loaded config/checkpoint lacks cross-attention weights.

Common situations: Loading an image-text retrieval checkpoint (no cross-attention) and then running image-text matching (which needs it), or mismatched config between the BLIP variant and the task.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/051a81358ff8c089. Report an issue: GitHub.