What causes "conv-block is not supported" in huggingface/candle?

Set use_conv_block to false in the transformer Config

conv-block is not supported

Error message

conv-block is not supported

What it means

StreamingTransformerLayer::new rejects configs that enable the convolutional block (cfg.use_conv_block). The candle mimi transformer only implements LayerNorm/MLP layers without the extra conv branch present in some reference architectures.

Source

Thrown at candle-transformers/src/models/mimi/transformer.rs:481

}

#[derive(Debug, Clone)]
pub struct StreamingTransformerLayer {
    self_attn: StreamingMultiheadAttention,
    mlp: Mlp,
    norm1: Norm,
    norm2: Norm,
    layer_scale_1: Option<LayerScale>,
    layer_scale_2: Option<LayerScale>,
    cross_attn: Option<(candle_nn::LayerNorm, StreamingMultiheadCrossAttention)>,
    norm_first: bool,
    span: tracing::Span,
}

impl StreamingTransformerLayer {
    pub fn new(rope: &Option<Arc<RotaryEmbedding>>, cfg: &Config, vb: VarBuilder) -> Result<Self> {
        if cfg.use_conv_block {
            candle::bail!("conv-block is not supported")
        }
        let d_model = cfg.d_model;
        let mlp = Mlp::new(cfg, vb.clone())?;
        let (norm1, norm2) = match cfg.norm {
            super::NormType::LayerNorm => {
                let norm1 = candle_nn::layer_norm(d_model, 1e-5, vb.pp("input_layernorm"))?;
                let norm2 =
                    candle_nn::layer_norm(d_model, 1e-5, vb.pp("post_attention_layernorm"))?;
                (Norm::LayerNorm(norm1), Norm::LayerNorm(norm2))
            }
            super::NormType::RmsNorm => {
                let norm1 = RmsNorm::new(d_model, 1e-8, vb.pp("input_rmsnorm"))?;
                let norm2 = RmsNorm::new(d_model, 1e-8, vb.pp("post_attention_rmsnorm"))?;
                (Norm::RmsNorm(norm1), Norm::RmsNorm(norm2))
            }
        };
        let layer_scale_1 = match cfg.layer_scale {
            None => None,

View on GitHub (pinned to d5fee525bf)

Solutions

Set use_conv_block to false in the transformer Config
Use a checkpoint/config without conv blocks
Add conv-block support to candle's mimi transformer if required

Example fix

// before
let cfg = Config { use_conv_block: true, ..base };
StreamingTransformerLayer::new(&rope, &cfg, vb)?; // bails
// after
let cfg = Config { use_conv_block: false, ..base };
StreamingTransformerLayer::new(&rope, &cfg, vb)?;

Defensive patterns

Strategy: validation

Validate before calling

if cfg.use_conv_block {
    return Err(anyhow::anyhow!("conv-block layers are unsupported in candle mimi"));
}
let layer = StreamingTransformerLayer::new(&rope, &cfg, vb)?;

Type guard

fn supports_layer(cfg: &Config) -> bool { !cfg.use_conv_block }

Try / catch

let layer = StreamingTransformerLayer::new(&rope, &cfg, vb)
    .map_err(|e| if e.to_string().contains("conv-block is not supported") {
        anyhow::anyhow!("set use_conv_block = false in the transformer config")
    } else { e.into() })?;

Prevention

Force use_conv_block = false when importing Moshi configs
Validate the whole transformer Config before building layers
Choose checkpoints trained without conv-block layers

When it happens

Trigger: Building StreamingTransformerLayer with a Config where use_conv_block is true, typically imported from a mimi/Moshi config that enables conv blocks.

Common situations: Copying a config from the reference implementation with "use_conv_block": true; a checkpoint trained with conv-block layers.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/776b50b6d1d83229. Report an issue: GitHub.