{"record":{"id":"00ba9ea25f6f3e2e","repo":"huggingface/candle","slug":"sliding-window-is-not-supported","errorCode":null,"errorMessage":"sliding window is not supported","messagePattern":"sliding window is not supported","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/qwen3.rs","lineNumber":127,"sourceCode":"    // hyper params\n    num_heads: usize,\n    num_kv_heads: usize,\n    num_kv_groups: usize,\n    head_dim: usize,\n    hidden_size: usize,\n    // utils\n    rotary_emb: Arc<Qwen3RotaryEmbedding>,\n    kv_cache: ConcatKvCache,\n}\n\nimpl Qwen3Attention {\n    pub(crate) fn new(\n        cfg: &Config,\n        rotary_emb: Arc<Qwen3RotaryEmbedding>,\n        vb: VarBuilder,\n    ) -> Result<Self> {\n        if cfg.use_sliding_window {\n            candle::bail!(\"sliding window is not supported\")\n        }\n\n        let head_dim = cfg.head_dim;\n        let num_heads = cfg.num_attention_heads;\n        let num_kv_heads = cfg.num_key_value_heads;\n        let num_kv_groups = num_heads / num_kv_heads;\n\n        let q_proj = linear_b(\n            cfg.hidden_size,\n            num_heads * head_dim,\n            cfg.attention_bias,\n            vb.pp(\"q_proj\"),\n        )?;\n        let k_proj = linear_b(\n            cfg.hidden_size,\n            num_kv_heads * head_dim,\n            cfg.attention_bias,\n            vb.pp(\"k_proj\"),","sourceCodeStart":109,"sourceCodeEnd":145,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/qwen3.rs#L109-L145","documentation":"Thrown by Qwen3Attention::new when the model config has use_sliding_window set to true. Candle's Qwen3 implementation does not implement sliding-window attention, so it refuses to construct the attention layer rather than silently producing wrong results.","triggerScenarios":"Instantiating qwen3::Model (or the attention layer directly via new) with a Config where use_sliding_window: true, typically loaded from a Qwen3 config.json of a model variant that uses interleaved sliding-window attention.","commonSituations":"Loading a Qwen3 long-context variant whose config enables sliding window; hand-editing or copying a config from Qwen2 with sliding window enabled; deserializing a config.json from HuggingFace that includes sliding_window parameters.","solutions":["Use a Qwen3 model/config variant that does not enable sliding window (use_sliding_window: false)","Find or implement sliding-window support in the qwen3 model code, or use a different candle model implementation that supports it","Edit the loaded Config to set use_sliding_window = false only if you understand the accuracy implications for long contexts"],"exampleFix":"// before\nlet mut cfg: Config = serde_json::from_reader(&mut f)?;\ncfg.use_sliding_window = true;\nlet model = qwen3::Model::new(&cfg, vb)?;\n// after\nlet cfg: Config = serde_json::from_reader(&mut f)?;\ndebug_assert!(!cfg.use_sliding_window, \"sliding window unsupported\");\nlet model = qwen3::Model::new(&cfg, vb)?;","handlingStrategy":"validation","validationCode":"if cfg.use_sliding_window {\n    anyhow::bail!(\"this Qwen3 config needs sliding-window attention, unsupported here\");\n}","typeGuard":null,"tryCatchPattern":"match qwen3::Model::new(&cfg, vb) {\n    Ok(m) => m,\n    Err(e) if e.to_string().contains(\"sliding window\") => {\n        anyhow::bail!(\"pick a non sliding-window Qwen3 variant or patch the model impl\")\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Check cfg.use_sliding_window right after deserializing the config","Choose Qwen3 checkpoints whose configs disable sliding window","Track candle upstream for sliding-window support before adopting those variants"],"tags":["rust","candle","qwen3","unsupported-feature","config"],"backgroundTag":"unsupported-feature-for-model","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}