{"record":{"id":"5d4683a9bd3f18d3","repo":"huggingface/candle","slug":"empty-cache-despite-pos-0","errorCode":null,"errorMessage":"empty cache despite pos > 0","messagePattern":"empty cache despite pos > 0","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/quantized_recurrent_gemma.rs","lineNumber":153,"sourceCode":"            let x_len = x_branch.dim(D::Minus1)?;\n            let pad = self.conv1d_width as i64 - x_len as i64 - 1;\n            let padded = match pad.cmp(&0) {\n                std::cmp::Ordering::Equal => x_branch.clone(),\n                std::cmp::Ordering::Less => {\n                    let rev_pad = (-pad) as usize;\n                    x_branch.narrow(D::Minus1, rev_pad, x_len - rev_pad)?\n                }\n                std::cmp::Ordering::Greater => {\n                    x_branch.pad_with_zeros(D::Minus1, pad as usize, 0)?\n                }\n            };\n            self.conv1d_state = Some(padded);\n            x_branch\n                .apply(&self.conv_1d)?\n                .narrow(D::Minus1, 0, seq_len)?\n        } else {\n            let conv_state = match self.conv1d_state.as_ref() {\n                None => candle::bail!(\"empty cache despite pos > 0\"),\n                Some(s) => Tensor::cat(&[s, &x_branch], D::Minus1)?,\n            };\n            let w = self.conv_1d.weight().i((.., 0, ..))?;\n            let x_branch = conv_state.broadcast_mul(&w)?.sum(D::Minus1)?;\n            let x_branch = match self.conv_1d.bias() {\n                None => x_branch,\n                Some(b) => x_branch.broadcast_add(b)?,\n            };\n            let x_branch = x_branch.unsqueeze(D::Minus1)?;\n            self.conv1d_state = Some(conv_state.i((.., .., 1..))?);\n            x_branch\n        };\n        let x_branch = x_branch.transpose(1, 2)?;\n        let x_branch = self.rg_lru.forward(&x_branch, pos)?;\n        (x_branch * y_branch)?.apply(&self.linear_out)\n    }\n}\n","sourceCodeStart":135,"sourceCodeEnd":171,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/quantized_recurrent_gemma.rs#L135-L171","documentation":"Thrown by quantized_recurrent_gemma's forward when the sequence position is past 0 (pos > 0) and incremental decoding is expected, but the internal conv1d_state cache is None. The streaming conv path needs the cached previous inputs to build the full convolution window; without it the model cannot compute the conv output.","triggerScenarios":"Calling forward with pos > 0 (continuing a sequence) on a model whose conv1d_state was never initialized, was consumed/reset, or whose earlier forward ran a different branch that never populated the cache.","commonSituations":"Feeding prompts position-by-position without first running a forward that seeds the cache; calling forward on a reused model after a manual cache reset; mixing caching and non-caching forward paths across steps.","solutions":["Ensure the first forward call starts at pos 0 so the conv1d cache is populated before subsequent pos > 0 calls","Use a single forward code path consistently (don't alternate between cache-seeding and incremental calls)","Create a fresh Model instance if the cache state was invalidated mid-generation"],"exampleFix":"// before (starts generation at pos > 0 with empty cache)\nlet logits = model.forward(&tokens, 5)?;\n// after (seed cache from pos 0 first)\nlet logits = model.forward(&prompt_tokens, 0)?;\nlet logits = model.forward(&next_token, prompt_tokens.len())?;","handlingStrategy":"validation","validationCode":"if pos > 0 && model_state_conv_cache_is_none() {\n    anyhow::bail!(\"restart generation from pos 0 to seed the conv cache\");\n}","typeGuard":null,"tryCatchPattern":"match model.forward(&tokens, pos) {\n    Ok(l) => l,\n    Err(e) if e.to_string().contains(\"empty cache despite pos > 0\") => {\n        // restart: seed the cache from position 0\n        let _ = model.forward(&full_prompt, 0)?;\n        model.forward(&tokens, pos)?\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Always run the first forward at pos 0 with the full prompt","Don't mix cache-seeding and incremental forward paths","Recreate the model instead of continuing after cache resets"],"tags":["rust","candle","kv-cache","recurrent-gemma","state-management"],"backgroundTag":"empty-cache-state","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}