{"record":{"id":"32ba5273411e44b2","repo":"sgl-project/sglang","slug":"rope-cos-sin-cache-is-too-short-for-fused-kv-mater","errorCode":null,"errorMessage":"RoPE cos/sin cache is too short for fused KV materialization: max_position={max_position}, cache_len={int(cos_sin_cache.shape[0])}.","messagePattern":"RoPE cos/sin cache is too short for fused KV materialization: max_position=(.+?), cache_len=(.+?)\\.","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/speculative/fused_kv_materialize.py","lineNumber":353,"sourceCode":"        )\n\n        if self.max_position_hint is not None:\n            self._ensure_rope_cache(self.max_position_hint)\n\n    def _ensure_rope_cache(self, max_position: int) -> torch.Tensor:\n        if max_position + 1 > self._reserved_rope_cache_len:\n            ensure_cos_sin_cache_length = getattr(\n                self.rotary_emb, \"_ensure_cos_sin_cache_length\", None\n            )\n            if callable(ensure_cos_sin_cache_length):\n                ensure_cos_sin_cache_length(max_position)\n                self._reserved_rope_cache_len = int(\n                    self.rotary_emb.cos_sin_cache.shape[0]\n                )\n\n        cos_sin_cache = self.rotary_emb.cos_sin_cache\n        if max_position >= int(cos_sin_cache.shape[0]):\n            raise RuntimeError(\n                \"RoPE cos/sin cache is too short for fused KV materialization: \"\n                f\"max_position={max_position}, cache_len={int(cos_sin_cache.shape[0])}.\"\n            )\n        if cos_sin_cache.device != self.device:\n            cos_sin_cache = cos_sin_cache.to(self.device)\n        return cos_sin_cache\n\n    def _ensure_workspace(self, total_ctx: int, dtype: torch.dtype) -> None:\n        if (\n            self._workspace_capacity >= total_ctx\n            and self._workspace_dtype == dtype\n            and self._proj_workspace is not None\n            and self._k_workspace is not None\n            and self._v_workspace is not None\n        ):\n            return\n\n        new_capacity = max(1, total_ctx)","sourceCodeStart":335,"sourceCodeEnd":371,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/speculative/fused_kv_materialize.py#L335-L371","documentation":"The RoPE cos_sin_cache (indexed by absolute position) must be longer than the maximum position being materialized. The code raises when max_position >= cos_sin_cache.shape[0].","triggerScenarios":"Materializing KV for token positions at or beyond the cache length — e.g. long contexts exceeding the rotary_emb's max_position_embeddings-derived cache.","commonSituations":"Running contexts longer than the model's trained max_position_embeddings, or a dynamically extended context without rebuilding the rotary cache; also RoPE scaling changes that shrink the effective cache.","solutions":["Increase max_position_embeddings / rebuild the rotary_emb with a larger cos_sin_cache before materializing.","Truncate or chunk positions to stay below cache_len.","Check that _reserved_rope_cache_len reservations account for the full context length."],"exampleFix":"// before\nrotary_emb = RotaryEmbedding(head_dim, max_position=8192)  # positions reach 20000\n// after\nrotary_emb = RotaryEmbedding(head_dim, max_position=32768)","handlingStrategy":"validation","validationCode":"max_pos = int(positions.max())\nassert max_pos < int(rotary_emb.cos_sin_cache.shape[0])","typeGuard":null,"tryCatchPattern":"try:\n    mat.materialize(...)\nexcept RuntimeError as e:\n    if 'cos/sin cache' in str(e):\n        rotary_emb.extend_cache(...)  # or rebuild with larger max_position","preventionTips":["Size the rotary cache for the maximum serving context length up front.","Watch for context-extension features that bypass cache rebuild."],"tags":["rope","context-length","kv-cache"],"backgroundTag":"position-embedding-cache-overflow","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}