{"record":{"id":"6cb61b7a4859bb60","repo":"sgl-project/sglang","slug":"invalid-vla-prefix-cache-layer-layer-idx","errorCode":null,"errorMessage":"Invalid VLA prefix cache layer: {layer_idx}","messagePattern":"Invalid VLA prefix cache layer: (.+?)","errorType":"exception","errorClass":"IndexError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/vla/prefix_cache.py","lineNumber":81,"sourceCode":"    def update(\n        self,\n        key_states: torch.Tensor,\n        value_states: torch.Tensor,\n        layer_idx: int,\n    ) -> tuple[torch.Tensor, torch.Tensor]:\n        \"\"\"update the cache with fresh kv from each layer, return the appended prefix kv\"\"\"\n        if self.read_only:\n            prefix_keys, prefix_values = self.get_prefix(layer_idx)\n            return (\n                torch.cat([prefix_keys, key_states], dim=-2),\n                torch.cat([prefix_values, value_states], dim=-2),\n            )\n\n        if layer_idx == len(self.layers):\n            self.layers.append((key_states, value_states, None))\n            return key_states, value_states\n        if layer_idx > len(self.layers):\n            raise IndexError(f\"Invalid VLA prefix cache layer: {layer_idx}\")\n        cached_keys, cached_values, sliding_window = self.layers[layer_idx]\n        key_states = torch.cat([cached_keys, key_states], dim=-2)\n        value_states = torch.cat([cached_values, value_states], dim=-2)\n        self.layers[layer_idx] = (key_states, value_states, sliding_window)\n        return key_states, value_states\n\n\ndef slice_prefix_context(context: PrefixContext, index: int) -> PrefixContext:\n    return PrefixContext(\n        past_key_values=VLADensePrefixCache(\n            tuple(\n                (\n                    keys[index : index + 1],\n                    values[index : index + 1],\n                    sliding_window,\n                )\n                for keys, values, sliding_window in context.past_key_values\n            )","sourceCodeStart":63,"sourceCodeEnd":99,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/vla/prefix_cache.py#L63-L99","documentation":"Raised by VLAPrefixCache.update when layer_idx exceeds len(self.layers), i.e. the caller asks to update a layer deeper than one past the last cached layer. The cache only supports appending the next layer (layer_idx == len(layers)) or updating an existing one, so gaps are invalid.","triggerScenarios":"Calling update(key_states, value_states, layer_idx) with layer_idx > len(self.layers), e.g. skipping layers, calling out of order, or mixing a fresh cache with a model whose layer count/offsets differ.","commonSituations":"Changing model architecture or layer numbering without resetting the prefix cache; resuming generation with a stale cache; miscounting during layer loops with offset indices.","solutions":["Reset/clear the prefix cache before starting a new forward pass or model","Call update sequentially per layer starting from 0 so layer_idx only ever equals len(layers) or an existing index","Check the model's layer indexing/offsets if an offset is being added to layer_idx"],"exampleFix":"// before\ncache.update(k, v, layer_idx=layer_idx + offset)  # jumps past len(layers)\n// after\nfor layer_idx, (k, v) in enumerate(layer_kvs):\n    cache.update(k, v, layer_idx=layer_idx)  # strictly sequential","handlingStrategy":"validation","validationCode":"assert 0 <= layer_idx <= len(cache.layers), (\n    f\"layer_idx {layer_idx} out of range for cache with {len(cache.layers)} layers\"\n)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Reset the prefix cache when switching models or starting new episodes","Always iterate layers in order, starting at index 0","Log len(cache.layers) alongside layer_idx when debugging skips"],"tags":["vla","prefix-cache","kv-cache","index-error"],"backgroundTag":"cache-index-out-of-range","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}