What causes "Invalid VLA prefix cache layer: {layer_idx}" in sgl-project/sglang?

Reset/clear the prefix cache before starting a new forward pass or model

sgl-project/sglang · error · IndexError

Invalid VLA prefix cache layer: {layer_idx}

Error message

Invalid VLA prefix cache layer: {layer_idx}

What it means

Raised by VLAPrefixCache.update when layer_idx exceeds len(self.layers), i.e. the caller asks to update a layer deeper than one past the last cached layer. The cache only supports appending the next layer (layer_idx == len(layers)) or updating an existing one, so gaps are invalid.

Source

Thrown at python/sglang/multimodal_gen/runtime/vla/prefix_cache.py:81

    def update(
        self,
        key_states: torch.Tensor,
        value_states: torch.Tensor,
        layer_idx: int,
    ) -> tuple[torch.Tensor, torch.Tensor]:
        """update the cache with fresh kv from each layer, return the appended prefix kv"""
        if self.read_only:
            prefix_keys, prefix_values = self.get_prefix(layer_idx)
            return (
                torch.cat([prefix_keys, key_states], dim=-2),
                torch.cat([prefix_values, value_states], dim=-2),
            )

        if layer_idx == len(self.layers):
            self.layers.append((key_states, value_states, None))
            return key_states, value_states
        if layer_idx > len(self.layers):
            raise IndexError(f"Invalid VLA prefix cache layer: {layer_idx}")
        cached_keys, cached_values, sliding_window = self.layers[layer_idx]
        key_states = torch.cat([cached_keys, key_states], dim=-2)
        value_states = torch.cat([cached_values, value_states], dim=-2)
        self.layers[layer_idx] = (key_states, value_states, sliding_window)
        return key_states, value_states


def slice_prefix_context(context: PrefixContext, index: int) -> PrefixContext:
    return PrefixContext(
        past_key_values=VLADensePrefixCache(
            tuple(
                (
                    keys[index : index + 1],
                    values[index : index + 1],
                    sliding_window,
                )
                for keys, values, sliding_window in context.past_key_values
            )

View on GitHub (pinned to 0132848349)

Solutions

Reset/clear the prefix cache before starting a new forward pass or model
Call update sequentially per layer starting from 0 so layer_idx only ever equals len(layers) or an existing index
Check the model's layer indexing/offsets if an offset is being added to layer_idx

Example fix

// before
cache.update(k, v, layer_idx=layer_idx + offset)  # jumps past len(layers)
// after
for layer_idx, (k, v) in enumerate(layer_kvs):
    cache.update(k, v, layer_idx=layer_idx)  # strictly sequential

Defensive patterns

Strategy: validation

Validate before calling

assert 0 <= layer_idx <= len(cache.layers), (
    f"layer_idx {layer_idx} out of range for cache with {len(cache.layers)} layers"
)

Prevention

Reset the prefix cache when switching models or starting new episodes
Always iterate layers in order, starting at index 0
Log len(cache.layers) alongside layer_idx when debugging skips

When it happens

Trigger: Calling update(key_states, value_states, layer_idx) with layer_idx > len(self.layers), e.g. skipping layers, calling out of order, or mixing a fresh cache with a model whose layer count/offsets differ.

Common situations: Changing model architecture or layer numbering without resetting the prefix cache; resuming generation with a stale cache; miscounting during layer loops with offset indices.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6cb61b7a4859bb60. Report an issue: GitHub.