sgl-project/sglang · error · ValueError

The length of cache_salt should be equal to the batch size.

Error message

The length of cache_salt should be equal to the batch size.

What it means

Raised when cache_salt is provided as a list whose length differs from batch_size. cache_salt prefixes the radix cache key to separate cache namespaces, and each request in a batch needs exactly one salt.

Source

Thrown at python/sglang/srt/managers/io_struct.py:808

                    "The length of extra_key should be equal to the batch size."
                )
            if any(not isinstance(value, str) for value in self.extra_key):
                raise ValueError("Every extra_key should be a string.")
            self.extra_key = [value or None for value in self.extra_key]
            self.extra_key = self.extra_key * self.parallel_sample_num
        else:
            raise ValueError("extra_key should be a list or a string.")

    def _normalize_cache_salt(self, num):
        """Normalize cache_salt for batch processing."""
        if self.cache_salt is None:
            return
        if isinstance(self.cache_salt, str):
            value = self.cache_salt or None
            self.cache_salt = [value] * num
        elif isinstance(self.cache_salt, list):
            if len(self.cache_salt) != self.batch_size:
                raise ValueError(
                    "The length of cache_salt should be equal to the batch size."
                )
            if any(not isinstance(value, str) for value in self.cache_salt):
                raise ValueError("Every cache_salt should be a string.")
            self.cache_salt = [value or None for value in self.cache_salt]
            self.cache_salt = self.cache_salt * self.parallel_sample_num
        else:
            raise ValueError("cache_salt should be a list or a string.")

    def _normalize_bootstrap_params(self, num):
        """Normalize bootstrap parameters for batch processing."""
        # Normalize bootstrap_host
        if self.bootstrap_host is None:
            self.bootstrap_host = [None] * num
        elif not isinstance(self.bootstrap_host, list):
            self.bootstrap_host = [self.bootstrap_host] * num
        elif isinstance(self.bootstrap_host, list):
            self.bootstrap_host = self.bootstrap_host * self.parallel_sample_num

View on GitHub (pinned to 0132848349)

Solutions

Pass cache_salt as a single str to apply to all requests
Or build a list with exactly batch_size strings, one per request

Example fix

// before
GenerateReqInput(text=['a','b','c'], cache_salt=['s1','s2'])
// after
GenerateReqInput(text=['a','b','c'], cache_salt='s1')

Defensive patterns

Strategy: validation

Validate before calling

if isinstance(cache_salt, list):
    assert len(cache_salt) == len(text) or len(cache_salt) == len(input_ids)

Prevention

Default to a single str salt for uniform batches
Derive the salt list from the same loop that builds prompts

When it happens

Trigger: GenerateReqInput(text=[...3 items...], cache_salt=['s1','s2']) — list of 2 for batch of 3.

Common situations: Reusing a single-element salt list for multi-request batches; dynamic batch assembly where the salt list is built separately from the prompts.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/27214a6ff37ca360. Report an issue: GitHub.