sgl-project/sglang · error · ValueError

Grammar mask max_rows must be positive, got {max_rows}

Error message

Grammar mask max_rows must be positive, got {max_rows}

What it means

The grammar-backend mask buffer API requires a strictly positive row capacity; register_vocab_mask_buffer raises ValueError when max_rows <= 0 because a zero/negative-capacity buffer cannot hold any grammar mask rows.

Source

Thrown at python/sglang/srt/constrained/base_grammar_backend.py:307

            copied_value = value.copy()
            copied_value.maybe_init_reasoning(require_reasoning)
            return copied_value, True
        value = self.executor.submit(self._init_value_dispatch, key, require_reasoning)
        return value, False

    def set_cache(self, key: Tuple[str, str], value: BaseGrammarObject):
        self.cache[key] = value

    def reset(self):
        self.cache.clear()


def register_vocab_mask_buffer(
    name: str, vocab_mask: torch.Tensor, max_rows: int
) -> torch.Tensor:
    """Register a fixed-capacity mask buffer, preserving an equivalent one."""
    if max_rows <= 0:
        raise ValueError(f"Grammar mask max_rows must be positive, got {max_rows}")
    if vocab_mask.ndim == 0 or vocab_mask.shape[0] != max_rows:
        raise ValueError(
            f"Grammar mask buffer {name!r} must have {max_rows} rows, "
            f"got shape {tuple(vocab_mask.shape)}"
        )

    buffers = get_resources().buffers
    existing = buffers.get(name)
    if existing is not None:
        if (
            existing.shape != vocab_mask.shape
            or existing.dtype != vocab_mask.dtype
            or existing.device != vocab_mask.device
        ):
            raise RuntimeError(
                f"Grammar mask buffer {name!r} was already initialized as "
                f"{tuple(existing.shape)}, {existing.dtype}, {existing.device}; "
                f"new buffer is {tuple(vocab_mask.shape)}, {vocab_mask.dtype}, "

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure max_rows is derived from a positive quantity (e.g. max(1, num_active_grammar_requests) or the configured max running request count)
2. Check server args like --max-running-requests / batch-size settings for zero values
3. If registering lazily, skip registration when there are no grammar requests instead of passing 0

Example fix

// before
backend.register_vocab_mask_buffer("vocab_mask", mask, num_grammar_reqs)  # 0 when idle
// after
if num_grammar_reqs > 0:
    backend.register_vocab_mask_buffer("vocab_mask", mask, num_grammar_reqs)

if max_rows <= 0:
    raise SkipRegistration("no grammar rows")
backend.register_vocab_mask_buffer(name, vocab_mask, max_rows)

When it happens

Trigger: Calling register_vocab_mask_buffer(name, vocab_mask, max_rows) with max_rows=0 or negative, e.g. when the caller computes max_rows from request batch size or grammar count and that computation yields 0 (no active grammar requests).

Common situations: Scheduler/attention code sizing the constrained-decoding mask buffer from an empty batch or a misconfigured max_num_seqs of 0; refactors that pass a placeholder capacity of 0 during initialization.

Related errors

• v_cache must be provided
• q can only be None when only_qv=True
• q must be provided unless qv is provided with only_qv=True
• `dt_bias` must have {HV * K} elements (got {dt_bias.numel()}
• The batch size is expected to be 1 rather than {q.shape[0]}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ef63009cdd9f31d4. Report an issue: GitHub.