sgl-project/sglang · critical · RuntimeError

GLM-Image AR returned too few output_ids: got {actual_output

Error message

GLM-Image AR returned too few output_ids: got {actual_output_len}, need at least {expected_output_len} (large_image_offset={large_image_offset}, token_h={token_h}, token_w={token_w}).

What it means

The GLM-Image autoregressive (AR) stage produced fewer output token ids than required to fill the requested image token grid. The expected length is large_image_offset + token_h * token_w, derived from the requested resolution; anything shorter means the AR model stopped (e.g. emitted EOS) or was capped by max_new_tokens before covering the full latent grid. This is a hard RuntimeError because downstream stages cannot slice a full token grid from a truncated output.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/glm_image.py:384

            logger.error(
                "SGLang encoder request to %s failed: %s",
                server_args.srt_encoder_url,
                e,
            )
            raise
        return response.json()

    def _extract_prior_token_ids(
        self,
        generated_ids: Any,
        generation_shape: tuple[int, int, int],
        device: torch.device,
    ) -> torch.Tensor:
        large_image_offset, token_h, token_w = generation_shape
        expected_output_len = large_image_offset + token_h * token_w
        actual_output_len = 0 if generated_ids is None else len(generated_ids)
        if actual_output_len < expected_output_len:
            raise RuntimeError(
                "GLM-Image AR returned too few output_ids: "
                f"got {actual_output_len}, need at least {expected_output_len} "
                f"(large_image_offset={large_image_offset}, "
                f"token_h={token_h}, token_w={token_w})."
            )

        prior_token_ids_d32 = torch.tensor(
            generated_ids[large_image_offset : large_image_offset + token_h * token_w],
            device=device,
        )
        return self._upsample_token_ids(prior_token_ids_d32, token_h, token_w)

    def generate_prior_tokens(
        self,
        prompt: str,
        height: int,
        width: int,
        server_args: ServerArgs,

View on GitHub (pinned to 0132848349)

Solutions

Verify max_new_tokens is exactly grid_h * grid_w + 1 (the +1 accounts for EOS) before calling the AR stage
Check the requested height/width and the resulting token_h/token_w so the grid matches the token budget
If using an external AR server, inspect its response body to confirm output_ids length and that EOS is not terminating generation early
Upgrade/align the GLM-Image AR model version so large_image_offset matches the checkpoint convention
Retry the request after lowering resolution until output length reliably covers the grid

Example fix

# before
sampling_params = {"max_new_tokens": 1024}  # grid needs 36*36+1 = 1297

# after
expected = large_image_offset + token_h * token_w
sampling_params = {"max_new_tokens": expected + 1}  # +1 for EOS

Defensive patterns

Strategy: validation

Validate before calling

expected = large_image_offset + token_h * token_w
max_new_tokens = expected + 1  # +1 for EOS
if sampling_params.get("max_new_tokens", 0) < expected + 1:
    raise ValueError(f"max_new_tokens {sampling_params.get('max_new_tokens')} < required {expected + 1}")

Try / catch

except RuntimeError as e:
    if "too few output_ids" in str(e):
        lower_resolution_and_retry()  # smaller token grid
    else:
        raise

Prevention

Always compute max_new_tokens from the token grid, never hard-code it
Validate requested height/width maps to a token grid within the AR context limit
Log actual vs expected output length on every AR call to catch drift early

When it happens

Trigger: Calling generate_prior_tokens / generate_prior_tokens_batch with max_new_tokens set below grid_h*grid_w + 1; the AR server returning early because EOS was sampled; a mismatch between the requested image size (token_h/token_w) and the sampling params sent to the AR endpoint; or generated_ids being None/empty from a failed generation.

Common situations: Custom height/width that produce a larger token grid than the configured max_new_tokens; using an external SGLang encoder/AR server whose sampling params truncate output; model version change altering the large-image offset convention; batch requests where one item exhausts its token budget early.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7594355452814a80. Report an issue: GitHub.