sgl-project/sglang · error · ValueError

Invalid latent grid for memory RoPE: {latent_height=} {laten

Error message

Invalid latent grid for memory RoPE: {latent_height=} {latent_width=}

What it means

Memory video RoPE coordinates require a positive latent grid: latent_height * latent_width must be > 0 because each latent frame contributes exactly that many tokens. A zero or negative product means the VAE downsampling factors produced a degenerate grid (e.g. video shorter/smaller than the spatial/temporal stride), so positions cannot be computed.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py:442

    target_num_frames: int,
    latent_height: int,
    latent_width: int,
    device: torch.device,
    fps: float,
    memory_position_mode: str,
    memory_downscale_factor: int = 1,
    sp_target_start_offset: int = 0,
) -> torch.Tensor:
    """Build [memory | target] video RoPE coordinates.

    Under sequence parallelism the target video latents are time-sharded, so
    ``target_num_frames`` is the *local* shard frame count and
    ``sp_target_start_offset`` is the global frame index of this rank's first
    target frame. The memory prefix is replicated (full) on every rank.
    """
    tokens_per_latent_frame = int(latent_height) * int(latent_width)
    if tokens_per_latent_frame <= 0:
        raise ValueError(
            f"Invalid latent grid for memory RoPE: {latent_height=} {latent_width=}"
        )
    if memory_video_len % tokens_per_latent_frame != 0:
        raise ValueError(
            "memory_video_len must be a multiple of latent_height * latent_width, "
            f"got {memory_video_len=} {latent_height=} {latent_width=}"
        )

    memory_latent_frames = memory_video_len // tokens_per_latent_frame
    position_mode = normalize_memory_position_mode(memory_position_mode)

    memory_coords = rope.prepare_video_coords(
        batch_size=batch_size,
        num_frames=memory_latent_frames,
        height=latent_height,
        width=latent_width,
        device=device,
        fps=JOYAI_VIDEO_ROPE_FPS,

View on GitHub (pinned to 0132848349)

Solutions

Check the input video tensor's H and W before calling: ensure H >= vae_spatial_downsample and W >= vae_spatial_downsample
Verify the VAE downsample factor matches the checkpoint you loaded (e.g. 8x8 spatial, temporal stride) so computed latent dims are positive
Guard in the caller: skip memory-RoPE path or raise a clearer upstream error when the latent grid would be degenerate

Example fix

# before
coords = build_memory_video_rope_coords(memory_video_len=m, latent_height=h // 8, latent_width=w // 8)
# after
lh, lw = h // 8, w // 8
assert lh > 0 and lw > 0, f"video too small for VAE stride: {h=} {w=}"
coords = build_memory_video_rope_coords(memory_video_len=m, latent_height=lh, latent_width=lw)

Defensive patterns

Strategy: validation

Validate before calling

lh, lw = int(latent_height), int(latent_width)
if lh <= 0 or lw <= 0:
    raise ValueError(f"degenerate latent grid {lh=} {lw=}; check video size vs VAE stride")

Type guard

def has_valid_latent_grid(latent_height: int, latent_width: int) -> bool:
    return int(latent_height) * int(latent_width) > 0

Prevention

Validate input video H/W against the VAE downsample factor at the pipeline entry
Log computed latent dims per clip to catch zero-size frames early

When it happens

Trigger: Calling build_memory_video_rope_coords with latent_height=0 or latent_width=0 (or negative), typically derived from (H // vae_spatial_stride) or (W // vae_spatial_stride) for a very small or empty video tensor.

Common situations: Feeding a stub/test video of a few pixels; empty batch producing H=0/W=0; mismatched VAE downsample factor config vs actual model; upstream crop/resize stage producing a zero-sized frame.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9ce0909c8a5bb1c7. Report an issue: GitHub.