sgl-project/sglang · error · ValueError

memory_video_len must be a multiple of latent_height * laten

Error message

memory_video_len must be a multiple of latent_height * latent_width, got {memory_video_len=} {latent_height=} {latent_width=}

What it means

The flattened memory video token count must be an exact multiple of latent_height * latent_width, i.e. tokens must decompose into whole latent frames. If not, the RoPE builder cannot determine the number of memory latent frames, which usually means the latent grid or the token count is inconsistent with the VAE layout.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/joy_echo/memory.py:446

    fps: float,
    memory_position_mode: str,
    memory_downscale_factor: int = 1,
    sp_target_start_offset: int = 0,
) -> torch.Tensor:
    """Build [memory | target] video RoPE coordinates.

    Under sequence parallelism the target video latents are time-sharded, so
    ``target_num_frames`` is the *local* shard frame count and
    ``sp_target_start_offset`` is the global frame index of this rank's first
    target frame. The memory prefix is replicated (full) on every rank.
    """
    tokens_per_latent_frame = int(latent_height) * int(latent_width)
    if tokens_per_latent_frame <= 0:
        raise ValueError(
            f"Invalid latent grid for memory RoPE: {latent_height=} {latent_width=}"
        )
    if memory_video_len % tokens_per_latent_frame != 0:
        raise ValueError(
            "memory_video_len must be a multiple of latent_height * latent_width, "
            f"got {memory_video_len=} {latent_height=} {latent_width=}"
        )

    memory_latent_frames = memory_video_len // tokens_per_latent_frame
    position_mode = normalize_memory_position_mode(memory_position_mode)

    memory_coords = rope.prepare_video_coords(
        batch_size=batch_size,
        num_frames=memory_latent_frames,
        height=latent_height,
        width=latent_width,
        device=device,
        fps=JOYAI_VIDEO_ROPE_FPS,
        start_frame=0,
    )
    memory_coords = apply_memory_video_downscale(memory_coords, memory_downscale_factor)

View on GitHub (pinned to 0132848349)

Solutions

1. Recompute latent_height/latent_width from the same VAE config used to tokenize the memory video, and pass those exact values
2. Ensure any trimming/merging of the memory token sequence keeps its length a multiple of tokens_per_latent_frame (align cuts to frame boundaries)
3. Print/inspect memory_video_len % (latent_height*latent_width) at runtime to find which stage corrupts the invariant

Example fix

# before
coords = build_memory_video_rope_coords(memory_video_len=1920, latent_height=8, latent_width=8)
# after
tplf = latent_height * latent_width
assert memory_video_len % tplf == 0, f"misaligned memory len {memory_video_len} vs grid {latent_height}x{latent_width}"
coords = build_memory_video_rope_coords(memory_video_len=memory_video_len, latent_height=latent_height, latent_width=latent_width)

tplf = latent_height * latent_width
if memory_video_len % tplf != 0:
    raise ValueError(f"memory len {memory_video_len} not frame-aligned to {latent_height}x{latent_width}; fix tokenization config")

def is_frame_aligned(memory_video_len: int, lh: int, lw: int) -> bool:
    tplf = lh * lw
    return tplf > 0 and memory_video_len % tplf == 0

When it happens

Trigger: Calling build_memory_video_rope_coords where memory_video_len (total memory video tokens) was produced with a different latent_height/latent_width than the ones passed — e.g. tokens computed from a 12x20 grid but coordinates built with 8x8, or a truncated/dropped-token memory sequence.

Common situations: Mismatched VAE downsample config between tokenization and RoPE-coordinate stages; sequence trimming/chunking that cuts the memory prefix at a non-frame-aligned boundary; TP sharding that splits tokens without respecting frame boundaries.

Related errors

• memory_position_mode must be one of {'reference', 'legacy',
• Invalid latent grid for memory RoPE: {latent_height=} {laten
• Previous frame size does not match current delta payload
• Unsupported content type ${header.content_type}
• Error: --model-type requires a value.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d606b992b51c283c. Report an issue: GitHub.