sgl-project/sglang · error · ValueError

dimension ({dimension}) must be divisible by 2

Error message

dimension ({dimension}) must be divisible by 2

What it means

create_sinusoidal_pos_embedding builds a sinusoidal time embedding of the requested dimension by splitting it into sin/cos halves, so the dimension must be even. An odd dimension makes the construction impossible and the function raises immediately. It is called by embed_suffix during forward, so the bad dimension usually traces back to a model config (embedding dim).

Source

Thrown at python/sglang/multimodal_gen/runtime/models/vlas/pi05_core.py:833

        return GemmaVariantConfig(
            width=2048,
            depth=18,
            mlp_dim=16_384,
            num_heads=8,
            num_kv_heads=1,
            head_dim=256,
        )
    raise ValueError(f"Unknown Pi05 Gemma variant: {variant}")


def create_sinusoidal_pos_embedding(
    time: torch.Tensor,
    dimension: int,
    min_period: float,
    max_period: float,
) -> Tensor:
    if dimension % 2 != 0:
        raise ValueError(f"dimension ({dimension}) must be divisible by 2")
    if time.ndim != 1:
        raise ValueError("time must have shape [batch]")
    fraction = torch.linspace(
        0.0,
        1.0,
        dimension // 2,
        dtype=torch.float64,
        device=time.device,
    )
    period = min_period * (max_period / min_period) ** fraction
    scaling = 1.0 / period * 2 * math.pi
    sin_input = scaling[None, :] * time[:, None].to(torch.float64)
    return torch.cat([torch.sin(sin_input), torch.cos(sin_input)], dim=1)


def make_att_2d_masks(
    pad_masks: torch.Tensor,
    att_masks: torch.Tensor,

View on GitHub (pinned to 0132848349)

Solutions

1. Set the embedding dimension to an even number (e.g. 1024 instead of 1023) in the config that feeds embed_suffix
2. Trace where dimension comes from and round it: dimension = 2 * (dimension // 2)
3. Validate parity at config-load time before model construction

Example fix

# before
time_emb = create_sinusoidal_pos_embedding(t, dimension=1023, min_period=..., max_period=...)

# after
time_emb = create_sinusoidal_pos_embedding(t, dimension=1024, min_period=..., max_period=...)

dim = cfg.time_embedding_dim
assert dim % 2 == 0, f"time_embedding_dim must be even, got {dim}"

def is_even_dimension(dim: int) -> bool:
    """Type guard: dimension usable for sinusoidal embedding."""
    return isinstance(dim, int) and dim > 0 and dim % 2 == 0

try:
    emb = create_sinusoidal_pos_embedding(t, dim, pmin, pmax)
except ValueError as e:
    if "divisible by 2" in str(e):
        dim = 2 * (dim // 2)
        emb = create_sinusoidal_pos_embedding(t, dim, pmin, pmax)
    else:
        raise

When it happens

Trigger: Calling create_sinusoidal_pos_embedding(time, dimension=odd_number, ...) — typically because a config value like the action/time embedding dimension was set to an odd number (e.g. 1023) or computed dynamically to an odd size.

Common situations: Customizing Pi05 model dimensions without keeping them even; deriving dimension from another parameter (e.g. hidden_size - 1); porting configs from another codebase with different parity conventions.

Related errors

• Unknown Pi05 Gemma variant: {variant}
• time must have shape [batch]
• pad_masks and att_masks must be [batch, seq]
• Invalid Pi05 precision: {precision}
• Unsupported Pi05 dtype: {dtype_name}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e9adb3ee1cf8ad9a. Report an issue: GitHub.