sgl-project/sglang · error · ValueError

Cannot collate mixed VLA noise presence

Error message

Cannot collate mixed VLA noise presence

What it means

Raised by collate_vla_observation_batches when a batch of VLA observations mixes observations that carry a noise tensor with observations whose noise is None. The collated noise must be either a single concatenated tensor or None for the whole batch, so partial presence is rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/vla/observation.py:55

    camera_order = tuple(first.metadata.get("camera_order", ()))
    images = {
        name: torch.cat([obs.images[name] for obs in observations], dim=0)
        for name in camera_order
    }
    image_masks = {
        name: torch.cat([obs.image_masks[name] for obs in observations], dim=0)
        for name in camera_order
    }
    states = [obs.state for obs in observations]
    noises = [obs.noise for obs in observations]
    if any(item is None for item in states) and not all(
        item is None for item in states
    ):
        raise ValueError("Cannot collate mixed VLA state presence")
    if any(item is None for item in noises) and not all(
        item is None for item in noises
    ):
        raise ValueError("Cannot collate mixed VLA noise presence")
    state = (
        None
        if states[0] is None
        else torch.cat([item for item in states if item is not None], dim=0)
    )
    noise = (
        None
        if noises[0] is None
        else torch.cat([item for item in noises if item is not None], dim=0)
    )
    return VLAObservationBatch(
        prompt=[prompt for obs in observations for prompt in obs.prompt],
        images=images,
        image_masks=image_masks,
        state=state,
        noise=noise,
        tokens=torch.cat([obs.tokens for obs in observations], dim=0),
        token_masks=torch.cat([obs.token_masks for obs in observations], dim=0),

View on GitHub (pinned to 0132848349)

Solutions

1. Split the batch by noise presence and collate each subgroup separately
2. Supply explicit noise tensors (e.g. torch.randn with the right shape) for all observations in the group
3. Ensure the request builder always sets noise when the policy requires it

Example fix

// before
batch = collate_vla_observation_batches(obs_list)  # mixed noise presence
// after
noisy = [o for o in obs_list if o.noise is not None]
deterministic = [o for o in obs_list if o.noise is None]
batches = [collate_vla_observation_batches(g) for g in (noisy, deterministic) if g]

noises = [o.noise for o in observations]
if any(n is None for n in noises) and not all(n is None for n in noises):
    raise ValueError("group mixes noise presence; split it first")

def group_has_uniform_noise(obs: list[VLAObservation]) -> bool:
    presence = {o.noise is not None for o in obs}
    return len(presence) == 1

When it happens

Trigger: Calling run_grouped_requests with observations where some have obs.noise set (e.g. for diffusion-style action heads) and others have obs.noise=None.

Common situations: Partially enabling noise-conditioned action generation, mixing test seeds that supply noise with ones that do not, or default-initializing noise only on some code paths.

Related errors

• Cannot collate mixed VLA state presence
• batching config rule requires max_batch_size
• batching config rule cannot set both model and model_contain
• batching config rule requires model or model_contains
• batching config rule max_batch_size must be >= 1

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/df0c143e6603337c. Report an issue: GitHub.