sgl-project/sglang · error · ValueError

Cannot collate mixed VLA state presence

Error message

Cannot collate mixed VLA state presence

What it means

Raised by collate_vla_observation_batches when a batch of VLA observations mixes observations that carry a robot state tensor with observations whose state is None. Because the collated output is either a concatenated tensor or None, a partially-populated batch is ambiguous and rejected.

Source

Thrown at python/sglang/multimodal_gen/runtime/vla/observation.py:51

def collate_vla_observation_batches(
    observations: list[VLAObservationBatch],
) -> VLAObservationBatch:
    first = observations[0]
    camera_order = tuple(first.metadata.get("camera_order", ()))
    images = {
        name: torch.cat([obs.images[name] for obs in observations], dim=0)
        for name in camera_order
    }
    image_masks = {
        name: torch.cat([obs.image_masks[name] for obs in observations], dim=0)
        for name in camera_order
    }
    states = [obs.state for obs in observations]
    noises = [obs.noise for obs in observations]
    if any(item is None for item in states) and not all(
        item is None for item in states
    ):
        raise ValueError("Cannot collate mixed VLA state presence")
    if any(item is None for item in noises) and not all(
        item is None for item in noises
    ):
        raise ValueError("Cannot collate mixed VLA noise presence")
    state = (
        None
        if states[0] is None
        else torch.cat([item for item in states if item is not None], dim=0)
    )
    noise = (
        None
        if noises[0] is None
        else torch.cat([item for item in noises if item is not None], dim=0)
    )
    return VLAObservationBatch(
        prompt=[prompt for obs in observations for prompt in obs.prompt],
        images=images,
        image_masks=image_masks,

View on GitHub (pinned to 0132848349)

Solutions

1. Filter or split the batch so all observations uniformly have or lack state before calling run_grouped_requests
2. Backfill a zero/neutral state tensor for observations missing state if the model accepts it
3. Fix upstream observation construction so state presence is consistent across a group

Example fix

// before
batches = collate_vla_observation_batches(mixed_observations)  // some state=None
// after
with_state = [o for o in mixed_observations if o.state is not None]
without_state = [o for o in mixed_observations if o.state is None]
for group in (with_state, without_state):
    if group:
        batches = collate_vla_observation_batches(group)

states = [o.state for o in observations]
if any(s is None for s in states) and not all(s is None for s in states):
    raise ValueError("group mixes state presence; split it first")

def group_has_uniform_state(obs: list[VLAObservation]) -> bool:
    presence = {o.state is not None for o in obs}
    return len(presence) == 1

When it happens

Trigger: Calling run_grouped_requests (which calls collate_vla_observation_batches) with a group where some VLAObservation objects have obs.state set and others have obs.state=None.

Common situations: Building observation batches from heterogeneous sources (e.g. real robot teleop data with states vs. replay/image-only data without), or a data loader default-initializing state to None for some entries.

Related errors

• Cannot collate mixed VLA noise presence
• batching config rule requires max_batch_size
• batching config rule cannot set both model and model_contain
• batching config rule requires model or model_contains
• batching config rule max_batch_size must be >= 1

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/533a672ddb2c01f8. Report an issue: GitHub.