{"record":{"id":"533a672ddb2c01f8","repo":"sgl-project/sglang","slug":"cannot-collate-mixed-vla-state-presence","errorCode":null,"errorMessage":"Cannot collate mixed VLA state presence","messagePattern":"Cannot collate mixed VLA state presence","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/vla/observation.py","lineNumber":51,"sourceCode":"def collate_vla_observation_batches(\n    observations: list[VLAObservationBatch],\n) -> VLAObservationBatch:\n    first = observations[0]\n    camera_order = tuple(first.metadata.get(\"camera_order\", ()))\n    images = {\n        name: torch.cat([obs.images[name] for obs in observations], dim=0)\n        for name in camera_order\n    }\n    image_masks = {\n        name: torch.cat([obs.image_masks[name] for obs in observations], dim=0)\n        for name in camera_order\n    }\n    states = [obs.state for obs in observations]\n    noises = [obs.noise for obs in observations]\n    if any(item is None for item in states) and not all(\n        item is None for item in states\n    ):\n        raise ValueError(\"Cannot collate mixed VLA state presence\")\n    if any(item is None for item in noises) and not all(\n        item is None for item in noises\n    ):\n        raise ValueError(\"Cannot collate mixed VLA noise presence\")\n    state = (\n        None\n        if states[0] is None\n        else torch.cat([item for item in states if item is not None], dim=0)\n    )\n    noise = (\n        None\n        if noises[0] is None\n        else torch.cat([item for item in noises if item is not None], dim=0)\n    )\n    return VLAObservationBatch(\n        prompt=[prompt for obs in observations for prompt in obs.prompt],\n        images=images,\n        image_masks=image_masks,","sourceCodeStart":33,"sourceCodeEnd":69,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/vla/observation.py#L33-L69","documentation":"Raised by collate_vla_observation_batches when a batch of VLA observations mixes observations that carry a robot state tensor with observations whose state is None. Because the collated output is either a concatenated tensor or None, a partially-populated batch is ambiguous and rejected.","triggerScenarios":"Calling run_grouped_requests (which calls collate_vla_observation_batches) with a group where some VLAObservation objects have obs.state set and others have obs.state=None.","commonSituations":"Building observation batches from heterogeneous sources (e.g. real robot teleop data with states vs. replay/image-only data without), or a data loader default-initializing state to None for some entries.","solutions":["Filter or split the batch so all observations uniformly have or lack state before calling run_grouped_requests","Backfill a zero/neutral state tensor for observations missing state if the model accepts it","Fix upstream observation construction so state presence is consistent across a group"],"exampleFix":"// before\nbatches = collate_vla_observation_batches(mixed_observations)  // some state=None\n// after\nwith_state = [o for o in mixed_observations if o.state is not None]\nwithout_state = [o for o in mixed_observations if o.state is None]\nfor group in (with_state, without_state):\n    if group:\n        batches = collate_vla_observation_batches(group)","handlingStrategy":"validation","validationCode":"states = [o.state for o in observations]\nif any(s is None for s in states) and not all(s is None for s in states):\n    raise ValueError(\"group mixes state presence; split it first\")","typeGuard":"def group_has_uniform_state(obs: list[VLAObservation]) -> bool:\n    presence = {o.state is not None for o in obs}\n    return len(presence) == 1","tryCatchPattern":null,"preventionTips":["Construct observation groups from a single data source at a time","Add an assert on uniform state presence before run_grouped_requests"],"tags":["vla","batching","robotics","validation"],"backgroundTag":"mixed-none-values-in-batch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}