{"record":{"id":"66ffc64cf2114105","repo":"sgl-project/sglang","slug":"lingbot-causal-sequence-sharding-currently-support","errorCode":null,"errorMessage":"LingBot causal sequence sharding currently supports ulysses_degree > 1 with ring_degree = 1 only.","messagePattern":"LingBot causal sequence sharding currently supports ulysses_degree > 1 with ring_degree = 1 only\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py","lineNumber":1496,"sourceCode":"        timestep: torch.LongTensor,\n        encoder_hidden_states_image: torch.Tensor | list[torch.Tensor] | None = None,\n        kv_cache: list[CausalSelfAttentionKVCache] | None = None,\n        crossattn_cache: list[CrossAttentionKVCache] | None = None,\n        current_start: int = 0,\n        cache_start: int = 0,\n        start_frame: int = 0,\n        c2ws_plucker_emb: torch.Tensor | None = None,\n        skip_final_projection: bool = False,\n    ) -> torch.Tensor:\n        forward_batch = get_forward_context().forward_batch\n        sequence_shard_enabled = (\n            forward_batch is not None\n            and getattr(forward_batch, \"enable_sequence_shard\", False)\n            and self.sp_size > 1\n        )\n        if sequence_shard_enabled:\n            if get_ring_parallel_world_size() > 1:\n                raise NotImplementedError(\n                    \"LingBot causal sequence sharding currently supports ulysses_degree > 1 with ring_degree = 1 only.\"\n                )\n            if get_ulysses_parallel_world_size() <= 1:\n                raise NotImplementedError(\n                    \"LingBot causal sequence sharding currently requires ulysses_degree > 1.\"\n                )\n        orig_dtype = hidden_states.dtype\n        if not isinstance(encoder_hidden_states, torch.Tensor):\n            encoder_hidden_states = encoder_hidden_states[0]\n        if (\n            isinstance(encoder_hidden_states_image, list)\n            and len(encoder_hidden_states_image) > 0\n        ):\n            encoder_hidden_states_image = encoder_hidden_states_image[0]\n        else:\n            encoder_hidden_states_image = None\n\n        batch_size, _, num_frames, height, width = hidden_states.shape","sourceCodeStart":1478,"sourceCodeEnd":1514,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py#L1478-L1514","documentation":"LingBot's causal sequence-sharding implementation only handles Ulysses all-to-all with a single ring group (ring_degree=1). If ring parallelism is also active (ring world size > 1) the token partitioning logic is not implemented, so it raises NotImplementedError.","triggerScenarios":"Launching with both ring parallel degree > 1 and enable_sequence_shard with sp_size > 1, e.g. combining --ring-size with Ulysses sequence sharding.","commonSituations":"Trying to maximize multi-node parallelism by stacking ring + Ulysses; migrating configs from a Wan setup that used ring attention; default parallel plans enabling ring implicitly.","solutions":["Disable ring parallelism (run with ring_degree=1) while using sequence sharding","Or drop sequence sharding and rely on ring attention for long contexts","Wait for/port an implementation that composes ring + Ulysses sharding"],"exampleFix":"# before\n--ring-size 2 --ulysses-size 4  (with enable_sequence_shard)\n\n# after\n--ring-size 1 --ulysses-size 4  (sequence sharding active)","handlingStrategy":"validation","validationCode":"if shard_enabled and get_ring_parallel_world_size() > 1:\\n    raise SystemExit('disable ring parallelism before enabling sequence sharding')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Don't stack ring attention with Ulysses sharding","Encode supported parallel combos in launch scripts","Fail fast at launch rather than mid-forward"],"tags":["not-implemented","sequence-parallelism","ring-attention","lingbot"],"backgroundTag":"unsupported-parallel-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}