{"record":{"id":"8adf3f08a668f38f","repo":"sgl-project/sglang","slug":"lingbot-causal-sequence-sharding-requires-forward","errorCode":null,"errorMessage":"LingBot causal sequence sharding requires forward_batch.sequence_shard_splits.","messagePattern":"LingBot causal sequence sharding requires forward_batch\\.sequence_shard_splits\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py","lineNumber":284,"sourceCode":"            if sequence_shard_enabled:\n                raise NotImplementedError(\n                    \"LingBot causal sequence sharding currently requires kv_cache-backed inference.\"\n                )\n            return super().forward(\n                q,\n                k,\n                v,\n                (cos, sin),\n                block_mask,\n                kv_cache,\n                current_start,\n                cache_start,\n            )\n\n        if sequence_shard_enabled:\n            seq_splits = getattr(forward_batch, \"sequence_shard_splits\", None)\n            if seq_splits is None:\n                raise ValueError(\n                    \"LingBot causal sequence sharding requires forward_batch.sequence_shard_splits.\"\n                )\n            seq_splits = list(seq_splits)\n            uniform_seq_splits = _sequence_splits_are_uniform(seq_splits)\n            # Pack Q/K/V to avoid launching three Ulysses all-to-all collectives.\n            qkv = torch.cat([roped_query, roped_key, v], dim=-1)\n            qkv = (\n                _usp_input_all_to_all(qkv, head_dim=2)\n                if uniform_seq_splits\n                else _usp_input_all_to_all_varlen(qkv, seq_splits, head_dim=2)\n            )\n            roped_query, roped_key, v = qkv.chunk(3, dim=-1)\n\n        if (\n            not sequence_shard_enabled\n            and not update_cache_only\n            and kv_cache.can_direct_current_attention(roped_key.shape[1])\n        ):","sourceCodeStart":266,"sourceCodeEnd":302,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py#L266-L302","documentation":"When sequence sharding is active, LingBot packs Q/K/V into one all-to-all and needs to know how the packed sequence splits per request; it reads forward_batch.sequence_shard_splits. If that attribute is missing, it cannot partition tokens across Ulysses ranks and raises.","triggerScenarios":"forward_batch.enable_sequence_shard=True with ulysses_degree > 1 and a valid kv_cache, but the batch object was never given sequence_shard_splits (older batch class, manually built batch, or sharding not wired in the scheduler).","commonSituations":"Upgrading the model code without the matching scheduler change that populates sequence_shard_splits; constructing ForwardBatch manually in tests; version skew between model and runtime batch definitions.","solutions":["Update the scheduler/batch construction to set forward_batch.sequence_shard_splits (list of per-request split sizes) when enabling sharding","Upgrade sglang to a version where the model and batch plumbing are in sync","Disable enable_sequence_shard until the splits are provided"],"exampleFix":"# before\nbatch.enable_sequence_shard = True\nout = model(x, forward_batch=batch)\n\n# after\nbatch.enable_sequence_shard = True\nbatch.sequence_shard_splits = [S1, S2, ...]   # per-request token splits\nout = model(x, forward_batch=batch)","handlingStrategy":"type-guard","validationCode":"if getattr(forward_batch, 'enable_sequence_shard', False):\\n    splits = getattr(forward_batch, 'sequence_shard_splits', None)\\n    if splits is None:\\n        forward_batch.enable_sequence_shard = False  # or populate splits","typeGuard":"def shard_splits_present(forward_batch) -> bool:\\n    return getattr(forward_batch, 'sequence_shard_splits', None) is not None","tryCatchPattern":null,"preventionTips":["Populate sequence_shard_splits wherever you set enable_sequence_shard","Keep model and scheduler code versions in sync","Add an integration test for the sharded path"],"tags":["sequence-parallelism","missing-attribute","forward-batch","lingbot"],"backgroundTag":"missing-required-attribute","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}