{"record":{"id":"70c5603507734e5b","repo":"sgl-project/sglang","slug":"lingbot-causal-sequence-sharding-currently-require","errorCode":null,"errorMessage":"LingBot causal sequence sharding currently requires kv_cache-backed inference.","messagePattern":"LingBot causal sequence sharding currently requires kv_cache-backed inference\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py","lineNumber":267,"sourceCode":"            )\n            roped_query = roped_query.type_as(v)\n            roped_key = roped_key.type_as(v)\n        else:\n            roped_query = _apply_rotary_emb(q, cos, sin, is_neox_style=False).type_as(v)\n            roped_key = _apply_rotary_emb(k, cos, sin, is_neox_style=False).type_as(v)\n        forward_batch = get_forward_context().forward_batch\n        seq_splits = None\n        uniform_seq_splits = False\n        sequence_shard_enabled = (\n            kv_cache is not None\n            and forward_batch is not None\n            and getattr(forward_batch, \"enable_sequence_shard\", False)\n            and get_ulysses_parallel_world_size() > 1\n        )\n\n        if kv_cache is None:\n            if sequence_shard_enabled:\n                raise NotImplementedError(\n                    \"LingBot causal sequence sharding currently requires kv_cache-backed inference.\"\n                )\n            return super().forward(\n                q,\n                k,\n                v,\n                (cos, sin),\n                block_mask,\n                kv_cache,\n                current_start,\n                cache_start,\n            )\n\n        if sequence_shard_enabled:\n            seq_splits = getattr(forward_batch, \"sequence_shard_splits\", None)\n            if seq_splits is None:\n                raise ValueError(\n                    \"LingBot causal sequence sharding requires forward_batch.sequence_shard_splits.\"","sourceCodeStart":249,"sourceCodeEnd":285,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py#L249-L285","documentation":"LingBot's causal sequence sharding relies on a paged KV cache to gather full-sequence history on each shard. When sequence sharding is enabled (forward_batch.enable_sequence_shard with ulysses degree > 1) but kv_cache is None, the code paths can't reconstruct the causal context, so it raises NotImplementedError.","triggerScenarios":"Calling forward with enable_sequence_shard=True, ulysses world size > 1, and kv_cache=None (e.g. a pure eager forward with no radix cache attached, as in unit tests or offline single-step runs).","commonSituations":"Running the block standalone in tests without the scheduler's KV cache; a batch type that forgot to attach cache tensors; trying the sharded path in a non-serving script.","solutions":["Run through the serving/scheduler path so forward_batch carries a kv_cache","Disable sequence sharding for eager/offline runs (leave enable_sequence_shard unset/False)","Extend the model with a non-cache fallback if you need sharded eager execution (code change)"],"exampleFix":"# before\nout = attn(q, k, v, forward_batch=batch)  # batch.enable_sequence_shard=True, no kv_cache\n\n# after\nbatch.enable_sequence_shard = False       # eager / cacheless run\nout = attn(q, k, v, forward_batch=batch)","handlingStrategy":"type-guard","validationCode":"shard_ok = (forward_batch is not None\\n              and getattr(forward_batch, 'enable_sequence_shard', False)\\n              and get_ulysses_parallel_world_size() > 1\\n              and forward_batch.token_to_kv_pool is not None)\\nif not shard_ok:\\n    forward_batch.enable_sequence_shard = False","typeGuard":"def has_kv_cache(forward_batch) -> bool:\\n    return getattr(forward_batch, 'token_to_kv_pool', None) is not None","tryCatchPattern":"try:\\n    out = attn(q, k, v, forward_batch=batch, kv_cache=cache)\\nexcept NotImplementedError:\\n    batch.enable_sequence_shard = False\\n    out = attn(q, k, v, forward_batch=batch)","preventionTips":["Only enable sequence sharding through the scheduler path","Attach cache tensors to test batches","Guard the flag based on cache availability"],"tags":["not-implemented","sequence-parallelism","kv-cache","lingbot"],"backgroundTag":"feature-requires-kv-cache","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}