{"record":{"id":"018e342afa37fa9e","repo":"sgl-project/sglang","slug":"spec-info-is-unset-in-target-verify-mode-the-exte","errorCode":null,"errorMessage":"spec_info is unset in TARGET_VERIFY mode; the extend_* metadata can only be derived from spec_info for speculative verify batches.","messagePattern":"spec_info is unset in TARGET_VERIFY mode; the extend_\\* metadata can only be derived from spec_info for speculative verify batches\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/intel_amx_backend.py","lineNumber":83,"sourceCode":"        self.num_draft_tokens = get_spec().speculative_num_draft_tokens\n\n    def _build_extend_metadata(self, forward_batch: ForwardBatch):\n        \"\"\"Resolve (seq_lens, extend_seq_lens, extend_start_loc, tree_mask) for\n        forward_extend, once per forward pass.\n\n        In TARGET_VERIFY mode the batch carries no extend_* fields, so they are\n        derived from spec_info (mirrors the CUDA unified path in\n        triton_backend.py); each request extends by exactly num_draft_tokens\n        tokens. Outside spec decoding the fields are passed through.\n        \"\"\"\n        bs = forward_batch.batch_size\n        seq_lens = forward_batch.seq_lens\n        tree_mask = None\n\n        if forward_batch.forward_mode.is_target_verify():\n            spec_info = forward_batch.spec_info\n            if spec_info is None:\n                raise RuntimeError(\n                    \"spec_info is unset in TARGET_VERIFY mode; the extend_* \"\n                    \"metadata can only be derived from spec_info for \"\n                    \"speculative verify batches.\"\n                )\n            num_draft_tokens = spec_info.draft_token_num\n            extend_seq_lens = torch.full(\n                (bs,), num_draft_tokens, dtype=torch.int32, device=self.device\n            )\n            # Uniform extend lengths: start locations form a plain range.\n            extend_start_loc = torch.arange(\n                0,\n                bs * num_draft_tokens,\n                num_draft_tokens,\n                dtype=torch.int32,\n                device=self.device,\n            )\n            seq_lens = forward_batch.seq_lens + num_draft_tokens\n            # Speculative verify with a token tree: each draft token may only","sourceCodeStart":65,"sourceCodeEnd":101,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/intel_amx_backend.py#L65-L101","documentation":"The Intel AMX attention backend derives TARGET_VERIFY (speculative decoding verify) extend metadata — extend_seq_lens, tree/chain masks — from forward_batch.spec_info (e.g. EagleVerifyInput). If spec_info is None in TARGET_VERIFY mode, the metadata cannot be built and this RuntimeError is raised from _build_extend_metadata during init_forward_metadata.","triggerScenarios":"A ForwardBatch arrives with forward_mode=TARGET_VERIFY but forward_batch.spec_info unset (None), while running the intel_amx backend's init_forward_metadata.","commonSituations":"Scheduler/speculative-decoding wiring bugs where the verify batch is constructed without attaching EagleVerifyInput/TLVerifyInput; a new speculative algorithm that forgets to populate spec_info; or mixed-version code paths constructing verify batches manually.","solutions":["Ensure the code building TARGET_VERIFY batches always sets forward_batch.spec_info (EagleVerifyInput etc.) before attention metadata init","Debug upstream: log where the verify ForwardBatch is created and why spec_info is missing (often a partial refactoring or new spec algorithm)","If using a custom speculative path, port the spec_info attachment logic from the eagle scheduler path"],"exampleFix":"# before\nforward_batch = ForwardBatch(..., forward_mode=ForwardMode.TARGET_VERIFY)  # spec_info None\n# after\nforward_batch = ForwardBatch(..., forward_mode=ForwardMode.TARGET_VERIFY,\n                             spec_info=eagle_verify_input)","handlingStrategy":"validation","validationCode":"if forward_batch.forward_mode.is_target_verify():\n    assert forward_batch.spec_info is not None, (\n        'TARGET_VERIFY batch missing spec_info (EagleVerifyInput/TLVerifyInput)')\nbackend.init_forward_metadata(forward_batch)","typeGuard":"def verify_batch_is_valid(forward_batch) -> bool:\n    if forward_batch.forward_mode.is_target_verify():\n        return forward_batch.spec_info is not None\n    return True","tryCatchPattern":"try:\n    backend.init_forward_metadata(forward_batch)\nexcept RuntimeError as e:\n    if 'spec_info is unset' in str(e):\n        log.error('verify batch built without spec_info at %s', batch_origin)\n        raise\n    raise","preventionTips":["Always attach spec_info when constructing TARGET_VERIFY ForwardBatches","Centralize verify-batch construction in the scheduler instead of ad-hoc sites","Add asserts in ForwardBatch post-init for mode/spec_info invariants"],"tags":["speculative-decoding","target-verify","spec-info","intel-amx","metadata","sglang"],"backgroundTag":"missing-required-batch-field","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}