sgl-project/sglang · critical · Exception

NIXL memory registration failed for state tensors

Error message

NIXL memory registration failed for state tensors

What it means

When a model carries state tensors (e.g. Mamba/linear-attention state buffers via state_data_ptrs/state_data_lens), register_buffer_to_engine registers them as VRAM (after filtering zero ptr/len entries). If state_addrs is non-empty but agent.register_memory returns a falsy descriptor, this Exception is raised: NIXL could not register the GPU state buffers.

Source

Thrown at python/sglang/srt/disaggregation/nixl/conn.py:1427

        state_addrs = []
        for comp_ptrs, comp_lens in zip(
            self.kv_args.state_data_ptrs or [],
            self.kv_args.state_data_lens or [],
        ):
            for state_data_ptr, state_data_len in zip(comp_ptrs, comp_lens):
                if state_data_ptr == 0 or state_data_len == 0:
                    continue
                state_addrs.append(
                    (state_data_ptr, state_data_len, self.kv_args.gpu_id, "")
                )
        if state_addrs:
            self.state_descs = self.agent.register_memory(state_addrs, "VRAM")
            logger.debug(
                f"Register state tensors, len(state_addrs)= {len(state_addrs)}"
            )
            if not self.state_descs:
                raise Exception("NIXL memory registration failed for state tensors")

    def _add_remote_peer(self, decode_kv_args: KVArgsRegisterInfo):
        agent_name = decode_kv_args.agent_name
        if agent_name in self.decode_kv_args_table:
            logger.info(f"Peer {agent_name} was already registered, ignoring.")
            return
        decode_kv_args.requires_dcp_relayout = self.requires_dcp_relayout(
            decode_kv_args.dst_dcp_size, decode_kv_args.dst_dcp_rank
        )
        self.decode_kv_args_table[agent_name] = decode_kv_args
        self.agent.add_remote_agent(decode_kv_args.agent_metadata)
        if self.disaggregation_mode == DisaggregationMode.PREFILL:
            self._prepare_payload_xfer(decode_kv_args)

    def _send_kvcache_generic(
        self,
        peer_name: str,
        src_data_ptrs: list[int],

View on GitHub (pinned to 0132848349)

Solutions

1. Verify UCX has CUDA memory registration support (ucx_info -d; check cuda_copy/rc transports) and matching CUDA versions
2. Update SGLang/nixl/ucx-py to a compatible set — hybrid-model state transfer support is actively evolving
3. Confirm state_data_ptrs are valid device pointers after model load (log them before registration)
4. If NIXL transfer for the hybrid model is unsupported in your versions, fall back to a non-disaggregated deployment or another transfer backend

state_addrs = [(p, l) for ptrs, lens in zip(kv_args.state_data_ptrs or [], kv_args.state_data_lens or []) for p, l in zip(ptrs, lens) if p != 0 and l != 0]
if state_addrs:
    assert agent.register_memory(state_addrs, 'VRAM'), 'state VRAM registration failed'

try:
    conn.register_buffer_to_engine()
except Exception as e:
    if 'state tensors' in str(e):
        check_ucx_cuda_support(); pin_compatible_nixl_ucx_versions()

When it happens

Trigger: Serving a hybrid/Mamba model with PD disaggregation where state buffers exist and their GPU registration fails — invalid state pointers, or UCX lacking CUDA memory registration support.

Common situations: NIXL PD transfer of hybrid linear-attention models with a ucx-py build lacking CUDA support, GPU memory/allocator issues, or version mismatch between nixl and the CUDA runtime.

Related errors

• NIXL memory registration failed for {mem_kind} kv tensors
• NIXL memory registration failed for aux tensors
• NIXL KV transfer has no KV memory segments
• PD state transfer failed: kv_args.state_types is empty but s
• PD state transfer failed: state component count mismatch (lo

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/27b80e030c520e05. Report an issue: GitHub.