sgl-project/sglang · error · NotImplementedError

MLX auxiliary-state radix cache does not support enable_mamb

Error message

MLX auxiliary-state radix cache does not support enable_mamba_extra_buffer yet.

What it means

`MlxAuxiliaryStateComponent` is the MLX-native radix-cache component for Mamba-style auxiliary state snapshots. The optional `enable_mamba_extra_buffer` feature (extra buffering of mamba states for scheduling) has not been implemented for the MLX path, so __init__ raises NotImplementedError rather than silently misbehaving.

Source

Thrown at python/sglang/srt/hardware_backend/mlx/kv_cache/auxiliary_state.py:334

            # auxiliary slot, so return it with the request row. Keyed on
            # req.mamba_pool_idx (None-safe, nulled by free_mamba_cache), NOT
            # on req_index_to_auxiliary_state_index_mapping, which may point
            # at a slot the radix tree owns.
            self.free_mamba_cache(req)
        super().free(req)

    def clear(self):
        super().clear()
        self.auxiliary_state_pool.clear()
        self.req_index_to_auxiliary_state_index_mapping.zero_()


class MlxAuxiliaryStateComponent(MambaComponent):
    """Unified radix component for MLX native auxiliary-state snapshots."""

    def __init__(self, cache, params):
        if params.enable_mamba_extra_buffer:
            raise NotImplementedError(
                "MLX auxiliary-state radix cache does not support "
                "enable_mamba_extra_buffer yet."
            )
        pool = getattr(cache.req_to_token_pool, "auxiliary_state_pool", None)
        if not isinstance(pool, MlxAuxiliaryStatePool):
            raise TypeError(
                "MlxAuxiliaryStateComponent requires MlxAuxiliaryStatePool, "
                f"got {type(pool)}"
            )
        TreeComponent.__init__(self, cache, params)
        self.enable_mamba_extra_buffer = False
        self._mamba_pool_host = None

    @staticmethod
    def _tracked_value(req) -> tuple[object | None, bool]:
        track_buffer = getattr(req, "mamba_ping_pong_track_buffer", None)
        track_len = getattr(req, "mamba_last_track_seqlen", None)
        if track_buffer is not None and track_len is not None:

View on GitHub (pinned to 0132848349)

Solutions

1. Disable `enable_mamba_extra_buffer` for MLX runs.
2. Keep backend-specific server args separated per deployment instead of one shared config.
3. Track the SGLang MLX backend roadmap and re-enable once implemented.

Example fix

# before
server_args = ServerArgs(..., enable_mamba_extra_buffer=True)  # MLX backend

# after
server_args = ServerArgs(..., enable_mamba_extra_buffer=False)

if server_args.enable_mamba_extra_buffer and backend == "mlx":
    server_args.enable_mamba_extra_buffer = False  # not supported on MLX

When it happens

Trigger: Constructing the MLX auxiliary-state radix component with server args / params where `enable_mamba_extra_buffer=True` — e.g. enabling the flag in server_args for a hybrid attention+Mamba model running on the MLX backend.

Common situations: Copying CPU/GPU-server flags (`--enable-mamba-extra-buffer`) to an Apple-Silicon MLX deployment; a config shared across backends that turns the flag on globally.

Related errors

• MLX async runner does not support forward mode: {forward_mod
• --mamba-max-states-per-path must be -1 (unlimited) or a posi
• Stochastic rounding for the Mamba SSM cache requires --mamba
• Stochastic rounding for the Mamba SSM cache is only supporte
• --enable-int8-mamba-checkpoint is not supported together wit

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/32c2ad259859fe22. Report an issue: GitHub.