sgl-project/sglang · error · ValueError

MiniCPM fused top-k only supports bfloat16 and float16, got

Error message

MiniCPM fused top-k only supports bfloat16 and float16, got {self.model_dtype}.

What it means

Error "MiniCPM fused top-k only supports bfloat16 and float16, got {self.model_dtype}." thrown in sgl-project/sglang.

Source

Thrown at python/sglang/srt/layers/attention/minicpm/backend.py:227

        self.head_dim = model_runner.model_config.head_dim
        self.head_group_num = self.num_kv_heads
        self.heads_per_group = self.num_q_heads // self.head_group_num
        if self.heads_per_group != 16:
            raise ValueError(
                "MiniCPM sparse attention requires 16 query heads per KV head, "
                f"got {self.heads_per_group}."
            )
        self.k1_kernel_size = self.kernel_size
        self.k1_kernel_stride = self.kernel_stride
        self.k2_kernel_size = self.kernel_size * 4
        self.k2_kernel_stride = self.kernel_stride * 4

        self.minicpm_fuse_topk = (
            use_blackwell and use_flashinfer
        ) or envs.SGLANG_MINICPM_FUSE_TOPK.get()
        dtype_str = str(self.model_dtype).removeprefix("torch.")
        if self.minicpm_fuse_topk and dtype_str not in ("bfloat16", "float16"):
            raise ValueError(
                "MiniCPM fused top-k only supports bfloat16 and float16, "
                f"got {self.model_dtype}."
            )

        max_cache_len = self.max_context_len
        pooled_k_len = (max_cache_len + self.block_size - 1) // self.block_size

        output_topk = min(self.sparse_topk, pooled_k_len)

        # For the kernel, we need power of 2 topk
        topk_power2 = next_power_of_2(output_topk)
        kernel_topk = min(topk_power2, pooled_k_len)
        # Make sure it's still power of 2
        if kernel_topk != next_power_of_2(kernel_topk):
            kernel_topk = next_power_of_2(kernel_topk) // 2
        kernel_topk = max(8, kernel_topk)
        self.kernel_topk = kernel_topk
        self.decode_fused_kernels = {}

View on GitHub (pinned to 0132848349)

When it happens

Trigger: Thrown at python/sglang/srt/layers/attention/minicpm/backend.py:227 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0cdae1f35f02bbc1. Report an issue: GitHub.