sgl-project/sglang · error · NotImplementedError

Not support pos_emb_type: {pos_emb_type}

Error message

Not support pos_emb_type: {pos_emb_type}

What it means

The Kimi-K3 vision tower (kimi_k3_vl.py:301) only implements the 'divided_fixed' positional-embedding scheme (Learnable2DInterpPosEmbDividedFixed). Any other pos_emb_type string in the vision config is rejected at model construction. This is a config-surface guard: new checkpoint variants using a different pos-emb layout would need a new implementation.

Source

Thrown at python/sglang/srt/models/kimi_k3_vl.py:301

        pos_emb_type: str = "divided_fixed",
        pos_emb_interpolation_mode: str = "bicubic",
        patch_embed_proj_bias: bool = True,
    ):
        super().__init__()
        if isinstance(patch_size, int):
            patch_size = (patch_size, patch_size)
        self.patch_size = patch_size

        self.proj = nn.Conv2d(
            in_dim,
            out_dim,
            kernel_size=patch_size,
            stride=patch_size,
            bias=patch_embed_proj_bias,
        )

        if pos_emb_type != "divided_fixed":
            raise NotImplementedError(f"Not support pos_emb_type: {pos_emb_type}")
        self.pos_emb = Learnable2DInterpPosEmbDividedFixed(
            height=pos_emb_height,
            width=pos_emb_width,
            num_frames=pos_emb_time,
            dim=out_dim,
            interpolation_mode=pos_emb_interpolation_mode,
        )

    def forward(
        self,
        x: torch.Tensor,
        grid_thws: torch.Tensor,
        *,
        grid_thw_list: Optional[Sequence[Sequence[int]]] = None,
        position_embeddings: Optional[torch.Tensor] = None,
    ) -> torch.Tensor:
        # MIOpen can overflow grid_size for some patch shapes. Prefer AITER's
        # Triton convolution on AMD, with an equivalent linear fallback.

View on GitHub (pinned to 0132848349)

Solutions

1. Check the checkpoint's config.json vision pos_emb_type; the official weights use divided_fixed
2. Use the official Kimi-K3 vision config or a checkpoint revision known to work with this code
3. If you genuinely need the new type, implement a matching Learnable2DInterpPosEmb subclass and register it in the branch

assert vision_cfg["pos_emb_type"] == "divided_fixed", vision_cfg["pos_emb_type"]

When it happens

Trigger: Loading a Kimi-K3-VL checkpoint whose config.json vision section sets pos_emb_type to anything other than "divided_fixed" (e.g. a future "rope" or "absolute" variant).

Common situations: Using a new/finetuned community checkpoint that changed vision pos-emb, hand-edited config.json, or mixing configs from a different model revision.

Related errors

• Not support norm_type: {norm_type}
• Not support merge_type: {self.merge_type}
• Not support activation_func: {activation_func}
• Norm type {self.norm_type} not implemented
• Kimi K3 uses its model-native structural tag implementation

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c3946dbd5e7ce76b. Report an issue: GitHub.