sgl-project/sglang · error · NotImplementedError

Kimi K3 uses its model-native structural tag implementation

Error message

Kimi K3 uses its model-native structural tag implementation

What it means

KimiK3 detector's structure_info() is intentionally not implemented because Kimi K3 tool calls are enforced via model-native structural tags (get_auto_tool_call_structural_tag), not the generic StructureInfo constrained-generation path. Calling it is a misuse of this detector.

Source

Thrown at python/sglang/srt/function_call/kimik3_detector.py:89

    """

    def __init__(self):
        super().__init__()
        self.bot_token = TOOLS_OPEN
        self.eot_token = TOOLS_CLOSE
        self._sent_normal_idx = 0

    def has_tool_call(self, text: str) -> bool:
        return self.bot_token in text

    def supports_structural_tag(self) -> bool:
        return True

    def parses_required_natively(self) -> bool:
        return False

    def structure_info(self) -> _GetInfoFunc:
        raise NotImplementedError(
            "Kimi K3 uses its model-native structural tag implementation"
        )

    def get_auto_tool_call_structural_tag(
        self,
        tools: Union[List[Tool], None] = None,
        thinking_mode: bool = False,
        parallel_tool_calls: bool = True,
    ) -> Optional[StructuralTag]:
        return get_kimik3_auto_tool_call_structural_tag(
            tools or [],
            thinking_mode=thinking_mode,
            parallel_tool_calls=parallel_tool_calls,
        )

    def get_structural_tag(
        self,
        tools: Union[List[Tool], None] = None,

View on GitHub (pinned to 0132848349)

Solutions

1. Use get_auto_tool_call_structural_tag(...) instead, which returns the structural tag format Kimi K3 actually uses
2. Skip KimiK3 when calling structure_info across parsers
3. Route JSON-schema constraints through the constraint backend directly

Example fix

// before
fmt = detector.structure_info()
// after
fmt = detector.get_auto_tool_call_structural_tag(tools=tools)

def get_constraint_fn(detector):
    if hasattr(detector, "get_auto_tool_call_structural_tag"):
        return None  # use structural tags instead
    return detector.structure_info()

When it happens

Trigger: Invoking structure_info() on the Kimi K3 detector, e.g. generic code that assumes all FunctionCallParser subclasses expose StructureInfo.

Common situations: Adding a new function-call feature that iterates parsers; mixing the KimiK3 parser with logic built for StructureInfo-based parsers.

Related errors

• structure_info not used for JSON schema constraints
• Not support pos_emb_type: {pos_emb_type}
• Not support norm_type: {norm_type}
• Not support merge_type: {self.merge_type}
• /v1/models ${response.status}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/284db18d7129da1a. Report an issue: GitHub.