{"record":{"id":"92e4c4e5c0cf7ff8","repo":"sgl-project/sglang","slug":"inklingnvfp4moemethod-is-the-dense-shared-expert-m","errorCode":null,"errorMessage":"InklingNvfp4MoEMethod is the dense shared-expert method; routed NVFP4 MoE uses ModelOptNvFp4FusedMoEMethod.","messagePattern":"InklingNvfp4MoEMethod is the dense shared-expert method; routed NVFP4 MoE uses ModelOptNvFp4FusedMoEMethod\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/quantization/quant.py","lineNumber":162,"sourceCode":"        \"\"\"Process weights for the dense shared-expert NVFP4 path.\n\n        Routed NVFP4 MoE now uses ModelOptNvFp4FusedMoEMethod; this hook is reached\n        only for shared experts (InklingBatchDenseMLP), which carry an ``_fp4_strategy``\n        and run their own weight preparation.\n        \"\"\"\n        if getattr(layer, \"_fp4_strategy\", None) is not None:\n            layer.process_weights_after_loading()\n\n    def apply(\n        self,\n        layer: torch.nn.Module,\n        dispatch_output,  # type: ignore[override]\n    ):\n        # Kept only to satisfy the FusedMoEMethodBase abstract interface.\n        # InklingNvfp4MoEMethod serves the dense shared-expert path (InklingBatchDenseMLP,\n        # which uses its own FP4 serving); routed NVFP4 MoE uses\n        # ModelOptNvFp4FusedMoEMethod.\n        raise NotImplementedError(\n            \"InklingNvfp4MoEMethod is the dense shared-expert method; routed NVFP4 \"\n            \"MoE uses ModelOptNvFp4FusedMoEMethod.\"\n        )\n","sourceCodeStart":144,"sourceCodeEnd":166,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/quantization/quant.py#L144-L166","documentation":"InklingNvfp4MoEMethod.apply unconditionally raises NotImplementedError; the method exists only to satisfy the FusedMoEMethodBase abstract interface. Routed NVFP4 MoE layers are served by ModelOptNvFp4FusedMoEMethod, and Inkling's dense shared-expert MLP uses its own FP4 path.","triggerScenarios":"A FusedMoE layer being routed to InklingNvfp4MoEMethod.apply — i.e. the quant method was attached to a routed MoE weight instead of the dense shared-expert path, or generic code invoked apply().","commonSituations":"Wiring change that registers InklingNvfp4MoEMethod for routed experts; calling the method directly in tests or custom loaders.","solutions":["Ensure routed NVFP4 MoE layers use ModelOptNvFp4FusedMoEMethod","Keep InklingNvfp4MoEMethod attached only to the dense shared-expert MLP (InklingBatchDenseMLP)","Do not call .apply() on this method; if you hit it, audit the method-selection logic in the quant registry"],"exampleFix":"# before\nmethod = InklingNvfp4MoEMethod(...)\nmethod.apply(layer, output)  # raises\n# after\nfrom sglang.srt.layers.quantization.modelopt import ModelOptNvFp4FusedMoEMethod\nmethod = ModelOptNvFp4FusedMoEMethod(...)","handlingStrategy":"type-guard","validationCode":"assert type(method) is not InklingNvfp4MoEMethod or layer.is_dense_shared_expert","typeGuard":"def is_routed_moe_method(m) -> bool:\n    return not isinstance(m, InklingNvfp4MoEMethod)","tryCatchPattern":null,"preventionTips":["Never call .apply on InklingNvfp4MoEMethod; route via the quant registry"],"tags":["quantization","nvfp4","moe","not-implemented","inkling"],"backgroundTag":"abstract-method-not-implemented","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}