{"record":{"id":"90b05e2a003347c5","repo":"sgl-project/sglang","slug":"nvfp4-embedding-is-gather-only-reaching-here-mean","errorCode":null,"errorMessage":"NVFP4 embedding is gather-only. Reaching here means a tied lm_head is sharing this module; exclude the embedding from NVFP4 in the quantization recipe to serve such a checkpoint.","messagePattern":"NVFP4 embedding is gather-only\\. Reaching here means a tied lm_head is sharing this module; exclude the embedding from NVFP4 in the quantization recipe to serve such a checkpoint\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/modelopt_quant.py","lineNumber":742,"sourceCode":"        )\n        set_weight_attrs(\n            weight_scale_2,\n            {\"weight_loader\": lambda p, w: p.data.copy_(w.reshape(p.shape).float())},\n        )\n        layer.register_parameter(\"weight_scale_2\", weight_scale_2)\n\n        # A buffer; CUDA graph capture rejects host->device copies.\n        layer.register_buffer(\n            \"e2m1_lut\",\n            torch.tensor(_E2M1_LUT, dtype=torch.float32),\n            persistent=False,\n        )\n\n    def process_weights_after_loading(self, layer: torch.nn.Module) -> None:\n        pass\n\n    def apply(self, *args, **kwargs):\n        raise NotImplementedError(\n            \"NVFP4 embedding is gather-only. Reaching here means a tied lm_head \"\n            \"is sharing this module; exclude the embedding from NVFP4 in the \"\n            \"quantization recipe to serve such a checkpoint.\"\n        )\n\n    def embedding(self, layer: torch.nn.Module, input_: torch.Tensor) -> torch.Tensor:\n        index_shape = input_.shape\n        flat = input_.reshape(-1)\n        packed = layer.weight[flat]  # [T, H/2] uint8\n        scale = layer.weight_scale[flat]  # [T, H/16] e4m3\n        rows, half = packed.shape\n        hidden = half * 2\n\n        codes = packed.new_empty((rows, hidden))\n        codes[:, 0::2] = packed & 0x0F\n        codes[:, 1::2] = packed >> 4\n\n        mag = layer.e2m1_lut[(codes & 0x7).long()]","sourceCodeStart":724,"sourceCodeEnd":760,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/modelopt_quant.py#L724-L760","documentation":"Error \"NVFP4 embedding is gather-only. Reaching here means a tied lm_head is sharing this module; exclude the embedding from NVFP4 in the quantization recipe to serve such a checkpoint.\" thrown in sgl-project/sglang.","triggerScenarios":"Thrown at python/sglang/srt/layers/quantization/modelopt_quant.py:742 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":[],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}