{"record":{"id":"74cc6958a3af28de","repo":"sgl-project/sglang","slug":"comfy-int8-embedding-weights-support-lookup-only","errorCode":null,"errorMessage":"Comfy INT8 embedding weights support lookup only","messagePattern":"Comfy INT8 embedding weights support lookup only","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/comfy_nvfp4.py","lineNumber":86,"sourceCode":"            ),\n            extra_weight_attrs,\n            {\"input_dim\": 1, \"output_dim\": 0},\n        )\n        _register_parameter(\n            layer,\n            \"weight_scale\",\n            torch.empty(output_size_per_partition, 1, dtype=torch.float32),\n            extra_weight_attrs,\n            {\"output_dim\": 0},\n        )\n\n    def apply(\n        self,\n        layer: nn.Module,\n        x: torch.Tensor,\n        bias: torch.Tensor | None = None,\n    ) -> torch.Tensor:\n        raise NotImplementedError(\"Comfy INT8 embedding weights support lookup only\")\n\n    def embedding(self, layer: nn.Module, input_: torch.Tensor) -> torch.Tensor:\n        weight = F.embedding(input_, layer.weight).to(self.output_dtype)\n        scale = F.embedding(input_, layer.weight_scale).to(self.output_dtype)\n        return weight * scale\n\n\nclass ComfyFullPrecisionNvfp4LinearMethod(ModelOptFp4LinearMethod):\n    \"\"\"Keep NVFP4 storage and dequantize one active Linear for its matmul.\"\"\"\n\n    def __init__(\n        self,\n        quant_config: ComfyNvfp4Config,\n        *,\n        has_pre_quant_scale: bool,\n    ) -> None:\n        self.quant_config = quant_config\n        self.has_pre_quant_scale = has_pre_quant_scale","sourceCodeStart":68,"sourceCodeEnd":104,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/comfy_nvfp4.py#L68-L104","documentation":"Comfy INT8 embedding quantization stores weights as int8 codes plus scales; embeddings must be looked up and dequantized via the embedding() path. Calling apply() (the matmul interface) on such a layer is not implemented.","triggerScenarios":"Invoking apply(layer, x) on the int8 embedding quant method instead of calling layer via its embedding path (e.g. routing an embedding layer through a linear-style forward).","commonSituations":"Generic code that calls apply() on every quant method regardless of layer type; passing token indices to a path expecting hidden states.","solutions":["Call the embedding path (embedding(layer, input_ids)) instead of apply()","Route embedding layers to lookup, not matmul, in your dispatch logic"],"exampleFix":"// before\nout = quant_method.apply(layer, input_ids)\n\n// after\nout = quant_method.embedding(layer, input_ids)","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"def is_embedding_method(m) -> bool:\n    return hasattr(m, \"embedding\") and not hasattr(m, \"apply_matmul\")","tryCatchPattern":null,"preventionTips":["Dispatch embedding layers to embedding(), linear layers to apply()"],"tags":["quantization","int8","embedding","not-implemented"],"backgroundTag":"unsupported-operation-for-layer-type","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}