{"record":{"id":"ec23b7de57adf409","repo":"sgl-project/sglang","slug":"comfy-nvfp4-is-inferred-from-per-layer-checkpoint","errorCode":null,"errorMessage":"comfy_nvfp4 is inferred from per-layer checkpoint metadata; it is not an online quantization method","messagePattern":"comfy_nvfp4 is inferred from per-layer checkpoint metadata; it is not an online quantization method","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/comfy_nvfp4.py","lineNumber":212,"sourceCode":"    @classmethod\n    def get_name(cls) -> str:\n        return \"comfy_nvfp4\"\n\n    @classmethod\n    def get_supported_act_dtypes(cls) -> list[torch.dtype]:\n        return [torch.bfloat16, torch.float16]\n\n    @classmethod\n    def get_min_capability(cls) -> int:\n        return 0\n\n    @classmethod\n    def get_config_filenames(cls) -> list[str]:\n        return []\n\n    @classmethod\n    def from_config(cls, config: dict[str, Any]) -> ComfyNvfp4Config:\n        raise ValueError(\n            \"comfy_nvfp4 is inferred from per-layer checkpoint metadata; \"\n            \"it is not an online quantization method\"\n        )\n\n    def get_quant_method(\n        self, layer: nn.Module, prefix: str\n    ) -> QuantizeMethodBase | None:\n        marker = self.layer_markers.get(prefix)\n        if isinstance(layer, VocabParallelEmbedding):\n            if marker is None:\n                return None\n            if marker.get(\"format\") != \"int8_tensorwise\" or not marker.get(\n                \"_is_rowwise\"\n            ):\n                raise ValueError(\n                    f\"Unsupported quantized embedding marker for {prefix!r}: {marker}\"\n                )\n            self.selected.append(prefix)","sourceCodeStart":194,"sourceCodeEnd":230,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/comfy_nvfp4.py#L194-L230","documentation":"ComfyNvfp4Config is derived from per-layer checkpoint metadata, not from a config file or online (on-the-fly) quantization. from_config deliberately raises to block using comfy_nvfp4 as an online quantization method name.","triggerScenarios":"Passing quantization=\"comfy_nvfp4\" on a CLI/config path that resolves it via from_config, or calling ComfyNvfp4Config.from_config(...) directly — i.e. trying to quantize an unquantized model online.","commonSituations":"Users adding --quantization comfy_nvfp4 expecting online nvfp4 quantization of a bf16 checkpoint; generic loaders that call from_config for all methods.","solutions":["Remove the explicit quantization setting and let the loader infer comfy_nvfp4 from the checkpoint's per-layer markers","Quantize the model offline with a tool that emits nvfp4 markers, then load it","Use fp8 or another supported online quantization method instead"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"from ...comfy_nvfp4 import ComfyNvfp4Config\nif ComfyNvfp4Config.get_config_filenames() == []:\n    # marker-inferred method; do not pass as an online quantization name","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Don't set --quantization comfy_nvfp4; let checkpoint markers drive inference","Use fp8 for online quantization instead"],"tags":["quantization","nvfp4","comfy","api-misuse"],"backgroundTag":"invalid-constructor-usage","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}