{"record":{"id":"b42357aa049434ff","repo":"sgl-project/sglang","slug":"unsupported-comfy-int8-format-for-prefix-r-mar","errorCode":null,"errorMessage":"Unsupported Comfy INT8 format for {prefix!r}: {marker.get('format')!r}","messagePattern":"Unsupported Comfy INT8 format for (.+?): (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_int8_config.py","lineNumber":49,"sourceCode":"        layer_markers: dict[str, dict[str, Any]] | None = None,\n    ) -> None:\n        super().__init__()\n        if group_size not in _SUPPORTED_GROUP_SIZES:\n            raise ValueError(\n                f\"kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZES}, \"\n                f\"got {group_size}\"\n            )\n        self.group_size = group_size\n        self.ignored_layers = ignored_layers or []\n        self.packed_modules_mapping = packed_modules_mapping or {}\n        self.layer_markers = layer_markers\n        self.is_checkpoint_int8_serialized = layer_markers is not None\n        self.checkpoint_uses_native_qkv_layout = self.is_checkpoint_int8_serialized\n        self._serialized_group_sizes: dict[str, int] = {}\n        if layer_markers is not None:\n            for prefix, marker in layer_markers.items():\n                if marker.get(\"format\") != \"int8_tensorwise\":\n                    raise ValueError(\n                        f\"Unsupported Comfy INT8 format for {prefix!r}: \"\n                        f\"{marker.get('format')!r}\"\n                    )\n                if marker.get(\"convrot\") is not True:\n                    raise ValueError(\n                        f\"Serialized kitchen_int8 layer {prefix!r} must set \"\n                        \"convrot=true\"\n                    )\n                marker_group_size = marker.get(\"convrot_groupsize\")\n                if marker_group_size not in _SUPPORTED_GROUP_SIZES:\n                    raise ValueError(\n                        f\"Serialized kitchen_int8 layer {prefix!r} must declare \"\n                        f\"convrot_groupsize in {_SUPPORTED_GROUP_SIZES}, got \"\n                        f\"{marker_group_size!r}\"\n                    )\n                self._serialized_group_sizes[prefix] = marker_group_size\n        # Which layers actually got quantized is worth stating plainly in the\n        # log: a silent fallback to BF16 looks exactly like a slow kernel.","sourceCodeStart":31,"sourceCodeEnd":67,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/configs/kitchen_int8_config.py#L31-L67","documentation":"When KitchenInt8Config is built with layer_markers, every serialized marker must have format 'int8_tensorwise'. A marker with any other format means the checkpoint mixes schemes the kitchen_int8 path cannot dispatch, so construction aborts.","triggerScenarios":"Passing layer_markers containing an entry whose 'format' key is not 'int8_tensorwise' (e.g. 'nvfp4' or 'convrot_w4a4').","commonSituations":"Exporting a Comfy model with mixed per-layer quantization and loading it under kitchen_int8; concatenating marker dicts from different exports.","solutions":["Re-export the checkpoint with all-int8_tensorwise quantization","Split mixed markers and load each subset under its matching quant config (e.g. kitchen_w4a4 handles convrot_w4a4)"],"exampleFix":"// before\nlayer_markers = {\"blk.0\": {\"format\": \"convrot_w4a4\", \"convrot\": True}}\n// after\nlayer_markers = {\"blk.0\": {\"format\": \"int8_tensorwise\", \"convrot\": True, \"convrot_groupsize\": 128}}","handlingStrategy":"validation","validationCode":"for prefix, m in layer_markers.items():\n    assert m.get(\"format\") == \"int8_tensorwise\", (prefix, m.get(\"format\"))","typeGuard":"def is_int8_tensorwise_marker(m: dict) -> bool:\n    return m.get(\"format\") == \"int8_tensorwise\"","tryCatchPattern":null,"preventionTips":["Split mixed checkpoints by format and load each under its matching config"],"tags":["quantization","checkpoint","config-validation"],"backgroundTag":"unsupported-quantization-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}