{"record":{"id":"8c979c7dc15e253f","repo":"sgl-project/sglang","slug":"comfy-full-precision-matrix-mult-does-not-support","errorCode":null,"errorMessage":"Comfy full_precision_matrix_mult does not support fused linears","messagePattern":"Comfy full_precision_matrix_mult does not support fused linears","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/quantization/comfy_fp8.py","lineNumber":45,"sourceCode":"    PerTensorScaleParameter,\n)\n\n\nclass ComfyFullPrecisionFp8LinearMethod(LinearMethodBase):\n    \"\"\"Keep FP8 storage but honor Comfy's full-precision matmul marker.\"\"\"\n\n    def create_weights(\n        self,\n        layer: nn.Module,\n        input_size_per_partition: int,\n        output_partition_sizes: list[int],\n        input_size: int,\n        output_size: int,\n        params_dtype: torch.dtype,\n        **extra_weight_attrs: Any,\n    ) -> None:\n        if len(output_partition_sizes) != 1:\n            raise ValueError(\n                \"Comfy full_precision_matrix_mult does not support fused linears\"\n            )\n        weight_loader = extra_weight_attrs.get(\"weight_loader\")\n        layer.logical_widths = output_partition_sizes\n        layer.input_size_per_partition = input_size_per_partition\n        layer.output_size_per_partition = output_partition_sizes[0]\n        layer.orig_dtype = params_dtype\n        weight = ModelWeightParameter(\n            data=torch.empty(\n                output_partition_sizes[0],\n                input_size_per_partition,\n                dtype=torch.float8_e4m3fn,\n            ),\n            input_dim=1,\n            output_dim=0,\n            weight_loader=weight_loader,\n        )\n        layer.register_parameter(\"weight\", weight)","sourceCodeStart":27,"sourceCodeEnd":63,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/quantization/comfy_fp8.py#L27-L63","documentation":"The Comfy full_precision_matrix_mult FP8 path allocates a single weight per layer, so it cannot handle fused linear layers where output_partition_sizes has more than one entry (multiple shards concatenated in one weight).","triggerScenarios":"create_weights on a comfy_fp8 layer with len(output_partition_sizes) > 1, e.g. QKV-fused projections built as merged parallel linears.","commonSituations":"Loading Comfy FP8 diffusion checkpoints whose attention projections are fused (qkv merged) while this quant method expects unfused per-layer weights.","solutions":["Construct the layers unfused (separate q/k/v linears) so each has one output partition","Use a different quant method that supports fused output partitions","Check the checkpoint layout: export with per-projection weights"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if len(output_partition_sizes) != 1:\n    raise SystemExit(\"comfy_fp8 requires unfused (single-output) linear layers\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Don't fuse projections in comfy_fp8 models"],"tags":["quantization","fp8","comfy","fused-layers"],"backgroundTag":"unsupported-layer-fusion","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}