sgl-project/sglang · error · ValueError

Comfy full_precision_matrix_mult does not support fused line

Error message

Comfy full_precision_matrix_mult does not support fused linears

What it means

The Comfy full_precision_matrix_mult FP8 path allocates a single weight per layer, so it cannot handle fused linear layers where output_partition_sizes has more than one entry (multiple shards concatenated in one weight).

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/comfy_fp8.py:45

    PerTensorScaleParameter,
)


class ComfyFullPrecisionFp8LinearMethod(LinearMethodBase):
    """Keep FP8 storage but honor Comfy's full-precision matmul marker."""

    def create_weights(
        self,
        layer: nn.Module,
        input_size_per_partition: int,
        output_partition_sizes: list[int],
        input_size: int,
        output_size: int,
        params_dtype: torch.dtype,
        **extra_weight_attrs: Any,
    ) -> None:
        if len(output_partition_sizes) != 1:
            raise ValueError(
                "Comfy full_precision_matrix_mult does not support fused linears"
            )
        weight_loader = extra_weight_attrs.get("weight_loader")
        layer.logical_widths = output_partition_sizes
        layer.input_size_per_partition = input_size_per_partition
        layer.output_size_per_partition = output_partition_sizes[0]
        layer.orig_dtype = params_dtype
        weight = ModelWeightParameter(
            data=torch.empty(
                output_partition_sizes[0],
                input_size_per_partition,
                dtype=torch.float8_e4m3fn,
            ),
            input_dim=1,
            output_dim=0,
            weight_loader=weight_loader,
        )
        layer.register_parameter("weight", weight)

View on GitHub (pinned to 0132848349)

Solutions

1. Construct the layers unfused (separate q/k/v linears) so each has one output partition
2. Use a different quant method that supports fused output partitions
3. Check the checkpoint layout: export with per-projection weights

if len(output_partition_sizes) != 1:
    raise SystemExit("comfy_fp8 requires unfused (single-output) linear layers")

When it happens

Trigger: create_weights on a comfy_fp8 layer with len(output_partition_sizes) > 1, e.g. QKV-fused projections built as merged parallel linears.

Common situations: Loading Comfy FP8 diffusion checkpoints whose attention projections are fused (qkv merged) while this quant method expects unfused per-layer weights.

Related errors

• Unsupported Comfy FP8 layer formats: {unsupported}
• ComfyFp8Config must be constructed from safetensors layer ma
• Comfy full_precision_matrix_mult does not support fused line
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp
• int32-packed scale buffers require scale_ue8m0=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8c979c7dc15e253f. Report an issue: GitHub.