{"record":{"id":"b60927ce173608ea","repo":"sgl-project/sglang","slug":"only-per-tensor-fp8-scales-are-supported-for-diffu","errorCode":null,"errorMessage":"Only per-tensor FP8 scales are supported for diffusion checkpoints, got shape {tuple(scale.shape)}.","messagePattern":"Only per-tensor FP8 scales are supported for diffusion checkpoints, got shape (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/tools/build_modelopt_fp8_transformer.py","lineNumber":506,"sourceCode":"            )\n\n    return {\n        weight_name: scale_tensors\n        for weight_name, scale_tensors in scale_map.items()\n        if {\"weight_scale\", \"input_scale\"} <= set(scale_tensors)\n    }\n\n\ndef quantize_fp8_weight(\n    weight: torch.Tensor,\n    weight_scale: torch.Tensor,\n) -> torch.Tensor:\n    if weight.dtype == torch.float8_e4m3fn:\n        return weight.contiguous()\n\n    scale = weight_scale.to(weight.device, dtype=torch.float32)\n    if scale.numel() != 1:\n        raise ValueError(\n            \"Only per-tensor FP8 scales are supported for diffusion checkpoints, \"\n            f\"got shape {tuple(scale.shape)}.\"\n        )\n\n    quantized = (weight.to(torch.float32) / scale.reshape(1)).to(torch.float8_e4m3fn)\n    return quantized.cpu().contiguous()\n\n\ndef _copy_non_shard_files(source_dir: str, output_dir: str) -> None:\n    ignored = set(INDEX_FILENAMES)\n    for entry in os.listdir(source_dir):\n        if entry.endswith(\".safetensors\") or entry in ignored:\n            continue\n        source_path = os.path.join(source_dir, entry)\n        output_path = os.path.join(output_dir, entry)\n        if os.path.isdir(source_path):\n            shutil.copytree(source_path, output_path, dirs_exist_ok=True)\n        else:","sourceCodeStart":488,"sourceCodeEnd":524,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/tools/build_modelopt_fp8_transformer.py#L488-L524","documentation":"quantize_fp8_weight only supports per-tensor (scalar, numel==1) scales for diffusion checkpoints; a scale tensor with more than one element (per-channel/per-block) is rejected.","triggerScenarios":"Feeding a ModelOpt export whose weight_scale tensors are per-channel (e.g. shape [N]) or per-block instead of scalar.","commonSituations":"Exporting with a newer ModelOpt version that defaults to per-channel quantization; hand-modified checkpoints converting scales to vectors.","solutions":["Re-export from ModelOpt with per-tensor (scalar) quantization for weights","If conversion happens upstream in your pipeline, squeeze/reduce scales to per-tensor before invoking the tool","Verify the export's quantization_config specifies per-tensor scales"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if weight_scale.numel() != 1:\n    raise SystemExit(f\"non-scalar scale {tuple(weight_scale.shape)}; re-export per-tensor\")","typeGuard":null,"tryCatchPattern":"try:\n    q = quantize_fp8_weight(w, s)\nexcept ValueError as e:\n    logger.error(\"%s\", e)\n    raise","preventionTips":["Export ModelOpt checkpoints with per-tensor weight quantization","Validate scale shapes in any pre-processing/conversion step"],"tags":["modelopt","fp8","quantization"],"backgroundTag":"unsupported-quantization-scheme","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}