{"record":{"id":"21b9680ffefa5f89","repo":"sgl-project/sglang","slug":"failed-to-set-up-modelopt-quantization-e","errorCode":null,"errorMessage":"Failed to set up ModelOpt quantization: {e}","messagePattern":"Failed to set up ModelOpt quantization: (.+?)","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/model_loader/loader.py","lineNumber":3827,"sourceCode":"\n            if not model_parallel_is_initialized() or get_parallel().tp_rank == 0:\n                mtq.print_quant_summary(model)\n\n            # Save checkpoint if path provided\n            if quantized_ckpt_save_path:\n                try:\n                    mto.save(model, quantized_ckpt_save_path)\n                    rank0_log(f\"Quantized model saved to {quantized_ckpt_save_path}\")\n                except Exception as e:\n                    logger.warning(\n                        f\"Failed to save quantized checkpoint to {quantized_ckpt_save_path}: {e}\"\n                    )\n\n            # Export model if path provided\n            self._maybe_export_modelopt(model, export_path)\n\n        except Exception as e:\n            raise Exception(f\"Failed to set up ModelOpt quantization: {e}\") from e\n\n    def _maybe_export_modelopt(self, model, export_path: str | None) -> None:\n        \"\"\"Export model to HuggingFace format if export_path is provided.\"\"\"\n        if export_path:\n            try:\n                # Get the original model path from the model config\n                original_model_path = getattr(self, \"_original_model_path\", None)\n                self._export_modelopt_checkpoint(\n                    model, export_path, original_model_path\n                )\n                rank0_log(\n                    f\"Quantized model exported to HuggingFace format at {export_path}\"\n                )\n            except Exception as e:\n                rank0_log(\n                    f\"Warning: Failed to export quantized model to {export_path}: {e}\"\n                )\n","sourceCodeStart":3809,"sourceCodeEnd":3845,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/model_loader/loader.py#L3809-L3845","documentation":"This is a catch-all wrapper: any exception raised while setting up ModelOpt quantization (config resolution, mtq.quantize, calibration loop, or export) inside the ModelOpt loader is re-raised as Exception('Failed to set up ModelOpt quantization: {e}'). The root cause is the chained original exception `e` — always inspect `__cause__`/the log line above it.","triggerScenarios":"Calling the ModelOpt loader with an invalid quantization config, a model architecture unsupported by modelopt, calibration data issues, OOM during mtq.quantize, or failures inside _maybe_export_modelopt — any exception in the try block triggers it.","commonSituations":"Mismatched modelopt vs torch/transformers versions; quantizing a custom model class modelopt doesn't recognize; GPU OOM during FP8/FP4 calibration; export path permission errors; corrupt checkpoint.","solutions":["Read the chained exception (`raise ... from e`) — the original traceback names the real failure; fix that first","Check version compatibility: nvidia-modelopt nightly vs sglang requirements; upgrade/downgrade modelopt","Reduce batch size / calibration samples if the inner error is CUDA OOM","For export errors, verify export_path is writable and on a volume with enough space","If the inner error is unsupported architecture, quantize a supported model or update modelopt"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    loader.load_model(...)\nexcept Exception as e:\n    root = e.__cause__ or e\n    log.error(\"ModelOpt setup failed, root cause: %s\", root)\n    raise","preventionTips":["Always log the chained __cause__, not just the wrapper message","Validate quantization args and model architecture support before starting a long quantization run","Smoke-test modelopt on a tiny model before production quantization"],"tags":["modelopt","quantization","wrapper-exception","chained-exception","sglang"],"backgroundTag":"quantization-setup-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}