{"record":{"id":"3240b402b0844988","repo":"sgl-project/sglang","slug":"weight-cache-where-quantization-method-quant","errorCode":null,"errorMessage":"[weight_cache:{where}] quantization method {quant_method!r} is not verified for CUDA IPC zero-copy weight sharing. Its process_weights_after_loading may stamp Python-side metadata (e.g. format_ue8m0) or repack/transpose weights into shapes the meta-initialized client cannot reproduce, which would silently serve wrong-numerics weights. Verified methods: {verified}. Note: FP8 is only verified for block-wise configs (weight_block_size set), not per-tensor FP8. Disable the weight cache (--weight-cache-mode off)","messagePattern":"\\[weight_cache:\\{where\\}\\] quantization method \\{quant_method!r\\} is not verified for CUDA IPC zero-copy weight sharing\\. Its process_weights_after_loading may stamp Python-side metadata \\(e\\.g\\. format_ue8m0\\) or repack/transpose weights into shapes the meta-initialized client cannot reproduce, which would silently serve wrong-numerics weights\\. Verified methods: \\{verified\\}\\. Note: FP8 is only verified for block-wise configs \\(weight_block_size set\\), not per-tensor FP8\\. Disable the weight cache \\(--weight-cache-mode off\\)","errorType":"validation","errorClass":"UnsupportedQuantForIPCError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/weight_cache/protocol.py","lineNumber":189,"sourceCode":"    if predicate is None:\n        return False\n    return bool(predicate(quant_config))\n\n\ndef check_ipc_quant_support(\n    quant_method: str, quant_config: Any, *, where: str\n) -> None:\n    \"\"\"Hard-error unless `quant_method` is verified safe for IPC zero-copy sharing.\n\n    `where` is a short tag (e.g. \"daemon\"/\"client\") used only in the error\n    message. Raises UnsupportedQuantForIPCError with an actionable message.\n    \"\"\"\n    if is_ipc_quant_supported(quant_method, quant_config):\n        return\n    verified = \", \".join(\n        (repr(m) if m else \"'' (unquantized)\") for m in IPC_QUANT_ALLOWLIST\n    )\n    raise UnsupportedQuantForIPCError(\n        f\"[weight_cache:{where}] quantization method {quant_method!r} is not \"\n        f\"verified for CUDA IPC zero-copy weight sharing. Its \"\n        f\"process_weights_after_loading may stamp Python-side metadata \"\n        f\"(e.g. format_ue8m0) or repack/transpose weights into shapes the \"\n        f\"meta-initialized client cannot reproduce, which would silently serve \"\n        f\"wrong-numerics weights. Verified methods: {verified}. Note: FP8 is \"\n        f\"only verified for block-wise configs (weight_block_size set), not \"\n        f\"per-tensor FP8. Disable the weight cache (--weight-cache-mode off) \"\n        f\"for this model.\"\n    )\n\n\n# ---------------------------------------------------------------------------\n# Socket protocol helpers\n# ---------------------------------------------------------------------------\n\n\nMAX_MSG_SIZE = 256 * 1024 * 1024  # 256 MiB","sourceCodeStart":171,"sourceCodeEnd":207,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/weight_cache/protocol.py#L171-L207","documentation":"Raised by check_ipc_quant_support when the model's quantization method is not in the IPC allowlist. Zero-copy CUDA IPC sharing is only verified for specific quants (block-wise FP8 etc.); unverified process_weights_after_loading may produce wrong-numerics weights in the meta-initialized client, so the loader refuses.","triggerScenarios":"Launching with --weight-cache-mode ipc (or on) using per-tensor FP8, unverified quant methods (e.g. some AWQ/GPTQ builds), or '' (unquantized) when not allowlisted; called from load/load_model.","commonSituations":"Enabling weight cache on a model checkpoint quantized with a method outside IPC_QUANT_ALLOWLIST; switching a model from block-wise FP8 to per-tensor FP8 without changing cache settings.","solutions":["Run with --weight-cache-mode off to load weights normally","Switch to a verified quant config (e.g. FP8 with weight_block_size set)","Request/verify support for the quant method upstream before enabling IPC sharing"],"exampleFix":"# before\nserver_args.weight_cache_mode = \"ipc\"\n# after\nserver_args.weight_cache_mode = \"off\"","handlingStrategy":"validation","validationCode":"from sglang.srt.weight_cache.protocol import is_ipc_quant_supported\nif weight_cache_on and not is_ipc_quant_supported(quant_method, quant_config):\n    server_args.weight_cache_mode = 'off'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check the allowlist before enabling ipc weight cache","Keep block-wise FP8 configs if you need cache sharing"],"tags":["quantization","weight-cache","ipc","config"],"backgroundTag":"unsupported-quantization-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}