{"record":{"id":"990068620d28c80b","repo":"vllm-project/vllm","slug":"vllm-rocm-quick-reduce-min-size-bytes-mb-must-be-l","errorCode":null,"errorMessage":"VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be less than or equal to the effective QuickReduce max size","messagePattern":"VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be less than or equal to the effective QuickReduce max size","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/device_communicators/quick_all_reduce.py","lineNumber":273,"sourceCode":"                \"Custom quick allreduce: quantization codec threshold = %d KB\",\n                self.qr_quantization_min_size // KB,\n            )\n        self.create_shared_buffer()\n        self.disabled = False\n\n    @staticmethod\n    def _get_qr_min_size(qr_max_size: int | None) -> int | None:\n        qr_min_size = envs.VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB\n        if qr_min_size is None:\n            return None\n        if qr_min_size < 0:\n            raise ValueError(\n                \"VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be non-negative, \"\n                f\"got {qr_min_size}\"\n            )\n        qr_min_size *= MB\n        if qr_max_size is not None and qr_min_size > qr_max_size:\n            raise ValueError(\n                \"VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be less than or \"\n                \"equal to the effective QuickReduce max size\"\n            )\n        return qr_min_size\n\n    @staticmethod\n    def _get_qr_quantization_min_size() -> int | None:\n        quantization_min_size = envs.VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB\n        if quantization_min_size is None:\n            return None\n        if quantization_min_size < 0:\n            raise ValueError(\n                \"VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB must be \"\n                f\"non-negative, got {quantization_min_size}\"\n            )\n        return quantization_min_size * KB\n\n    def _rocm_arch_available(self):","sourceCodeStart":255,"sourceCodeEnd":291,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/device_communicators/quick_all_reduce.py#L255-L291","documentation":"After scaling VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB to bytes, _get_qr_min_size checks it against the effective QuickReduce max buffer size (qr_max_size, derived from the engine's buffer budget). A minimum threshold larger than the maximum usable buffer is contradictory, so it raises ValueError.","triggerScenarios":"Setting VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB to a value (in MiB) whose byte size exceeds the configured QuickReduce max buffer size, e.g. min=512MB while max is 256MB.","commonSituations":"Tuning quick-reduce thresholds without checking the max buffer budget; lowering the max buffer for memory reasons while leaving a large min; copying tuning values between machines with different buffer configs.","solutions":["Lower VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB to at most the effective max size","Or raise the QuickReduce max buffer size so it covers the desired minimum","Or unset the min-size variable to let the default thresholding apply"],"exampleFix":"# before\nexport VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB=512  # > max buffer 256MB\n\n# after\nexport VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB=128  # <= max buffer","handlingStrategy":"validation","validationCode":"v = os.environ.get(\"VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB\")\nif v is not None:\n    assert int(v) * (1 << 20) <= qr_max_size, \"min size (MiB->bytes) must not exceed QuickReduce max buffer\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive thresholds from a single config so min/max cannot diverge","Document the max buffer size next to the env var","Validate the pair (min, max) in a startup check"],"tags":["rocm","allreduce","env-var","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}