vllm-project/vllm · error · ValueError

VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB must be non-

Error message

VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB must be non-negative, got {quantization_min_size}

What it means

CustomQuickAllReduce._get_qr_quantization_min_size validates VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB: negative values raise ValueError. The variable sets the KB threshold above which messages are quantized on the quick allreduce path.

Source

Thrown at vllm/distributed/device_communicators/quick_all_reduce.py:285

            raise ValueError(
                "VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be non-negative, "
                f"got {qr_min_size}"
            )
        qr_min_size *= MB
        if qr_max_size is not None and qr_min_size > qr_max_size:
            raise ValueError(
                "VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be less than or "
                "equal to the effective QuickReduce max size"
            )
        return qr_min_size

    @staticmethod
    def _get_qr_quantization_min_size() -> int | None:
        quantization_min_size = envs.VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB
        if quantization_min_size is None:
            return None
        if quantization_min_size < 0:
            raise ValueError(
                "VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB must be "
                f"non-negative, got {quantization_min_size}"
            )
        return quantization_min_size * KB

    def _rocm_arch_available(self):
        if not current_platform.is_rocm():
            return False
        try:
            props = torch.cuda.get_device_properties(0)
            gcn_arch = getattr(props, "gcnArchName", "")
            supported_archs = ["gfx94", "gfx95"]
            return any(gfx in gcn_arch for gfx in supported_archs)
        except Exception as e:
            logger.warning("Failed to determine ROCm for quick allreduce: %s", e)
            return False

    def create_shared_buffer(self):

View on GitHub (pinned to c794754062)

Solutions

1. Set the variable to a non-negative KB value or unset it
2. Use 0 if the intent was 'quantize everything' rather than a negative sentinel

Example fix

# before
export VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB=-16

# after
unset VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB
# or
export VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB=16

v = os.environ.get("VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB")
if v is not None:
    assert int(v) >= 0, "VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB must be >= 0"

When it happens

Trigger: Launching vLLM on a supported ROCm platform with VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB set to a negative number.

Common situations: Sign typos in env config; trying to disable quantization with -1 instead of 0 or unsetting the var; stale values from older tuning docs.

Related errors

• VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be non-negativ
• VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be less than o
• Flashinfer allreduce quantization fusion is not supported fo
• chat request must contain at least one message
• Invalid wheel filename format: {wheel_name}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/fc22ece722e97aa3. Report an issue: GitHub.