vllm-project/vllm · error · ValueError

VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be less than o

Error message

VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be less than or equal to the effective QuickReduce max size

What it means

After scaling VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB to bytes, _get_qr_min_size checks it against the effective QuickReduce max buffer size (qr_max_size, derived from the engine's buffer budget). A minimum threshold larger than the maximum usable buffer is contradictory, so it raises ValueError.

Source

Thrown at vllm/distributed/device_communicators/quick_all_reduce.py:273

                "Custom quick allreduce: quantization codec threshold = %d KB",
                self.qr_quantization_min_size // KB,
            )
        self.create_shared_buffer()
        self.disabled = False

    @staticmethod
    def _get_qr_min_size(qr_max_size: int | None) -> int | None:
        qr_min_size = envs.VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB
        if qr_min_size is None:
            return None
        if qr_min_size < 0:
            raise ValueError(
                "VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be non-negative, "
                f"got {qr_min_size}"
            )
        qr_min_size *= MB
        if qr_max_size is not None and qr_min_size > qr_max_size:
            raise ValueError(
                "VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB must be less than or "
                "equal to the effective QuickReduce max size"
            )
        return qr_min_size

    @staticmethod
    def _get_qr_quantization_min_size() -> int | None:
        quantization_min_size = envs.VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB
        if quantization_min_size is None:
            return None
        if quantization_min_size < 0:
            raise ValueError(
                "VLLM_ROCM_QUICK_REDUCE_QUANTIZATION_MIN_SIZE_KB must be "
                f"non-negative, got {quantization_min_size}"
            )
        return quantization_min_size * KB

    def _rocm_arch_available(self):

View on GitHub (pinned to c794754062)

Solutions

Lower VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB to at most the effective max size
Or raise the QuickReduce max buffer size so it covers the desired minimum
Or unset the min-size variable to let the default thresholding apply

Example fix

# before
export VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB=512  # > max buffer 256MB

# after
export VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB=128  # <= max buffer

Defensive patterns

Strategy: validation

Validate before calling

v = os.environ.get("VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB")
if v is not None:
    assert int(v) * (1 << 20) <= qr_max_size, "min size (MiB->bytes) must not exceed QuickReduce max buffer"

Prevention

Derive thresholds from a single config so min/max cannot diverge
Document the max buffer size next to the env var
Validate the pair (min, max) in a startup check

When it happens

Trigger: Setting VLLM_ROCM_QUICK_REDUCE_MIN_SIZE_BYTES_MB to a value (in MiB) whose byte size exceeds the configured QuickReduce max buffer size, e.g. min=512MB while max is 256MB.

Common situations: Tuning quick-reduce thresholds without checking the max buffer budget; lowering the max buffer for memory reasons while leaving a large min; copying tuning values between machines with different buffer configs.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/990068620d28c80b. Report an issue: GitHub.