What causes "local_buffer_size must be > 0" in vllm-project/vllm?

Set local_buffer_size to a positive size appropriate for your largest batch of KV blocks, e.g. "1GB" or 1073741824

vllm-project/vllm · error · ValueError

local_buffer_size must be > 0

Error message

local_buffer_size must be > 0

What it means

MooncakeStoreConfig requires local_buffer_size (the per-worker staging buffer for KV transfer) to be a positive byte count. It is parsed from the config file via _parse_size, so 0, a negative number, or a string like "0GB"/'-1' fails validation in __post_init__.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/mooncake/store/worker.py:144

    contributes 0; an external ``mooncake_client`` process owns the pool
    and the SSD tier).
    """

    metadata_server: str
    master_server_address: str
    protocol: str
    device_name: str
    mode: MooncakeMode = "embedded"
    global_segment_size: int = DEFAULT_GLOBAL_SEGMENT_SIZE
    local_buffer_size: int = DEFAULT_LOCAL_BUFFER_SIZE
    enable_offload: bool = False
    tenant_id: str = DEFAULT_TENANT_ID

    def __post_init__(self) -> None:
        if self.mode not in ("embedded", "standalone-store"):
            raise ValueError(f"unknown Mooncake mode: {self.mode!r}")
        if self.local_buffer_size <= 0:
            raise ValueError("local_buffer_size must be > 0")
        if self.mode == "embedded" and self.global_segment_size == 0:
            raise ValueError("embedded mode requires global_segment_size > 0")
        if self.mode == "standalone-store" and self.global_segment_size != 0:
            raise ValueError("standalone-store mode requires global_segment_size == 0")

    @staticmethod
    def from_file(file_path: str) -> "MooncakeStoreConfig":
        with open(file_path) as file:
            config = json.load(file)
        return MooncakeStoreConfig(
            metadata_server=config.get("metadata_server", ""),
            master_server_address=config.get("master_server_address", ""),
            protocol=config.get("protocol", "rdma"),
            device_name=config.get("device_name", ""),
            mode=config.get("mode", "embedded"),
            global_segment_size=_parse_size(
                config.get("global_segment_size", DEFAULT_GLOBAL_SEGMENT_SIZE)
            ),

View on GitHub (pinned to c794754062)

Solutions

Set local_buffer_size to a positive size appropriate for your largest batch of KV blocks, e.g. "1GB" or 1073741824
If you intended to reduce memory, lower it gradually (e.g. 64MB) rather than to zero

Example fix

// before
{ "local_buffer_size": "0" }

// after
{ "local_buffer_size": "1GB" }

Defensive patterns

Strategy: validation

Validate before calling

sz = raw.get("local_buffer_size")
parsed = _parse_size(sz) if isinstance(sz, (int, str)) else int(sz)
if parsed <= 0:
    raise ValueError("local_buffer_size must parse to a positive byte count")

Prevention

Never zero the staging buffer to 'disable' it — size it to your batch instead
Use unit strings ("512MB") for readability and lint them in config checks

When it happens

Trigger: "local_buffer_size": 0 / -1 / "0KB" in the mooncake store config file.

Common situations: Disabling the buffer by zeroing it (not supported — it is mandatory); size strings with typos that parse to 0; minimal/test configs copied from examples with tiny values.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/1b1dc4873bcb3afc. Report an issue: GitHub.