vllm-project/vllm · error · ValueError

ec_cpu_bytes must be specified in ec_connector_extra_config

Error message

ec_cpu_bytes must be specified in ec_connector_extra_config

What it means

The CPU encoder-cache shared region is sized in blocks derived from ec_cpu_bytes, which must be supplied through ec_connector_extra_config (a JSON-style extra config on ECTransferConfig). Without it the region cannot be dimensioned (num_blocks = ec_cpu_bytes // block_size_bytes), so build_region raises ValueError with the exact key name to set.

Source

Thrown at vllm/distributed/ec_transfer/ec_connector/cpu/common.py:81

    """Build the EC mmap region from `vllm_config`.

    Both `ECCPUScheduler` and `ECCPUWorker` call this to get the same
    shared region (same engine_id, same block_size_bytes).
    """
    ec_config = vllm_config.ec_transfer_config
    assert ec_config is not None, "ec_transfer_config required to build region"

    dp_rank = vllm_config.parallel_config.data_parallel_rank
    engine_id = f"{vllm_config.instance_id}_dp{dp_rank}"

    dtype = vllm_config.model_config.dtype
    hidden_dim = _get_encoder_cache_hidden_dim(vllm_config)
    element_size = torch.empty(0, dtype=dtype).element_size()
    block_size_bytes = hidden_dim * element_size

    cpu_bytes = ec_config.ec_connector_extra_config.get("ec_cpu_bytes")
    if not cpu_bytes:
        raise ValueError("ec_cpu_bytes must be specified in ec_connector_extra_config")
    cpu_bytes = int(cpu_bytes)
    num_blocks = cpu_bytes // block_size_bytes

    return ECSharedRegion(
        engine_id=engine_id,
        num_blocks=num_blocks,
        block_size_bytes=block_size_bytes,
    )

View on GitHub (pinned to c794754062)

Solutions

Add ec_cpu_bytes to the extra config, sized as hidden_dim * dtype_size * desired_block_count (e.g. a few GiB for multimodal encoder caches)
Verify the key is exactly 'ec_cpu_bytes' and the value is a positive integer string
Cross-check against _get_encoder_cache_hidden_dim(vllm_config) * element_size to ensure num_blocks comes out > 0

Example fix

# before
ECTransferConfig(ec_connector="ECCPUConnector", ec_connector_extra_config={})
# ValueError: ec_cpu_bytes must be specified in ec_connector_extra_config

# after
ECTransferConfig(
    ec_connector="ECCPUConnector",
    ec_connector_extra_config={"ec_cpu_bytes": str(8 * 1024**3)},
)

Defensive patterns

Strategy: validation

Validate before calling

cfg = ec_transfer_config.ec_connector_extra_config
cpu_bytes = int(cfg.get("ec_cpu_bytes", 0))
assert cpu_bytes > 0, "set ec_connector_extra_config['ec_cpu_bytes'] (bytes)"

Type guard

def ec_region_configured(ec_transfer_config) -> bool:
    return bool(ec_transfer_config.ec_connector_extra_config.get("ec_cpu_bytes"))

Prevention

Always pair the CPU EC connector with an explicit ec_cpu_bytes
Validate extra-config keys at startup, not at first region build
Size ec_cpu_bytes from hidden_dim * dtype_size * block count

When it happens

Trigger: Enabling the CPU EC connector but omitting "ec_cpu_bytes" in --ec-transfer-extra-config / ec_connector_extra_config; passing a falsy value like 0 or an empty string (the code checks `if not cpu_bytes`, so 0 also fails); typo in the key name.

Common situations: First-time setup of CPU-based encoder-cache transfer between producer and consumer vLLM instances; copying extra-config JSON from docs that omit this mandatory key; passing the value under a different key like cpu_bytes.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/828728e389d0e8d4. Report an issue: GitHub.