{"record":{"id":"f56c538f6aa629cd","repo":"vllm-project/vllm","slug":"offload-num-in-group-self-prefetch-offload-num-i","errorCode":null,"errorMessage":"offload_num_in_group ({self.prefetch.offload_num_in_group}) must be <= offload_group_size ({self.prefetch.offload_group_size})","messagePattern":"offload_num_in_group \\((.+?)\\) must be <= offload_group_size \\((.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/offload.py","lineNumber":102,"sourceCode":"    \"\"\"The backend for weight offloading. Options:\n    - \"auto\": Selects based on which sub-config has non-default values\n      (prefetch if offload_group_size > 0, uva if cpu_offload_gb > 0).\n    - \"uva\": UVA (Unified Virtual Addressing) zero-copy offloading.\n    - \"prefetch\": Async prefetch with group-based layer offloading.\n    \"\"\"\n\n    uva: UVAOffloadConfig = Field(default_factory=UVAOffloadConfig)\n    \"\"\"Parameters for UVA offloading backend.\"\"\"\n\n    prefetch: PrefetchOffloadConfig = Field(default_factory=PrefetchOffloadConfig)\n    \"\"\"Parameters for prefetch offloading backend.\"\"\"\n\n    @model_validator(mode=\"after\")\n    def validate_offload_config(self) -> \"OffloadConfig\":\n        \"\"\"Validate offload configuration constraints.\"\"\"\n        if self.offload_backend == \"prefetch\" or self.prefetch.offload_group_size > 0:\n            if self.prefetch.offload_num_in_group > self.prefetch.offload_group_size:\n                raise ValueError(\n                    f\"offload_num_in_group ({self.prefetch.offload_num_in_group})\"\n                    f\" must be <= offload_group_size\"\n                    f\" ({self.prefetch.offload_group_size})\"\n                )\n            if self.prefetch.offload_prefetch_step < 1:\n                raise ValueError(\n                    f\"offload_prefetch_step\"\n                    f\" ({self.prefetch.offload_prefetch_step})\"\n                    f\" must be >= 1 when prefetch offloading is enabled\"\n                    f\" (offload_group_size > 0)\"\n                )\n\n        # Warn if both backends have non-default values\n        uva_active = self.uva.cpu_offload_gb > 0\n        prefetch_active = self.prefetch.offload_group_size > 0\n        if self.offload_backend == \"uva\" and prefetch_active:\n            warnings.warn(\n                \"Prefetch offload fields are set but offload_backend='uva'. \"","sourceCodeStart":84,"sourceCodeEnd":120,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/offload.py#L84-L120","documentation":"OffloadConfig.validate_offload_config rejects prefetch-offload settings where offload_num_in_group exceeds offload_group_size. The group defines how many KV-cache blocks form an offload group; you cannot offload more blocks per group than the group contains. Checked whenever offload_backend == 'prefetch' or offload_group_size > 0.","triggerScenarios":"Passing --offload-num-in-group 8 with --offload-group-size 4 (or defaults/CLI combos where num_in_group > group_size), either with backend 'prefetch' explicitly or merely a positive group size.","commonSituations":"Tuning CPU offloading knobs independently without understanding their relationship; setting num_in_group from an old benchmark note while shrinking group_size to save memory.","solutions":["Lower offload_num_in_group to at most offload_group_size (e.g. group 4, num_in_group 4).","Or raise offload_group_size to be >= offload_num_in_group.","Re-read the prefetch offloader docs to pick a sane ratio (commonly num_in_group == group_size or half)."],"exampleFix":"# before\nvllm serve model --offload-backend prefetch --offload-num-in-group 8 --offload-group-size 4\n\n# after\nvllm serve model --offload-backend prefetch --offload-num-in-group 4 --offload-group-size 4","handlingStrategy":"validation","validationCode":"def check_offload_group(num_in_group: int, group_size: int) -> None:\n    if group_size > 0 and num_in_group > group_size:\n        raise SystemExit(f\"offload_num_in_group ({num_in_group}) must be <= offload_group_size ({group_size})\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive num_in_group from group_size in your config generator instead of hardcoding both.","When tuning offloading, change group_size first and recompute num_in_group."],"tags":["vllm","config","offloading","kv-cache","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}