{"record":{"id":"5685a43b3a68fbc9","repo":"vllm-project/vllm","slug":"mm-encoder-fp8-scale-path-and-mm-encoder-fp8-sc","errorCode":null,"errorMessage":"'mm_encoder_fp8_scale_path' and 'mm_encoder_fp8_scale_save_path' require 'mm_encoder_attn_dtype' to be 'fp8'.","messagePattern":"'mm_encoder_fp8_scale_path' and 'mm_encoder_fp8_scale_save_path' require 'mm_encoder_attn_dtype' to be 'fp8'\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/multimodal.py","lineNumber":323,"sourceCode":"        )\n        return AttentionBackendEnum[value.upper()]\n\n    @model_validator(mode=\"after\")\n    def _validate_multimodal_config(self):\n        if self.mm_processor_cache_type != \"shm\" and (\n            self.mm_shm_cache_max_object_size_mb\n            != MultiModalConfig.mm_shm_cache_max_object_size_mb\n        ):\n            raise ValueError(\n                \"'mm_shm_cache_max_object_size_mb' should only be set when \"\n                \"'mm_processor_cache_type' is 'shm'.\"\n            )\n        # Validate FP8 scale path combinations.\n        if self.mm_encoder_attn_dtype != \"fp8\" and (\n            self.mm_encoder_fp8_scale_path is not None\n            or self.mm_encoder_fp8_scale_save_path is not None\n        ):\n            raise ValueError(\n                \"'mm_encoder_fp8_scale_path' and \"\n                \"'mm_encoder_fp8_scale_save_path' require \"\n                \"'mm_encoder_attn_dtype' to be 'fp8'.\"\n            )\n        if (\n            self.mm_encoder_fp8_scale_path is not None\n            and self.mm_encoder_fp8_scale_save_path is not None\n        ):\n            raise ValueError(\n                \"'mm_encoder_fp8_scale_save_path' cannot be used with \"\n                \"'mm_encoder_fp8_scale_path' (saving requires dynamic scaling).\"\n            )\n\n        # Validate file paths exist.\n        if self.mm_encoder_fp8_scale_path is not None:\n            scale_path = Path(self.mm_encoder_fp8_scale_path)\n            if not scale_path.is_file():\n                raise FileNotFoundError(f\"FP8 scale file not found: {scale_path}\")","sourceCodeStart":305,"sourceCodeEnd":341,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/multimodal.py#L305-L341","documentation":"MultiModalConfig's post-init validator requires mm_encoder_attn_dtype to be 'fp8' when either mm_encoder_fp8_scale_path or mm_encoder_fp8_scale_save_path is set. Static or saved FP8 scale files only apply to an encoder running FP8 attention, so any other dtype makes them meaningless and is rejected.","triggerScenarios":"Setting --mm-encoder-fp8-scale-path /path/scales.pt or --mm-encoder-fp8-scale-save-path while mm_encoder_attn_dtype is omitted (None) — note the field is typed Literal[\"fp8\"] | None, so it must be explicitly set to \"fp8\".","commonSituations":"Calibrating FP8 for a multimodal encoder but forgetting the dtype flag; reordering CLI flags during a refactor and dropping --mm-encoder-attn-dtype fp8.","solutions":["Add --mm-encoder-attn-dtype fp8 alongside the scale path flags.","If you do not want FP8 encoder attention, remove --mm-encoder-fp8-scale-path and --mm-encoder-fp8-scale-save-path."],"exampleFix":"# before\nvllm serve Qwen/Qwen2.5-VL-7B --mm-encoder-fp8-scale-path scales.pt\n\n# after\nvllm serve Qwen/Qwen2.5-VL-7B --mm-encoder-attn-dtype fp8 --mm-encoder-fp8-scale-path scales.pt","handlingStrategy":"validation","validationCode":"def check_fp8_opts(attn_dtype, scale_path, save_path):\n    if (scale_path or save_path) and attn_dtype != \"fp8\":\n        raise SystemExit(\"--mm-encoder-attn-dtype fp8 is required when scale paths are set\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always emit the FP8 trio together: attn dtype, then exactly one scale-path flag.","In config templates, make the scale-path fields conditional on dtype == 'fp8'."],"tags":["vllm","config","multimodal","fp8","quantization","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}