vllm-project/vllm · error · ValueError

'mm_encoder_fp8_scale_path' and 'mm_encoder_fp8_scale_save_p

Error message

'mm_encoder_fp8_scale_path' and 'mm_encoder_fp8_scale_save_path' require 'mm_encoder_attn_dtype' to be 'fp8'.

What it means

MultiModalConfig's post-init validator requires mm_encoder_attn_dtype to be 'fp8' when either mm_encoder_fp8_scale_path or mm_encoder_fp8_scale_save_path is set. Static or saved FP8 scale files only apply to an encoder running FP8 attention, so any other dtype makes them meaningless and is rejected.

Source

Thrown at vllm/config/multimodal.py:323

        )
        return AttentionBackendEnum[value.upper()]

    @model_validator(mode="after")
    def _validate_multimodal_config(self):
        if self.mm_processor_cache_type != "shm" and (
            self.mm_shm_cache_max_object_size_mb
            != MultiModalConfig.mm_shm_cache_max_object_size_mb
        ):
            raise ValueError(
                "'mm_shm_cache_max_object_size_mb' should only be set when "
                "'mm_processor_cache_type' is 'shm'."
            )
        # Validate FP8 scale path combinations.
        if self.mm_encoder_attn_dtype != "fp8" and (
            self.mm_encoder_fp8_scale_path is not None
            or self.mm_encoder_fp8_scale_save_path is not None
        ):
            raise ValueError(
                "'mm_encoder_fp8_scale_path' and "
                "'mm_encoder_fp8_scale_save_path' require "
                "'mm_encoder_attn_dtype' to be 'fp8'."
            )
        if (
            self.mm_encoder_fp8_scale_path is not None
            and self.mm_encoder_fp8_scale_save_path is not None
        ):
            raise ValueError(
                "'mm_encoder_fp8_scale_save_path' cannot be used with "
                "'mm_encoder_fp8_scale_path' (saving requires dynamic scaling)."
            )

        # Validate file paths exist.
        if self.mm_encoder_fp8_scale_path is not None:
            scale_path = Path(self.mm_encoder_fp8_scale_path)
            if not scale_path.is_file():
                raise FileNotFoundError(f"FP8 scale file not found: {scale_path}")

View on GitHub (pinned to c794754062)

Solutions

Add --mm-encoder-attn-dtype fp8 alongside the scale path flags.
If you do not want FP8 encoder attention, remove --mm-encoder-fp8-scale-path and --mm-encoder-fp8-scale-save-path.

Example fix

# before
vllm serve Qwen/Qwen2.5-VL-7B --mm-encoder-fp8-scale-path scales.pt

# after
vllm serve Qwen/Qwen2.5-VL-7B --mm-encoder-attn-dtype fp8 --mm-encoder-fp8-scale-path scales.pt

Defensive patterns

Strategy: validation

Validate before calling

def check_fp8_opts(attn_dtype, scale_path, save_path):
    if (scale_path or save_path) and attn_dtype != "fp8":
        raise SystemExit("--mm-encoder-attn-dtype fp8 is required when scale paths are set")

Prevention

Always emit the FP8 trio together: attn dtype, then exactly one scale-path flag.
In config templates, make the scale-path fields conditional on dtype == 'fp8'.

When it happens

Trigger: Setting --mm-encoder-fp8-scale-path /path/scales.pt or --mm-encoder-fp8-scale-save-path while mm_encoder_attn_dtype is omitted (None) — note the field is typed Literal["fp8"] | None, so it must be explicitly set to "fp8".

Common situations: Calibrating FP8 for a multimodal encoder but forgetting the dtype flag; reordering CLI flags during a refactor and dropping --mm-encoder-attn-dtype fp8.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/5685a43b3a68fbc9. Report an issue: GitHub.