vllm-project/vllm · error · ValueError

'mm_encoder_fp8_scale_save_path' cannot be used with 'mm_enc

Error message

'mm_encoder_fp8_scale_save_path' cannot be used with 'mm_encoder_fp8_scale_path' (saving requires dynamic scaling).

What it means

MultiModalConfig forbids setting both mm_encoder_fp8_scale_path and mm_encoder_fp8_scale_save_path. mm_encoder_fp8_scale_path supplies static, pre-computed scales, while mm_encoder_fp8_scale_save_path dumps dynamically-computed scales to disk; the save feature only exists for the dynamic-scaling path, so the two are mutually exclusive.

Source

Thrown at vllm/config/multimodal.py:332

            raise ValueError(
                "'mm_shm_cache_max_object_size_mb' should only be set when "
                "'mm_processor_cache_type' is 'shm'."
            )
        # Validate FP8 scale path combinations.
        if self.mm_encoder_attn_dtype != "fp8" and (
            self.mm_encoder_fp8_scale_path is not None
            or self.mm_encoder_fp8_scale_save_path is not None
        ):
            raise ValueError(
                "'mm_encoder_fp8_scale_path' and "
                "'mm_encoder_fp8_scale_save_path' require "
                "'mm_encoder_attn_dtype' to be 'fp8'."
            )
        if (
            self.mm_encoder_fp8_scale_path is not None
            and self.mm_encoder_fp8_scale_save_path is not None
        ):
            raise ValueError(
                "'mm_encoder_fp8_scale_save_path' cannot be used with "
                "'mm_encoder_fp8_scale_path' (saving requires dynamic scaling)."
            )

        # Validate file paths exist.
        if self.mm_encoder_fp8_scale_path is not None:
            scale_path = Path(self.mm_encoder_fp8_scale_path)
            if not scale_path.is_file():
                raise FileNotFoundError(f"FP8 scale file not found: {scale_path}")
        if self.mm_encoder_fp8_scale_save_path is not None:
            save_parent = Path(self.mm_encoder_fp8_scale_save_path).parent
            if not save_parent.is_dir():
                raise FileNotFoundError(
                    f"Parent directory for FP8 scale save path not found: {save_parent}"
                )
        return self

    @staticmethod

View on GitHub (pinned to c794754062)

Solutions

For a calibration/dump run keep only --mm-encoder-attn-dtype fp8 --mm-encoder-fp8-scale-save-path out.pt.
For serving with pre-computed scales keep only --mm-encoder-attn-dtype fp8 --mm-encoder-fp8-scale-path out.pt.
Split your launch script into a calibrate phase and a serve phase instead of one combined command.

Example fix

# before
vllm serve model --mm-encoder-attn-dtype fp8 \
  --mm-encoder-fp8-scale-path scales.pt \
  --mm-encoder-fp8-scale-save-path new.pt

# after
vllm serve model --mm-encoder-attn-dtype fp8 \
  --mm-encoder-fp8-scale-path scales.pt

Defensive patterns

Strategy: validation

Validate before calling

def check_scale_paths(scale_path, save_path):
    if scale_path and save_path:
        raise SystemExit("Use only one of mm_encoder_fp8_scale_path (static) or mm_encoder_fp8_scale_save_path (dynamic dump)")

Prevention

Separate calibration and serving launch scripts so the two flags never co-occur.
Document in-repo that save_path implies dynamic scaling and is incompatible with a static scale file.

When it happens

Trigger: Passing both --mm-encoder-fp8-scale-path scales.pt and --mm-encoder-fp8-scale-save-path out.pt (with or without --mm-encoder-attn-dtype fp8; the dtype check at line 319 fires first if dtype is not fp8).

Common situations: A single 'do everything' launch script that keeps flags for both a calibration run (save path) and a serving run (scale path); merging two working configs into one.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/79d8b918b8c64246. Report an issue: GitHub.