vllm-project/vllm · error · ValueError

Inkling LoRA requires INKLING_MULTIMEM_AR=0: the Lamport fus

Error message

Inkling LoRA requires INKLING_MULTIMEM_AR=0: the Lamport fused-collective path bypasses the LoRA-wrapped wo_ud and dense down_proj layers on decode-sized batches, silently dropping their LoRA. Set INKLING_MULTIMEM_AR=0.

What it means

Error "Inkling LoRA requires INKLING_MULTIMEM_AR=0: the Lamport fused-collective path bypasses the LoRA-wrapped wo_ud and dense down_proj layers on decode-sized batches, silently dropping their LoRA. Set INKLING_MULTIMEM_AR=0." thrown in vllm-project/vllm.

Source

Thrown at vllm/config/lora.py:144

            raise ValueError("Dual CUDA streams are only supported on CUDA platforms.")
        if envs.VLLM_LORA_ENABLE_DUAL_STREAM and self.fully_sharded_loras:
            logger.warning_once(
                "fully_sharded_loras isn't compatible with "
                "VLLM_LORA_ENABLE_DUAL_STREAM, set VLLM_LORA_ENABLE_DUAL_STREAM=False"
            )
            envs.VLLM_LORA_ENABLE_DUAL_STREAM = False
        return self

    def verify_with_model_config(self, model_config: ModelConfig):
        if self.lora_dtype in (None, "auto"):
            self.lora_dtype = model_config.dtype
        elif isinstance(self.lora_dtype, str):
            self.lora_dtype = getattr(torch, self.lora_dtype)

        architectures = getattr(model_config, "architectures", None) or []
        is_inkling = any("Inkling" in arch for arch in architectures)
        if is_inkling and os.environ.get("INKLING_MULTIMEM_AR", "1") != "0":
            raise ValueError(
                "Inkling LoRA requires INKLING_MULTIMEM_AR=0: the Lamport "
                "fused-collective path bypasses the LoRA-wrapped wo_ud and dense "
                "down_proj layers on decode-sized batches, silently dropping "
                "their LoRA. Set INKLING_MULTIMEM_AR=0."
            )

View on GitHub (pinned to c794754062)

Solutions

1. Set INKLING_MULTIMEM_AR=0 in the environment when using Inkling LoRA.
2. Or do not use Inkling LoRA with the multimem all-reduce path.

When it happens

Trigger: Raised at vllm/config/lora.py:144 when validation fails: Inkling LoRA requires INKLING_MULTIMEM_AR=0. Typically triggered by an incompatible or incomplete vLLM configuration, an unsupported platform/backend combination, or a runtime resource/dependency that is missing.

Common situations: Commonly encountered at vllm/config/lora.py:144 during vLLM startup/config validation or runtime setup when: (1) conflicting CLI flags or config fields are combined, (2) the current platform (CUDA/ROCm/CPU/XPU) or installed optional packages do not support the requested feature, or (3) a required value is absent or out of range. Resolve by correcting the configuration as described in the message, or by selecting a supported alternative.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/de0dc4782da88777. Report an issue: GitHub.