OpenBMB/VoxCPM · error · ValueError

VoxCPMModel can only be optimized on CUDA device

Error message

VoxCPMModel can only be optimized on CUDA device

What it means

VoxCPMModel.optimize uses torch.compile on forward steps, which requires the model to live on a CUDA device; calling optimize() on CPU/MPS raises ValueError (unless disable=True).

Source

Thrown at src/voxcpm/model/voxcpm.py:234

            apply_lora_to_named_linear_modules(
                self.feat_decoder.estimator, target_submodule_names=cfg.target_modules_dit, **lora_kwargs
            )

        # 投影层
        if cfg.enable_proj:
            from ..modules.layers.lora import LoRALinear

            for attr_name in cfg.target_proj_modules:
                module = getattr(self, attr_name, None)
                if isinstance(module, nn.Linear):
                    setattr(self, attr_name, LoRALinear(base=module, **lora_kwargs))

    def optimize(self, disable: bool = False):
        if disable:
            return self
        try:
            if self.device != "cuda":
                raise ValueError("VoxCPMModel can only be optimized on CUDA device")
            try:
                import triton  # noqa: F401
            except ImportError:
                raise ValueError("triton is not installed")
            self.base_lm.forward_step = torch.compile(self.base_lm.forward_step, mode="reduce-overhead", fullgraph=True)
            self.residual_lm.forward_step = torch.compile(
                self.residual_lm.forward_step, mode="reduce-overhead", fullgraph=True
            )
            self._feat_encoder_raw = self.feat_encoder
            self.feat_encoder = torch.compile(self.feat_encoder, mode="reduce-overhead", fullgraph=True)
            self.feat_decoder.estimator = torch.compile(
                self.feat_decoder.estimator, mode="reduce-overhead", fullgraph=True
            )
        except Exception as e:
            print(f"Warning: torch.compile disabled - {e}", file=sys.stderr)
        return self

    def forward(

View on GitHub (pinned to f5a1c6a6b9)

Solutions

1. Set optimize=False (or disable=True) when not on CUDA
2. Run on a CUDA device if you need the compiled fast path
3. Confirm the effective device (model.device) before enabling optimize

Example fix

# before
model = VoxCPM(arch="v1", device="cpu", optimize=True)
# after
model = VoxCPM(arch="v1", device="cpu", optimize=False)

effective_device = model.tts_model.device if hasattr(model, 'tts_model') else None
optimize = optimize and effective_device == "cuda"

When it happens

Trigger: model = VoxCPM(device='cpu' or 'mps', optimize=True) — from_local passes optimize through, or calling model.optimize() manually on non-CUDA.

Common situations: Prototyping on a laptop/Mac with optimize=True copied from a GPU recipe, or device='auto' resolving to CPU in a GPU-less container.

Related errors

• triton is not installed

AI-assisted analysis of OpenBMB/VoxCPM@f5a1c6a6b9 (2026-08-27). Data as JSON: /api/errors/66c4364ae2dca3a9. Report an issue: GitHub.