sgl-project/sglang · error · ValueError

MiniMax H3 AdaLN cache must be built on CUDA

Error message

MiniMax H3 AdaLN cache must be built on CUDA

What it means

The AdaLN cache precomputation kernels require CUDA; the tool refuses to run when --device is not a CUDA device or when torch.cuda.is_available() is False, even if the device string itself says cuda.

Source

Thrown at python/sglang/multimodal_gen/tools/build_minimax_h3_adaln_cache.py:159

    *,
    weight_map: dict[str, str],
    files: dict[str, Any],
    device: torch.device,
) -> torch.Tensor:
    tensor_file = files[weight_map[name]]
    return tensor_file.get_tensor(name).to(device)


def main() -> None:
    args = _parse_args()
    if args.num_inference_steps < 2 and args.timesteps is None:
        raise ValueError("--num-inference-steps must be at least 2")
    mode_variant = _MODE_VARIANTS[args.mode]
    if args.model_variant != mode_variant:
        raise ValueError(f"--mode {args.mode} requires {mode_variant}")
    device = torch.device(args.device)
    if device.type != "cuda" or not torch.cuda.is_available():
        raise ValueError("MiniMax H3 AdaLN cache must be built on CUDA")

    index_path = args.transformer_path / "model.safetensors.index.json"
    with index_path.open() as f:
        weight_map = json.load(f)["weight_map"]

    plans = _cache_timestep_plans(args)
    if not plans or any(plan.numel() == 0 for plan in plans):
        raise ValueError("AdaLN cache must cover at least one timestep plan")
    max_plan_length = max(plan.numel() for plan in plans)
    plan_timesteps = torch.zeros((len(plans), max_plan_length), dtype=torch.float32)
    plan_lengths = torch.tensor([plan.numel() for plan in plans], dtype=torch.int64)
    block_params = torch.empty(
        (len(plans), max_plan_length, _NUM_BLOCKS, _BLOCK_PARAM_WIDTH),
        dtype=torch.bfloat16,
    )
    final_params = torch.empty(
        (len(plans), max_plan_length, _FINAL_PARAM_WIDTH), dtype=torch.bfloat16
    )

View on GitHub (pinned to 0132848349)

Solutions

1. Run on a machine with a working GPU and CUDA toolchain
2. Pass --device cuda (or a specific cuda:N)
3. If GPU is present, fix the environment (driver, CUDA-visible-devices, container runtime) so torch.cuda.is_available() returns True

Example fix

# before
--device cpu
# after
--device cuda

import torch
if not torch.cuda.is_available():
    raise SystemExit("CUDA required; no GPU visible")

When it happens

Trigger: Running the tool with --device cpu, or on a machine/container where no usable CUDA device is visible.

Common situations: Building the cache on a CPU-only box or inside a container without GPU passthrough; CUDA driver mismatch making torch.cuda.is_available() False; typo in the device string.

Related errors

• cuda.bindings.driver is required for CUDA VMM operations
• no supported CUDA VMM allocation handle type
• Unknown serve backend {name!r}. Available values: {available
• Multiple distributions register serve backend {name!r}: {pro
• Failed to load serve backend {name!r} from {self._entry_poin

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0e9a840aca00c1db. Report an issue: GitHub.