sgl-project/sglang · error · RuntimeError

{failure_msg}: {error_msg}

Error message

{failure_msg}: {error_msg}

What it means

A LoRA management call (set_lora, merge_lora_weights, unmerge_lora_weights, list_loras) sent a synchronous control request to the scheduler via sync_scheduler_client.forward, and the scheduler responded with an error. The failure message is prefixed with a per-operation label (failure_msg) and the scheduler's error string appended.

Source

Thrown at python/sglang/multimodal_gen/runtime/entrypoints/diffusion_generator.py:568

            raise RuntimeError(
                f"Expected {request_count} outputs, got {output_count} from scheduler"
            )

    def _send_to_scheduler_and_wait_for_response(self, batch: list[Req]) -> OutputBatch:
        """
        Sends a request to the scheduler and waits for a response.
        """
        return sync_scheduler_client.forward(batch)

    # LoRA
    def _send_lora_request(self, req: Any, success_msg: str, failure_msg: str):
        response = sync_scheduler_client.forward(req)
        if response.error is None:
            logger.info(success_msg)
            return response
        else:
            error_msg = response.error
            raise RuntimeError(f"{failure_msg}: {error_msg}")

    def set_lora(
        self,
        lora_nickname: Union[str, List[str]],
        lora_path: Union[str, None, List[Union[str, None]]] = None,
        target: Union[str, List[str]] = "all",
        strength: Union[float, List[float]] = 1.0,
        merge_mode: str | None = None,
        lora_alpha: Optional[Union[int, List[Optional[int]]]] = None,
    ) -> None:
        """
        Set LoRA adapter(s) for the specified transformer(s).
        Supports both single LoRA (backward compatible) and multiple LoRA adapters.

        Args:
            lora_nickname: The nickname(s) of the adapter(s). Can be a string or a list of strings.
            lora_path: Path(s) to the LoRA adapter(s). Can be a string, None, or a list of strings/None.
            target: Which transformer(s) to apply the LoRA to. Can be a string or a list of strings.

View on GitHub (pinned to 0132848349)

Solutions

  1. Verify the LoRA adapter path exists and was trained for this exact base model
  2. Ensure set_lora succeeded (and list_loras shows the nickname) before merge/unmerge
  3. Read the suffix of the message — it is the scheduler's specific error (load failure, unknown nickname, etc.)
  4. Retry after the scheduler is idle; avoid concurrent LoRA control calls

Example fix

# before
generator.set_lora("my-lora", "/tmp/lora")  # typo'd/incomplete path
generator.merge_lora_weights("my-lora")
# after
from pathlib import Path
adapter = Path("/models/my-lora").resolve()
assert (adapter / "adapter_config.json").exists()
generator.set_lora("my-lora", str(adapter))
generator.merge_lora_weights("my-lora")
Defensive patterns

Strategy: try-catch

Validate before calling

from pathlib import Path
def lora_ready(nickname, path):
    p = Path(path)
    return nickname and (p / "adapter_config.json").exists()

Try / catch

try:
    generator.set_lora(nick, path)
except RuntimeError as e:
    logger.error("set_lora failed: %s", e)  # message contains scheduler-side cause
    # fall back to baseline model, do not merge

Prevention

When it happens

Trigger: set_lora('nick','/bad/path') where the adapter fails to load or the path does not exist; merging weights for a nickname that was never registered; LoRA rank/shape incompatible with the base model; scheduler busy or in a state that rejects LoRA ops.

Common situations: Wrong or missing adapter path; LoRA trained against a different base model; calling merge before set; concurrent LoRA operations racing on the scheduler; adapter file corrupted or incomplete download.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ca3cc3f6534389ca. Report an issue: GitHub.