sgl-project/sglang · error · NotImplementedError

SANA-WM streaming does not support CFG parallel; run replica

Error message

SANA-WM streaming does not support CFG parallel; run replicated.

What it means

Streaming SANA-WM uses its own streaming_cfg_scale (officially 1.0, i.e. no CFG on the distilled 4-step model). If classifier-free guidance is requested (do_classifier_free_guidance and cfg_scale > 1.0) while enable_cfg_parallel is on, the streaming path raises NotImplementedError because CFG-parallel sharding is not implemented for it.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/sana_wm/streaming.py:451

        *,
        device: torch.device,
        target_dtype: torch.dtype,
        iload=None,
    ):
        """Resolve sampler config + text/camera conditioning (shared by the offline loop and realtime path)."""
        pcfg = server_args.pipeline_config
        sampler_cfg = SanaWMSelfForcingSamplerConfig.from_pipeline_config(pcfg)
        explicit_sigmas = SanaWMSelfForcingSampler.build_per_chunk_sigmas(
            sampler_cfg.denoising_step_list
        )

        # Streaming uses its OWN cfg scale (official StreamingGenerationConfig.cfg_scale=1.0
        # => no CFG on the distilled 4-step model). The general guidance_scale (e.g. 4.5)
        # is for the dense path; using it here ran CFG=4.5 vs the reference's none.
        cfg_scale = sampler_cfg.streaming_cfg_scale
        do_cfg = bool(batch.do_classifier_free_guidance) and cfg_scale > 1.0
        if server_args.enable_cfg_parallel and do_cfg:
            raise NotImplementedError(
                "SANA-WM streaming does not support CFG parallel; run replicated."
            )

        # --- text conditioning ---
        pos_embeds = _to_device_dtype(
            _first_tensor(pcfg.get_pos_prompt_embeds(batch)),
            device=device,
            dtype=target_dtype,
        )
        pos_mask = _to_device_dtype(
            _first_tensor(batch.prompt_attention_mask), device=device
        )
        if pos_embeds is None:
            raise ValueError("SANA-WM streaming requires positive prompt embeds.")
        neg_embeds = neg_mask = None
        if do_cfg:
            neg_embeds = _to_device_dtype(
                _first_tensor(pcfg.get_neg_prompt_embeds(batch)),

View on GitHub (pinned to 0132848349)

Solutions

1. Disable CFG parallel for streaming runs (run replicated)
2. Keep streaming_cfg_scale at 1.0 so do_cfg is False and the check never trips
3. Route CFG-hungry requests to the dense/offline path instead of streaming

Example fix

# before
server_args.enable_cfg_parallel = True
server_args.pipeline_config.sampler.streaming_cfg_scale = 4.5
# after
server_args.enable_cfg_parallel = False
server_args.pipeline_config.sampler.streaming_cfg_scale = 1.0

do_cfg = bool(batch.do_classifier_free_guidance) and cfg_scale > 1.0
assert not (server_args.enable_cfg_parallel and do_cfg), "streaming + CFG parallel unsupported"

null

try:
    resp = stage.forward(batch, server_args)
except NotImplementedError:
    server_args.enable_cfg_parallel = False
    resp = stage.forward(batch, server_args)  # rerun replicated

When it happens

Trigger: Launching the server with --enable-cfg-parallel together with a streaming_cfg_scale > 1.0 and CFG-flagged requests.

Common situations: Reusing dense-path server flags (guidance_scale ~4.5 + CFG parallel) on a streaming deployment; a client toggling do_classifier_free_guidance on realtime requests.

Related errors

• Server was launched with --enable-cfg-parallel but this requ
• SANA-WM realtime denoising expects this tick's pre-noised ch
• SANA-WM realtime denoising requires a realtime session
• chunk plan {plan} does not cover the incoming {incoming.shap
• SANA-WM streaming requires positive prompt embeds.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/452c634436ff5861. Report an issue: GitHub.