sgl-project/sglang · error · ValueError

additional customized generation output is not supported by

Error message

additional customized generation output is not supported by Rust egress

What it means

OutputStreamer refuses to combine Rust-based egress with requests that carry additional customized generation output; the Rust output path does not serialize the extra per-request customized info fields.

Source

Thrown at python/sglang/srt/managers/scheduler_components/output_streamer.py:70

    has_additional_customized_info: ClassVar[bool] = False

    send_to_detokenizer: zmq.Socket
    tree_cache: BasePrefixCache
    ps: ParallelState
    server_args: ServerArgs
    is_generation: bool
    spec_algorithm: SpeculativeAlgorithm
    disaggregation_mode: DisaggregationMode
    enable_hicache_storage: Callable[[], bool]
    # When SGLANG_RUST_SERVER is on, generation output is pushed to the embedded
    # Rust egress ring via `rust_server.push_generation` instead of the zmq
    # detokenizer. None otherwise. (Rust-specific state lives in RustServer.)
    rust_server: Optional[RustServer] = None
    _test_stream_output_count: int = 0

    def __post_init__(self) -> None:
        if self.has_additional_customized_info and self.rust_server is not None:
            raise ValueError(
                "additional customized generation output is not supported by "
                "Rust egress"
            )

    def _get_storage_backend_type(self) -> str:
        """Get storage backend type from tree_cache."""
        storage_backend_type = "none"
        cache_controller = getattr(self.tree_cache, "cache_controller", None)
        if cache_controller and hasattr(cache_controller, "storage_backend"):
            storage_backend = cache_controller.storage_backend
            if storage_backend is not None:
                storage_backend_type = type(storage_backend).__name__
        return storage_backend_type

    def get_cached_tokens_details(self, req: Req) -> Optional[CachedTokensDetails]:
        """Get detailed cache breakdown for a request, if available.

        Returns:

View on GitHub (pinned to 0132848349)

Solutions

1. Disable the additional customized output feature for requests when using Rust egress
2. Switch back to the Python egress path (do not set rust_server) if customized outputs are required
3. File/track support for customized output in the Rust egress path

if streamer.rust_server is not None and request_has_customized_output:
    raise SystemExit('disable customized output or Rust egress before starting')

When it happens

Trigger: Constructing OutputStreamer with rust_server set while has_additional_customized_info is true (e.g. using the Rust tokenizer/detokenizer path with customized output fields enabled on requests).

Common situations: Enabling the Rust egress/detokenizer feature while a model or middleware attaches additional customized generation outputs.

Related errors

• QVGPackedCausalKVCache does not support pinned-sink (longliv
• [weight_cache] {op} of model weights is not supported while
• LoRA is only compatible with NGRAM, EAGLE, NEXTN, EAGLE3, DF
• LoRA with EAGLE/NEXTN/EAGLE3 speculative decoding {reason}.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0cf7f73ebbd34697. Report an issue: GitHub.