sgl-project/sglang · error · ValueError

additional customized generation output is not supported by…

Error message

additional customized generation output is not supported by Rust egress

What it means

OutputStreamer refuses to combine Rust-based egress with requests that carry additional customized generation output; the Rust output path does not serialize the extra per-request customized info fields.

Solutions

  1. Disable the additional customized output feature for requests when using Rust egress
  2. Switch back to the Python egress path (do not set rust_server) if customized outputs are required
  3. File/track support for customized output in the Rust egress path
Defensive patterns

Strategy: validation

Validate before calling

if streamer.rust_server is not None and request_has_customized_output:
    raise SystemExit('disable customized output or Rust egress before starting')

Prevention

When it happens

Trigger: Constructing OutputStreamer with rust_server set while has_additional_customized_info is true (e.g. using the Rust tokenizer/detokenizer path with customized output fields enabled on requests).

Common situations: Enabling the Rust egress/detokenizer feature while a model or middleware attaches additional customized generation outputs.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0cf7f73ebbd34697. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/managers/scheduler_components/output_streamer.py:70

    has_additional_customized_info: ClassVar[bool] = False

    send_to_detokenizer: zmq.Socket
    tree_cache: BasePrefixCache
    ps: ParallelState
    server_args: ServerArgs
    is_generation: bool
    spec_algorithm: SpeculativeAlgorithm
    disaggregation_mode: DisaggregationMode
    enable_hicache_storage: Callable[[], bool]
    # When SGLANG_RUST_SERVER is on, generation output is pushed to the embedded
    # Rust egress ring via `rust_server.push_generation` instead of the zmq
    # detokenizer. None otherwise. (Rust-specific state lives in RustServer.)
    rust_server: Optional[RustServer] = None
    _test_stream_output_count: int = 0

    def __post_init__(self) -> None:
        if self.has_additional_customized_info and self.rust_server is not None:
            raise ValueError(
                "additional customized generation output is not supported by "
                "Rust egress"
            )

    def _get_storage_backend_type(self) -> str:
        """Get storage backend type from tree_cache."""
        storage_backend_type = "none"
        cache_controller = getattr(self.tree_cache, "cache_controller", None)
        if cache_controller and hasattr(cache_controller, "storage_backend"):
            storage_backend = cache_controller.storage_backend
            if storage_backend is not None:
                storage_backend_type = type(storage_backend).__name__
        return storage_backend_type

    def get_cached_tokens_details(self, req: Req) -> Optional[CachedTokensDetails]:
        """Get detailed cache breakdown for a request, if available.

        Returns:

View on GitHub (pinned to 0132848349)