sgl-project/sglang · warning

LONG GARBAGE COLLECTION DETECTED | Generation {} | Duration:

Error message

LONG GARBAGE COLLECTION DETECTED | Generation {} | Duration: {:.4f}s | # Objects: gen0={}, gen1={}, gen2={} | This may cause latency jitter. Consider calling the freeze_gc API after sending a few warmup requests.

What it means

SGLang installs a gc.callbacks hook that measures wall-clock duration of every Python garbage collection cycle per generation. When any single GC 'stop' event exceeds the warn threshold (typically ~1s), it logs 'LONG GARBAGE COLLECTION DETECTED' with the generation, duration, and object counts, because long GC pauses stall the scheduler event loop and show up as latency jitter / TTFT spikes for served requests.

Source

Thrown at python/sglang/srt/utils/common.py:4003

    g1 = len(gc.get_objects(1))
    g2 = len(gc.get_objects(2))
    return g0, g1, g2


def configure_gc_warning(warn_threshold_secs):
    import gc

    gc_start_time = {}

    def gc_callback(phase, info):
        gen = info.get("generation", "?")
        if phase == "start":
            gc_start_time[gen] = time.time()
        elif phase == "stop":
            duration = time.time() - gc_start_time.get(gen, time.time())
            if duration > warn_threshold_secs:
                g0, g1, g2 = gc_object_counts()
                logger.warn(
                    f"LONG GARBAGE COLLECTION DETECTED | Generation {gen} | Duration: {duration:.4f}s | # Objects: gen0={g0}, gen1={g1}, gen2={g2} | "
                    f"This may cause latency jitter. Consider calling the freeze_gc API after sending a few warmup requests."
                )

    gc.callbacks.append(gc_callback)


def freeze_gc(context: str):
    g0_before, g1_before, g2_before = gc_object_counts()
    gc.freeze()
    g0_after, g1_after, g2_after = gc_object_counts()
    logger.info(
        f"Freezing GC in {context} process. "
        f"gen0: {g0_before}->{g0_after}, "
        f"gen1: {g1_before}->{g1_after}, "
        f"gen2: {g2_before}->{g2_after}"
    )

View on GitHub (pinned to 0132848349)

Solutions

1. Send a few warmup requests, then hit the freeze_gc API (or call sglang.srt.utils.freeze_gc / the freeze_gc endpoint if exposed) so gc.freeze() moves surviving objects out of collection and gc.disable() pauses collection
2. Reduce cyclic garbage: lower --max-running-requests or batch sizes so fewer Req/TokenMetadata objects are alive at once
3. If GC still runs, tune thresholds via gc.set_threshold() instead of full disable
4. Use the SGLANG_... GC-related env/flags if available in your version to control the monitor threshold

Example fix

# after warmup
import gc, sglang.srt.utils as sgu
sgu.freeze_gc()  # gc.freeze() + gc.disable() inside the scheduler process
# or via engine API if exposed: engine.freeze_gc()

from sglang.srt.utils import freeze_gc
# after N warmup requests:
freeze_gc()  # call inside the scheduler/tokenizer process or via exposed API

When it happens

Trigger: Running an SGLang server under workloads that allocate many Python objects (large batches, long contexts, many small requests) so gen0/gen1/gen2 collections take longer than the threshold; frequently triggered after heavy request bursts retire and cyclic garbage is collected.

Common situations: Latency spikes observed in production serving; benchmark profiles showing periodic pauses; happens especially with high concurrency, large numbers of Req objects, or libraries that create reference cycles.

Related errors

• module {__name__!r} has no attribute {name!r}
• {response.error}
• action policy returned no output
• Could not connect to remote scheduler at {self.server_args.s
• {output_batch.error}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b3503e8870cdb663. Report an issue: GitHub.