{"record":{"id":"b3503e8870cdb663","repo":"sgl-project/sglang","slug":"long-garbage-collection-detected-generation","errorCode":null,"errorMessage":"LONG GARBAGE COLLECTION DETECTED | Generation {} | Duration: {:.4f}s | # Objects: gen0={}, gen1={}, gen2={} | This may cause latency jitter. Consider calling the freeze_gc API after sending a few warmup requests.","messagePattern":"LONG GARBAGE COLLECTION DETECTED \\| Generation (.+?) \\| Duration: (.+?)s \\| # Objects: gen0=(.+?), gen1=(.+?), gen2=(.+?) \\| This may cause latency jitter\\. Consider calling the freeze_gc API after sending a few warmup requests\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"python/sglang/srt/utils/common.py","lineNumber":4003,"sourceCode":"    g1 = len(gc.get_objects(1))\n    g2 = len(gc.get_objects(2))\n    return g0, g1, g2\n\n\ndef configure_gc_warning(warn_threshold_secs):\n    import gc\n\n    gc_start_time = {}\n\n    def gc_callback(phase, info):\n        gen = info.get(\"generation\", \"?\")\n        if phase == \"start\":\n            gc_start_time[gen] = time.time()\n        elif phase == \"stop\":\n            duration = time.time() - gc_start_time.get(gen, time.time())\n            if duration > warn_threshold_secs:\n                g0, g1, g2 = gc_object_counts()\n                logger.warn(\n                    f\"LONG GARBAGE COLLECTION DETECTED | Generation {gen} | Duration: {duration:.4f}s | # Objects: gen0={g0}, gen1={g1}, gen2={g2} | \"\n                    f\"This may cause latency jitter. Consider calling the freeze_gc API after sending a few warmup requests.\"\n                )\n\n    gc.callbacks.append(gc_callback)\n\n\ndef freeze_gc(context: str):\n    g0_before, g1_before, g2_before = gc_object_counts()\n    gc.freeze()\n    g0_after, g1_after, g2_after = gc_object_counts()\n    logger.info(\n        f\"Freezing GC in {context} process. \"\n        f\"gen0: {g0_before}->{g0_after}, \"\n        f\"gen1: {g1_before}->{g1_after}, \"\n        f\"gen2: {g2_before}->{g2_after}\"\n    )\n","sourceCodeStart":3985,"sourceCodeEnd":4021,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/common.py#L3985-L4021","documentation":"SGLang installs a gc.callbacks hook that measures wall-clock duration of every Python garbage collection cycle per generation. When any single GC 'stop' event exceeds the warn threshold (typically ~1s), it logs 'LONG GARBAGE COLLECTION DETECTED' with the generation, duration, and object counts, because long GC pauses stall the scheduler event loop and show up as latency jitter / TTFT spikes for served requests.","triggerScenarios":"Running an SGLang server under workloads that allocate many Python objects (large batches, long contexts, many small requests) so gen0/gen1/gen2 collections take longer than the threshold; frequently triggered after heavy request bursts retire and cyclic garbage is collected.","commonSituations":"Latency spikes observed in production serving; benchmark profiles showing periodic pauses; happens especially with high concurrency, large numbers of Req objects, or libraries that create reference cycles.","solutions":["Send a few warmup requests, then hit the freeze_gc API (or call sglang.srt.utils.freeze_gc / the freeze_gc endpoint if exposed) so gc.freeze() moves surviving objects out of collection and gc.disable() pauses collection","Reduce cyclic garbage: lower --max-running-requests or batch sizes so fewer Req/TokenMetadata objects are alive at once","If GC still runs, tune thresholds via gc.set_threshold() instead of full disable","Use the SGLANG_... GC-related env/flags if available in your version to control the monitor threshold"],"exampleFix":"# after warmup\nimport gc, sglang.srt.utils as sgu\nsgu.freeze_gc()  # gc.freeze() + gc.disable() inside the scheduler process\n# or via engine API if exposed: engine.freeze_gc()","handlingStrategy":"validation","validationCode":"from sglang.srt.utils import freeze_gc\n# after N warmup requests:\nfreeze_gc()  # call inside the scheduler/tokenizer process or via exposed API","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always run warmup then freeze_gc before benchmarking or serving traffic","Monitor p99 latency for periodic spikes as a GC signal","Keep per-request Python object graphs small; avoid reference cycles in custom schedulers"],"tags":["gc","latency-jitter","performance","scheduler","python"],"backgroundTag":"gc-pause-latency","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}