BerriAI/litellm · error · HTTPException

Failed to set GC thresholds: {e}

Error message

Failed to set GC thresholds: {e}

What it means

The admin endpoint POST /debug/memory/gc/configure calls gc.set_threshold(generation_0, generation_1, generation_2). If Python rejects the values (classically a negative integer, since gc.set_threshold requires non-negative ints), the wrapper converts the exception into HTTPException 500 with detail 'Failed to set GC thresholds: {e}'. The old thresholds stay in effect; nothing is mutated.

Source

Thrown at litellm/proxy/common_utils/debug_utils.py:659

    curl -X POST "http://localhost:4000/debug/memory/gc/configure?generation_0=500" -H "Authorization: Bearer sk-1234"

    Example for less aggressive collection:
    curl -X POST "http://localhost:4000/debug/memory/gc/configure?generation_0=1000" -H "Authorization: Bearer sk-1234"

    Monitor memory usage with GET /debug/memory/summary after changes.
    """
    # Get current thresholds for logging
    old_thresholds: Final = gc.get_threshold()

    # Set new thresholds with error handling
    try:
        gc.set_threshold(generation_0, generation_1, generation_2)
        verbose_proxy_logger.info(
            "GC thresholds updated from %s to (%s, %s, %s)", old_thresholds, generation_0, generation_1, generation_2
        )
    except Exception as e:
        verbose_proxy_logger.error("Failed to set GC thresholds: %s", e)
        raise HTTPException(status_code=500, detail=f"Failed to set GC thresholds: {e}")

    # Get current object count to show immediate impact
    current_count: Final = gc.get_count()[0]

    return {
        "message": "GC thresholds updated",
        "previous_thresholds": f"{old_thresholds[0]}, {old_thresholds[1]}, {old_thresholds[2]}",
        "new_thresholds": f"{generation_0}, {generation_1}, {generation_2}",
        "objects_awaiting_collection": current_count,
        "tip": f"Next collection will run after {generation_0 - current_count} more allocations",
    }


@router.get(
    "/otel-spans",
    dependencies=[Depends(user_api_key_auth)],
    include_in_schema=False,
)

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Retry the call with non-negative integers, e.g. POST /debug/memory/gc/configure?generation_0=700&generation_1=10&generation_2=10.
2. If values came from a script, clamp them to >= 0 before the request.
3. Check the response detail text: it embeds the Python error, which confirms which value was rejected.

Example fix

# before
curl -X POST "http://localhost:4000/debug/memory/gc/configure?generation_0=-500" -H "Authorization: Bearer sk-1234"
# 500 Failed to set GC thresholds

# after
curl -X POST "http://localhost:4000/debug/memory/gc/configure?generation_0=500" -H "Authorization: Bearer sk-1234"

def valid_gc_params(g0: int, g1: int, g2: int) -> bool:
    vals = [g0, g1, g2]
    return all(isinstance(v, int) and not isinstance(v, bool) and v >= 0 for v in vals)

assert valid_gc_params(700, 10, 10)
# only then call POST /debug/memory/gc/configure

import httpx

resp = httpx.post(f"{base}/debug/memory/gc/configure", params={"generation_0": g0, "generation_1": g1, "generation_2": g2}, headers=headers)
if resp.status_code == 500 and "Failed to set GC thresholds" in resp.text:
    # thresholds unchanged; fix values and retry once with valid input
    resp = httpx.post(f"{base}/debug/memory/gc/configure", params={"generation_0": max(g0, 0), "generation_1": max(g1, 0), "generation_2": max(g2, 0)}, headers=headers)
resp.raise_for_status()

When it happens

Trigger: Call POST /debug/memory/gc/configure?generation_0=-1 (any negative value for generation_0/1/2). Non-integer input is normally caught earlier by FastAPI as 422, so negative ints are the usual trigger. Custom GC tooling that computes thresholds can also pass negative values after drift.

Common situations: Tuning memory with the debug endpoint and typing a negative number. A script copies an example query and edits values by hand. An automated tuner probes limits and hits the invalid range.

Related errors

• Key is blocked. Update via `/key/unblock` if you're an admin
• {e}
• {e}
• Error updating key: {response_text}
• {e}

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/b33cadd3589de970. Report an issue: GitHub.