vllm-project/vllm · critical · RuntimeError

Worker has been garbage collected

Error message

Worker has been garbage collected

What it means

ElasticEPScalingExecutor holds its worker via weakref.ref(worker) so the executor never keeps the model worker alive. The `worker` property dereferences the ref; if the referent has been garbage collected (no strong references remain anywhere), it raises RuntimeError — a use-after-free guard for the async reconfiguration thread accessing a dead worker.

Source

Thrown at vllm/distributed/elastic_ep/elastic_execute.py:161

    return physical_to_logical, num_local_physical_experts, num_logical_experts


class ElasticEPScalingExecutor:
    def __init__(self, worker):
        self.worker_ref = weakref.ref(worker)
        self.reconfig_request = None
        self._staged_moe_quant_methods: dict[nn.Module, FusedMoEMethodBase] = {}
        self._async_executor = ThreadPoolExecutor(
            max_workers=1, thread_name_prefix="ElasticEPAsync"
        )
        self._async_future: Future[None] | None = None

    @property
    def worker(self):
        worker = self.worker_ref()
        if worker is None:
            raise RuntimeError("Worker has been garbage collected")
        return worker

    def execute(self, execute_method: str, *args, **kwargs):
        method = getattr(self, execute_method, None)
        if method is None:
            raise ValueError(f"Unknown execute method: {execute_method}")
        return method(*args, **kwargs)

    def start_async(self, execute_method: str, *args, **kwargs) -> str:
        if self._async_future is not None:
            raise RuntimeError("Another Elastic EP async method is active")
        if args and isinstance(args[0], ReconfigureDistributedRequest):
            self.reconfig_request = args[0]
        dp_rank = self.worker.vllm_config.parallel_config.data_parallel_rank
        done_key = f"eep_async/{execute_method}/{dp_rank}/{self.worker.rank}"
        self._async_future = self._async_executor.submit(
            self._run_async, execute_method, *args, **kwargs
        )

View on GitHub (pinned to c794754062)

Solutions

1. Keep a strong reference to the worker for the executor's lifetime (the owner that creates the executor should retain the worker)
2. On shutdown, wait for/join the async future (executor._async_future / shutdown of the ThreadPoolExecutor) before dropping worker references
3. In tests, assign the worker to a long-lived variable and shut the executor down before teardown

Example fix

# before
executor = ElasticEPScalingExecutor(make_worker())  # temp worker
executor.start_async("reconfigure", req)  # later: RuntimeError

# after
worker = make_worker()  # strong reference kept by owner
executor = ElasticEPScalingExecutor(worker)
...
executor.shutdown()  # join async work before worker goes out of scope

worker = executor.worker_ref()
if worker is None:
    raise RuntimeError("worker collected; abort before scheduling async work")

def executor_alive(executor: "ElasticEPScalingExecutor") -> bool:
    return executor.worker_ref() is not None

try:
    worker = executor.worker
except RuntimeError:
    # worker gone: cancel pending async work instead of using it
    executor._async_executor.shutdown(wait=False, cancel_futures=True)
    raise

When it happens

Trigger: Creating ElasticEPScalingExecutor(worker) while all other strong references to the worker are dropped (e.g. the constructing scope returns and only the executor survives), then calling execute()/start_async() which touches self.worker; the worker being deliberately torn down during engine shutdown while an async reconfigure is still pending.

Common situations: Engine shutdown races where the async thread pool task outlives the worker; test code constructing the executor with a temporary worker object; refactors that accidentally store the executor globally but not the worker.

Related errors

• Endpoint not ready after {0}s: {1}
• Backend error: {0}
• chat template error: {0}
• Elastic EP is only supported with enable_eplb=True.
• Elastic EP is not supported with pipeline parallelism (pipel

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/d0b70ffc7f7fe992. Report an issue: GitHub.