{"record":{"id":"fbbd1b88cafc97c4","repo":"hashicorp/nomad","slug":"no-driver-handle","errorCode":null,"errorMessage":"no driver handle","messagePattern":"no driver handle","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/taskrunner/lazy_handle.go","lineNumber":105,"sourceCode":"\tfor i := range retrieveFailureLimit {\n\t\tl.h = l.retrieveHandle()\n\t\tif l.h != nil {\n\t\t\treturn l.h, nil\n\t\t}\n\n\t\t// Calculate the new backoff\n\t\tbackoff := min((1<<(2*uint64(i)))*retrieveBackoffBaseline, retrieveBackoffLimit)\n\n\t\tl.logger.Debug(\"failed to retrieve handle\", \"backoff\", backoff)\n\n\t\tselect {\n\t\tcase <-l.shutdownCtx.Done():\n\t\t\treturn nil, l.shutdownCtx.Err()\n\t\tcase <-time.After(backoff):\n\t\t}\n\t}\n\n\treturn nil, fmt.Errorf(\"no driver handle\")\n}\n\nfunc (l *LazyHandle) Exec(timeout time.Duration, cmd string, args []string) ([]byte, int, error) {\n\th, err := l.getHandle()\n\tif err != nil {\n\t\treturn nil, 0, err\n\t}\n\n\t// Only retry once\n\tfirst := true\n\nTRY:\n\tout, c, err := h.Exec(timeout, cmd, args)\n\tif err == bstructs.ErrPluginShutdown && first {\n\t\tfirst = false\n\n\t\th, err = l.refreshHandle()\n\t\tif err == nil {","sourceCodeStart":87,"sourceCodeEnd":123,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/taskrunner/lazy_handle.go#L87-L123","documentation":"LazyHandle wraps driver handle acquisition; refreshHandleLocked retries creating a driver handle with backoff until the shutdown context is canceled, and if it exhausts (shutdown) or otherwise never obtains a handle it returns the sentinel 'no driver handle'. Any LazyHandle operation (Exec, stats, etc.) surfaces this when the underlying driver plugin cannot be (re)started, e.g. after the task has exited or the driver is unavailable.","triggerScenarios":"getHandle→refreshHandleLocked cannot create a driver handle: driver plugin crashed/was reattached and the task no longer exists, driver not running on the client, or shutdownCtx already canceled during client shutdown — then Exec/Stats/etc. return 'no driver handle'","commonSituations":"Calling nomad alloc exec against a task whose driver handle was lost after a client agent restart, docker driver not running or failing to reattach, querying an already-completed/dead allocation","solutions":["Check the allocation state — if the alloc/task is dead, re-run against a live allocation or resubmit the job","Verify the driver (e.g. docker) is healthy on the client (nomad node status -verbose, driver health checks)","Restart the nomad client agent or the specific driver plugin so handles can be recreated","If seen transiently during client shutdown, simply retry after the client is back up"],"exampleFix":"// before (CLI against dead alloc)\nnomad alloc exec <dead-alloc-id> ls /\n// after\nnomad alloc status <alloc-id>   # confirm alloc is running\nnomad alloc exec <running-alloc-id> ls /","handlingStrategy":"retry","validationCode":"// check alloc/task liveness and driver health first\nnomad alloc status <alloc-id>      # must be running\nnomad node status -verbose <node>  # driver must be healthy","typeGuard":null,"tryCatchPattern":"// retry handle acquisition with backoff, bail on shutdown\nfor {\n    h, err := lh.getHandle()\n    if err == nil { return h }\n    select {\n    case <-ctx.Done(): return ctx.Err()\n    case <-time.After(backoff):\n    }\n}","preventionTips":["Confirm allocation is running before Exec/Stats calls","Keep driver plugins (docker etc.) healthy on clients; enable driver health checks","Expect this error during client shutdown — serialize ops with shutdownCtx","After client restarts, re-check task liveness before issuing commands"],"tags":["driver","task-runtime","handle"],"backgroundTag":"no-driver-handle","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}