{"record":{"id":"17acaf99f0028033","repo":"hashicorp/nomad","slug":"error-renewing-the-lease-w","errorCode":null,"errorMessage":"error renewing the lease: %w","messagePattern":"error renewing the lease: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"api/locks.go","lineNumber":332,"sourceCode":"\n\t\t\t// Execute the lock protected function.\n\t\t\tgo func() {\n\t\t\t\tdefer funcCancel()\n\t\t\t\tfor _, f := range protectedFuncs {\n\t\t\t\t\terr := f(funcCtx)\n\t\t\t\t\tif err != nil {\n\t\t\t\t\t\terrChannel <- fmt.Errorf(\"error executing protected function %w\", err)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\tcancel()\n\t\t\t\t}\n\t\t\t}()\n\n\t\t\t// Maintain lease is a blocking function, it will return if there is\n\t\t\t// an error maintaining the lease or the protected function returned.\n\t\t\terr = ll.maintainLease(funcCtx)\n\t\t\tif err != nil && !errors.Is(err, ErrLockConflict) {\n\t\t\t\terrChannel <- fmt.Errorf(\"error renewing the lease: %w\", err)\n\t\t\t}\n\t\t}\n\n\t\twaitTicker.Stop()\n\t\twaitTicker = time.NewTicker(ll.waitPeriod)\n\t\tselect {\n\t\tcase <-ctx.Done():\n\t\t\treturn nil\n\n\t\tcase err := <-errChannel:\n\t\t\treturn fmt.Errorf(\"locks: %w\", err)\n\n\t\tcase <-waitTicker.C:\n\t\t}\n\t}\n}\n\nfunc (ll *LockLeaser) maintainLease(ctx context.Context) error {","sourceCodeStart":314,"sourceCodeEnd":350,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/api/locks.go#L314-L350","documentation":"This error is produced by LockLeaser's internal start when maintainLease fails to renew the lock with an error other than ErrLockConflict. It means the lease-maintenance heartbeat (Renew) failed for a non-conflict reason — e.g. network or API errors — so the library cannot guarantee the lock is still held. ErrLockConflict renewals are deliberately filtered out and not reported through this message.","triggerScenarios":"maintainLease's periodic locker.Renew call returns a non-ErrLockConflict error: server 500s, connection refused/timeouts, invalid TTL configuration causing retryPut to exhaust its retries, or the client losing connectivity mid-lease.","commonSituations":"Consul/API server outage or restart while the leaser holds a lock; network partition between the client and server; misconfigured Variable.Lock.TTL string (e.g. unparseable duration) leading to failed calls; load balancer dropping long-lived connections during renewal; retry limits (defaultNumberOfRetries) exhausted under transient errors.","solutions":["Inspect the wrapped cause with errors.Is/As on the error from Start (prefix 'error renewing the lease:') — any surfaced cause is a transport/API error, since ErrLockConflict is filtered out earlier.","Check connectivity to the server and retry; the internal exponential retry already retries until 5 attempts or TTL expiry, so persistent causes need infrastructure fixes.","Validate the lock TTL is a parseable duration and large enough that renewals are not starved by retries.","If the error stems from context cancellation, cancel the parent context yourself so the shutdown path is clean instead of letting Renew fail unexpectedly.","For lost-lock semantics, rely on errors.Is(err, api.ErrLockConflict) handling; this specific error indicates non-conflict failures."],"exampleFix":"// before\nif err := leaser.Start(ctx, work); err != nil {\n    log.Fatal(err)\n}\n// after\nif err := leaser.Start(ctx, work); err != nil {\n    var netErr net.Error\n    if errors.As(err, &netErr) {\n        log.Println(\"transient lease renewal failure, restarting leaser\")\n        restartLeaserWithBackoff()\n    } else {\n        log.Fatal(err)\n    }\n}","handlingStrategy":"retry","validationCode":"// preflight: validate TTL config and server reachability before starting the leaser\nif _, err := time.ParseDuration(v.Lock.TTL); err != nil {\n    return fmt.Errorf(\"invalid lock TTL %q: %w\", v.Lock.TTL, err)\n}\nif err := pingServer(ctx); err != nil {\n    return fmt.Errorf(\"server unreachable before starting leaser: %w\", err)\n}","typeGuard":"func IsLeaseRenewalFailure(err error) bool {\n    return err != nil && strings.Contains(err.Error(), \"error renewing the lease\")\n}","tryCatchPattern":"err := leaser.Start(ctx, work)\nif err != nil {\n    if strings.Contains(err.Error(), \"error renewing the lease\") {\n        // transient transport/API failure: restart the leaser with backoff\n        return restartWithBackoff(ctx)\n    }\n    return err\n}","preventionTips":["Ensure network reliability to the API server; renewal heartbeats run every 0.7*TTL","Set a TTL that absorbs transient blips and the client's 5-attempt retry window","Monitor the server's health; restarts/outages surface as renewal failures","Filter context cancellation in your own code so shutdowns don't masquerade as renewal errors","Re-acquire the lock and resume work after transient renewal failures rather than crashing"],"tags":["go","consul","distributed-lock","lease-renewal","network"],"backgroundTag":"lease-renewal-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}