{"record":{"id":"13b1a4030a5fc09e","repo":"HangfireIO/Hangfire","slug":"timeout-expired-the-timeout-elapsed-prior-to-obta","errorCode":null,"errorMessage":"Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the '{resource}' resource.","messagePattern":"Timeout expired\\. The timeout elapsed prior to obtaining a distributed lock on the '(.+?)' resource\\.","errorType":"exception","errorClass":"DistributedLockTimeoutException","httpStatus":null,"severity":"warning","filePath":"src/Hangfire.SqlServer/SqlServerDistributedLock.cs","lineNumber":220,"sourceCode":"\n                command.ExecuteNonQuery();\n\n                var lockResult = (int)resultParameter.Value;\n\n                if (lockResult >= 0)\n                {\n                    // The lock has been successfully obtained on the specified resource.\n                    return;\n                }\n\n                if (lockResult == -999 /* Indicates a parameter validation or other call error. */)\n                {\n                    throw new SqlServerDistributedLockException(\n                        $\"Could not place a lock on the resource '{resource}': {(LockErrorMessages.TryGetValue(lockResult, out var message) ? message : $\"Server returned the '{lockResult}' error.\")}.\");\n                }\n            } while (started.Elapsed < timeout);\n\n            throw new DistributedLockTimeoutException(resource);\n        }\n\n        internal static void Release(DbConnection connection, string resource)\n        {\n            using (var command = CreateReleaseCommand(connection, resource, out var resultParameter))\n            {\n                command.ExecuteNonQuery();\n\n                var releaseResult = (int)resultParameter.Value;\n\n                if (releaseResult < 0)\n                {\n                    throw new SqlServerDistributedLockException(\n                        $\"Could not release a lock on the resource '{resource}': Server returned the '{releaseResult}' error.\");\n                }\n            }\n        }\n","sourceCodeStart":202,"sourceCodeEnd":238,"githubUrl":"https://github.com/HangfireIO/Hangfire/blob/c236dd0f930f831ec151e436e138ddc429a02a72/src/Hangfire.SqlServer/SqlServerDistributedLock.cs#L202-L238","documentation":"Hangfire exhausted the configured timeout while polling for a SQL Server application lock because another process continuously held it. The exception is DistributedLockTimeoutException (derived from TimeoutException) and is expected under lock contention. Hangfire itself catches this in several internal services (ExpirationManager, DelayedJobScheduler, RecurringJobScheduler) and treats it as non-fatal.","triggerScenarios":"Multiple Hangfire server instances or background workers competing for the same lock resource simultaneously; a long-running maintenance query (large expiration batch, heavy counters aggregation) holding the lock longer than the timeout; database under heavy load causing each sp_getapplock attempt (1-second LockTimeout per iteration) to fail within the overall timeout window.","commonSituations":"Scaling out to many server instances all hitting the same Hangfire database; large job tables making expiration sweeps slow; SQL Azure throttling or resource governor connection termination; deadlocks on the lock resource during peak load.","solutions":["Reduce the number of Hangfire server instances competing for the same database, or shard across multiple databases.","Optimize the Hangfire schema: run schema migrations, enable UseIgnoreDupKeyOption, increase DeleteExpiredBatchSize to speed up expiration sweeps.","Monitor SQL Server for blocking sessions and long-running queries during the timeout window.","If calling distributed-lock APIs directly, catch DistributedLockTimeoutException and retry with backoff or skip gracefully as Hangfire's own services do."],"exampleFix":"// before — unhandled timeout propagates and crashes the operation\nusing (storage.GetConnection().AcquireDistributedLock(\"my-lock\", TimeSpan.FromSeconds(5)))\n{\n    DoWork();\n}\n// after — catch and retry/skip gracefully\ntry\n{\n    using (storage.GetConnection().AcquireDistributedLock(\"my-lock\", TimeSpan.FromSeconds(5)))\n    {\n        DoWork();\n    }\n}\ncatch (DistributedLockTimeoutException)\n{\n    // Lock is held by another instance; skip this cycle.\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"int attempts = 0;\nwhile (true)\n{\n    try\n    {\n        using (storage.GetConnection().AcquireDistributedLock(resource, TimeSpan.FromSeconds(5)))\n        {\n            DoWork();\n            return;\n        }\n    }\n    catch (DistributedLockTimeoutException) when (attempts++ < maxRetries)\n    {\n        Thread.Sleep(TimeSpan.FromSeconds(backoffSeconds));\n    }\n}","preventionTips":["Limit the number of Hangfire server instances sharing a single database to reduce lock contention.","Keep expiration and aggregation batches fast: enable schema optimizations and tune batch sizes.","Treat DistributedLockTimeoutException as recoverable — Hangfire's own services (ExpirationManager, RecurringJobScheduler, DelayedJobScheduler) already swallow it internally.","Monitor SQL Server for blocking chains during peak load."],"tags":["distributed-lock","sql-server","timeout","contention","sp-getapplock"],"backgroundTag":null,"analyzedSha":"c236dd0f930f831ec151e436e138ddc429a02a72","analyzedAt":"2026-08-13T20:27:11.027Z","schemaVersion":2},"datasetVersion":"2026-08-14T00:17:13.853Z"}