{"record":{"id":"80dd3e17c3abcdb8","repo":"jaegertracing/jaeger","slug":"failed-to-forfeit-resource-lock-due-to-cassandra-e","errorCode":null,"errorMessage":"failed to forfeit resource lock due to cassandra error: %w","messagePattern":"failed to forfeit resource lock due to cassandra error: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/storage/distributedlock/cassandra/lock.go","lineNumber":70,"sourceCode":"\t\t// The lock was successfully created\n\t\treturn true, nil\n\t}\n\tif owner == l.tenantID {\n\t\t// This host already owns the lock, extend the lease\n\t\tif err = l.extendLease(resource, ttl); err != nil {\n\t\t\treturn false, fmt.Errorf(\"failed to extend lease on resource lock: %w\", err)\n\t\t}\n\t\treturn true, nil\n\t}\n\treturn false, nil\n}\n\n// Forfeit forfeits an existing lease around a given resource.\nfunc (l *Lock) Forfeit(resource string) (bool, error) {\n\tvar name, owner string\n\tapplied, err := l.session.Query(cqlDeleteLock, resource, l.tenantID).ScanCAS(&name, &owner)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"failed to forfeit resource lock due to cassandra error: %w\", err)\n\t}\n\tif applied {\n\t\t// The lock was successfully deleted\n\t\treturn true, nil\n\t}\n\treturn false, fmt.Errorf(\"failed to forfeit resource lock: %w\", errLockOwnership)\n}\n\n// extendLease will attempt to extend the lease of an existing lock on a given resource.\nfunc (l *Lock) extendLease(resource string, ttl time.Duration) error {\n\tttlSec := int(ttl.Seconds())\n\tvar owner string\n\tapplied, err := l.session.Query(cqlUpdateLock, ttlSec, l.tenantID, resource, l.tenantID).ScanCAS(&owner)\n\tif err != nil {\n\t\treturn err\n\t}\n\tif applied {\n\t\treturn nil","sourceCodeStart":52,"sourceCodeEnd":88,"githubUrl":"https://github.com/jaegertracing/jaeger/blob/806f4447841ecdb60519f408b004a599d515f437/internal/storage/distributedlock/cassandra/lock.go#L52-L88","documentation":"Lock.Forfeit wraps Cassandra driver failures from the conditional DELETE (IF owner = ?) used to release a distributed lock. It means the release CAS could not be evaluated — not that ownership was rejected. The underlying gocql error is preserved via %w.","triggerScenarios":"Calling Forfeit(resource) when the cqlDeleteLock ScanCAS fails at the Cassandra level: node down, NoHostAvailable, LWT/SERIAL timeout, keyspace or leases table missing, or auth failure.","commonSituations":"Cassandra outage while a finished job tries to release its lock; network blip between job completion and release; schema never initialized so the leases table doesn't exist.","solutions":["Read the wrapped driver error to identify the Cassandra-level cause.","Check cluster health (nodetool status, credentials, keyspace) and retry Forfeit once connectivity is restored.","Ensure the Jaeger schema (leases table) exists before using the lock.","If the lock holder finished its work, a failed forfeit is safe to retry — the TTL will also eventually expire, freeing the lock.","Add retry with backoff around Forfeit for transient LWT timeouts."],"exampleFix":"// before\nif _, err := lock.Forfeit(\"index-cleaner\"); err != nil {\n    log.Error(\"forfeit failed\", zap.Error(err))\n}\n// after\nerr := retry.WithBackoff(ctx, func() error {\n    _, ferr := lock.Forfeit(\"index-cleaner\")\n    return ferr\n}, 3)\nif err != nil {\n    log.Warn(\"could not forfeit lock; it will expire via TTL\")\n}","handlingStrategy":"retry","validationCode":"// ensure leases table exists before attempting lock operations\nif err := ensureSchema(session, keyspace, \"leases\"); err != nil {\n    return err\n}","typeGuard":null,"tryCatchPattern":"err := retry.WithBackoff(ctx, 3, func() error {\n    _, ferr := lock.Forfeit(resource)\n    return ferr\n})\nif err != nil {\n    // safe: TTL will eventually release the lock\n    log.Printf(\"forfeit failed, relying on TTL: %v\", err)\n}","preventionTips":["Always run the schema init job before lock-dependent components.","Retry forfeits; they are idempotent-safe (the delete is conditional).","Rely on TTL expiry as a safety net when release fails.","Alert on Cassandra availability to catch this class of error early."],"tags":["cassandra","distributed-lock","lwt","release-failure"],"backgroundTag":"distributed-lock-release-failed","analyzedSha":"806f4447841ecdb60519f408b004a599d515f437","analyzedAt":"2026-09-01T02:39:22.140Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T05:18:18.240Z"}