{"record":{"id":"8980656af0568b49","repo":"apache/druid","slug":"could-not-set-credentials-for-user-s-due-to-conc","errorCode":null,"errorMessage":"Could not set credentials for user[%s] due to concurrent update contention.","messagePattern":"Could not set credentials for user\\[(.+?)\\] due to concurrent update contention\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authentication/db/updater/CoordinatorBasicAuthenticatorMetadataStorageUpdater.java","lineNumber":365,"sourceCode":"    } else {\n      credentials = new BasicAuthenticatorCredentials(update);\n    }\n\n    int attempts = 0;\n    while (attempts < NUM_RETRIES) {\n      if (setUserCredentialOnce(prefix, userName, credentials)) {\n        return;\n      } else {\n        attempts++;\n      }\n      try {\n        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));\n      }\n      catch (InterruptedException ie) {\n        throw new RuntimeException(ie);\n      }\n    }\n    throw new ISE(\"Could not set credentials for user[%s] due to concurrent update contention.\", userName);\n  }\n\n  private boolean createUserOnce(String prefix, String userName)\n  {\n    byte[] oldValue = getCurrentUserMapBytes(prefix);\n    Map<String, BasicAuthenticatorUser> userMap = BasicAuthUtils.deserializeAuthenticatorUserMap(\n        objectMapper,\n        oldValue\n    );\n    if (userMap.get(userName) != null) {\n      throw new BasicSecurityDBResourceException(\"User [%s] already exists.\", userName);\n    } else {\n      userMap.put(userName, new BasicAuthenticatorUser(userName, null));\n    }\n    byte[] newValue = BasicAuthUtils.serializeAuthenticatorUserMap(objectMapper, userMap);\n    return tryUpdateUserMap(prefix, userMap, oldValue, newValue);\n  }\n","sourceCodeStart":347,"sourceCodeEnd":383,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authentication/db/updater/CoordinatorBasicAuthenticatorMetadataStorageUpdater.java#L347-L383","documentation":"Thrown as an IllegalStateException after exhausting all retry attempts while trying to set credentials for a user in the authenticator metadata store. Every compare-and-swap of the serialized user map failed due to concurrent modification. Indicates the credentials update could not be committed within the retry budget.","triggerScenarios":"Calling setUserCredentials (or the startup path from start) for a user while other writers keep changing the same user map so each createUserOnce/setUserCredentialOnce CAS fails across all retries.","commonSituations":"Parallel credential-rotation scripts hitting multiple coordinators; another admin updating users at the same time; coordinator failover during credential update; metadata store under heavy load delaying writes.","solutions":["Ensure only the elected coordinator performs updates (verify leadership/replicated coordinators config)","Serialize credential updates: retry the setUserCredentials call after the failure","Check metadata store health and latency; reduce concurrent writers","Increase numRetries/UPDATE_RETRY_DELAY in the deployment if contention is frequent"],"exampleFix":"// before\nclient.setUserCredentials(authenticatorName, userName, credentials); // ISE on contention\n// after\ntry {\n  client.setUserCredentials(authenticatorName, userName, credentials);\n} catch (IllegalStateException e) {\n  backoffAndRetry(() -> client.setUserCredentials(authenticatorName, userName, credentials));\n}","handlingStrategy":"retry","validationCode":"// verify user exists before setting credentials\nResponse r = client.getUser(authenticatorName, userName);\nif (r.getStatus() != 200) { throw new IllegalStateException(\"create user first\"); }","typeGuard":null,"tryCatchPattern":"try {\n  client.setUserCredentials(authenticatorName, userName, creds);\n} catch (IllegalStateException e) {\n  backoffAndRetry(() -> client.setUserCredentials(authenticatorName, userName, creds));\n}","preventionTips":["Avoid parallel credential rotations against the same authenticator","Verify single-coordinator leadership","Retry with exponential backoff on ISE"],"tags":["concurrency","optimistic-locking","basic-security","credentials"],"backgroundTag":"internal-invariant-violation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}