apache/druid · error · IllegalStateException

Could not set credentials for user

Error message

Could not set credentials for user[%s] due to concurrent update contention.

What it means

Thrown as an IllegalStateException after exhausting all retry attempts while trying to set credentials for a user in the authenticator metadata store. Every compare-and-swap of the serialized user map failed due to concurrent modification. Indicates the credentials update could not be committed within the retry budget.

Solutions

1. Ensure only the elected coordinator performs updates (verify leadership/replicated coordinators config)
2. Serialize credential updates: retry the setUserCredentials call after the failure
3. Check metadata store health and latency; reduce concurrent writers
4. Increase numRetries/UPDATE_RETRY_DELAY in the deployment if contention is frequent

Example fix

// before
client.setUserCredentials(authenticatorName, userName, credentials); // ISE on contention
// after
try {
  client.setUserCredentials(authenticatorName, userName, credentials);
} catch (IllegalStateException e) {
  backoffAndRetry(() -> client.setUserCredentials(authenticatorName, userName, credentials));
}

// verify user exists before setting credentials
Response r = client.getUser(authenticatorName, userName);
if (r.getStatus() != 200) { throw new IllegalStateException("create user first"); }

try {
  client.setUserCredentials(authenticatorName, userName, creds);
} catch (IllegalStateException e) {
  backoffAndRetry(() -> client.setUserCredentials(authenticatorName, userName, creds));
}

When it happens

Trigger: Calling setUserCredentials (or the startup path from start) for a user while other writers keep changing the same user map so each createUserOnce/setUserCredentialOnce CAS fails across all retries.

Common situations: Parallel credential-rotation scripts hitting multiple coordinators; another admin updating users at the same time; coordinator failover during credential update; metadata store under heavy load delaying writes.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Could not create user
• Could not delete user
• Could not delete user
• Could not assign role
• Could not assign role

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/8980656af0568b49. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authentication/db/updater/CoordinatorBasicAuthenticatorMetadataStorageUpdater.java:365

    } else {
      credentials = new BasicAuthenticatorCredentials(update);
    }

    int attempts = 0;
    while (attempts < NUM_RETRIES) {
      if (setUserCredentialOnce(prefix, userName, credentials)) {
        return;
      } else {
        attempts++;
      }
      try {
        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
      }
      catch (InterruptedException ie) {
        throw new RuntimeException(ie);
      }
    }
    throw new ISE("Could not set credentials for user[%s] due to concurrent update contention.", userName);
  }

  private boolean createUserOnce(String prefix, String userName)
  {
    byte[] oldValue = getCurrentUserMapBytes(prefix);
    Map<String, BasicAuthenticatorUser> userMap = BasicAuthUtils.deserializeAuthenticatorUserMap(
        objectMapper,
        oldValue
    );
    if (userMap.get(userName) != null) {
      throw new BasicSecurityDBResourceException("User [%s] already exists.", userName);
    } else {
      userMap.put(userName, new BasicAuthenticatorUser(userName, null));
    }
    byte[] newValue = BasicAuthUtils.serializeAuthenticatorUserMap(objectMapper, userMap);
    return tryUpdateUserMap(prefix, userMap, oldValue, newValue);
  }

View on GitHub (pinned to 9b90983fd2)