apache/druid · error · IllegalStateException

Could not delete user

Error message

Could not delete user[%s] due to concurrent update contention.

What it means

Thrown as an IllegalStateException after the coordinator exhausts all retry attempts (numRetries) trying to delete a user from the metadata store via compare-and-swap. Every CAS attempt failed because another process concurrently modified the user map between read and write. Indicates persistent optimistic-locking contention on the shared user map in metadata storage.

Solutions

1. Check that only one coordinator is the leader and extra coordinators are not serving updates
2. Reduce concurrent user-management API calls against the same authenticator and retry the delete after contention subsides
3. Verify the metadata store is healthy and responsive; tune coordinator concurrency configs
4. Increase retry behavior (numRetries / UPDATE_RETRY_DELAY) or upgrade Druid if contention is a known issue

Example fix

// before
client.deleteUser(authenticatorName, userName); // ISE under contention
// after
try {
  client.deleteUser(authenticatorName, userName);
} catch (IllegalStateException e) {
  Thread.sleep(retryBackoffMs);
  client.deleteUser(authenticatorName, userName);
}

// before deleting, confirm user exists and quiesce other writers
Response r = client.getUser(authenticatorName, userName);
if (r.getStatus() != 200) { throw new IllegalStateException("user missing, nothing to delete"); }

try {
  client.deleteUser(authenticatorName, userName);
} catch (IllegalStateException e) {
  // contention: back off and retry a bounded number of times
  backoffAndRetry(() -> client.deleteUser(authenticatorName, userName));
}

When it happens

Trigger: Calling deleteUser (CoordinatorBasicAuthenticatorMetadataStorageUpdater) while other nodes or the coordinator itself repeatedly update the same authenticator user map, so every tryUpdateUserMap CAS within the retry window fails.

Common situations: Multiple coordinator leader transitions or several coordinators misconfigured simultaneously; bulk scripted user deletion from different clients; very high user-change API traffic against a single authenticator; slow metadata store (e.g. overloaded MySQL/PostgreSQL metadata store) making each retry window too short.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Could not create user
• Could not delete user
• Could not set credentials for user
• Could not assign role
• Could not assign role

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/31db6d09aad2adbd. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authentication/db/updater/CoordinatorBasicAuthenticatorMetadataStorageUpdater.java:319

        attempts++;
      }
      updateRetryDelay();
    }
    throw new ISE("Could not create user[%s] due to concurrent update contention.", userName);
  }

  private void deleteUserInternal(String prefix, String userName)
  {
    int attempts = 0;
    while (attempts < NUM_RETRIES) {
      if (deleteUserOnce(prefix, userName)) {
        return;
      } else {
        attempts++;
      }
      updateRetryDelay();
    }
    throw new ISE("Could not delete user[%s] due to concurrent update contention.", userName);
  }

  private void updateRetryDelay()
  {
    try {
      Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
    }
    catch (InterruptedException ie) {
      throw new RuntimeException(ie);
    }
  }

  private void setUserCredentialsInternal(String prefix, String userName, BasicAuthenticatorCredentialUpdate update)
  {
    BasicAuthenticatorCredentials credentials;

    // use default iteration count from Authenticator if not specified in request
    if (update.getIterations() == -1) {

View on GitHub (pinned to 9b90983fd2)