apache/druid · error · IllegalStateException

Could not assign role [%s] to user [%s] due to concurrent up

Error message

Could not assign role [%s] to user [%s] due to concurrent update contention.

What it means

Assigning a role to a user failed after exhausting numRetries compare-and-swap attempts because concurrent writers kept changing the user/role maps first. Each retry re-reads the user map, re-adds the role to the user's roles set, and attempts the CAS; persistent loss throws this ISE naming the role and user. It is a contention failure, not evidence that the user or role is invalid.

Source

Thrown at extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authorization/db/updater/CoordinatorBasicAuthorizerMetadataStorageUpdater.java:783

  }

  private void assignUserRoleInternal(String prefix, String userName, String roleName)
  {
    int attempts = 0;
    while (attempts < numRetries) {
      if (assignUserRoleOnce(prefix, userName, roleName)) {
        return;
      } else {
        attempts++;
      }
      try {
        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
      }
      catch (InterruptedException ie) {
        throw new RuntimeException(ie);
      }
    }
    throw new ISE("Could not assign role [%s] to user [%s] due to concurrent update contention.", roleName, userName);
  }

  private void unassignUserRoleInternal(String prefix, String userName, String roleName)
  {
    int attempts = 0;
    while (attempts < numRetries) {
      if (unassignUserRoleOnce(prefix, userName, roleName)) {
        return;
      } else {
        attempts++;
      }
      try {
        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
      }
      catch (InterruptedException ie) {
        throw new RuntimeException(ie);
      }
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

Retry assignUserRole after a delay.
Restrict basic-security metadata writes to a single leader/process.
Stagger or serialize bulk user-role assignment operations.
Raise numRetries for bursty concurrent update workloads.
Verify metadata storage latency and connectivity.

Example fix

// before
assignments.forEach(a -> client.assignUserRole(prefix, a.user, a.role));
// after: serialized with retry
assignments.forEach(a ->
  RetryUtils.retry(() -> client.assignUserRole(prefix, a.user, a.role),
                   e -> e instanceof IllegalStateException, MAX_ATTEMPTS));

Defensive patterns

Strategy: retry

Validate before calling

// Verify user and role exist before assignment
Map<String, BasicAuthorizerUser> users =
    BasicAuthUtils.deserializeAuthorizerUserMap(mapper, getCurrentUserMapBytes(prefix));
Map<String, BasicAuthorizerRole> roles =
    BasicAuthUtils.deserializeAuthorizerRoleMap(mapper, getCurrentRoleMapBytes(prefix));
if (users.get(userName) == null || roles.get(roleName) == null)
  throw new IllegalArgumentException("user or role missing");

Try / catch

try {
  updater.assignUserRole(prefix, userName, roleName);
} catch (IJSE e) {
  RetryUtils.retry(() -> updater.assignUserRole(prefix, userName, roleName),
                   ex -> ex instanceof IllegalStateException, MAX_ATTEMPTS);
}

Prevention

Stagger bulk user-role assignments
Only the leader writes metadata
Use randomized backoff on retries
Raise numRetries for concurrent workloads
Pre-validate user and role existence

When it happens

Trigger: Calling assignUserRole while other writers repeatedly mutate authorization metadata for the same prefix so every tryUpdateUserMap/role-map CAS fails through all retries.

Common situations: Bulk user onboarding scripts assigning roles in parallel; simultaneous role assignment and group mapping updates; multiple coordinators acting as writers.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/0dd841bf5564e8b9. Report an issue: GitHub.