apache/druid · error · IllegalStateException

Could not delete role [%s] due to concurrent update contenti

Error message

Could not delete role [%s] due to concurrent update contention.

What it means

Deletion of a basic authorizer role failed because all numRetries compare-and-swap attempts on the authorization metadata maps were beaten by concurrent writers. The updater re-reads and re-applies the role removal on each retry; persistent failure raises this ISE with the role name. It reflects update contention in metadata storage rather than a missing role.

Source

Thrown at extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authorization/db/updater/CoordinatorBasicAuthorizerMetadataStorageUpdater.java:764

  }

  private void deleteRoleInternal(String prefix, String roleName)
  {
    int attempts = 0;
    while (attempts < numRetries) {
      if (deleteRoleOnce(prefix, roleName)) {
        return;
      } else {
        attempts++;
      }
      try {
        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
      }
      catch (InterruptedException ie) {
        throw new RuntimeException(ie);
      }
    }
    throw new ISE("Could not delete role [%s] due to concurrent update contention.", roleName);
  }

  private void assignUserRoleInternal(String prefix, String userName, String roleName)
  {
    int attempts = 0;
    while (attempts < numRetries) {
      if (assignUserRoleOnce(prefix, userName, roleName)) {
        return;
      } else {
        attempts++;
      }
      try {
        Thread.sleep(ThreadLocalRandom.current().nextLong(UPDATE_RETRY_DELAY));
      }
      catch (InterruptedException ie) {
        throw new RuntimeException(ie);
      }
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

Retry deleteRole after a short backoff.
Ensure single-writer semantics: only the elected coordinator should mutate basic-security metadata.
Serialize role cleanup across tooling and environments.
Increase numRetries if concurrent admin operations are expected.
Check metadata storage health and latency.

Example fix

// before
client.deleteRole(prefix, roleName);
// after
for (int i = 0; i < 5; i++) {
  try { client.deleteRole(prefix, roleName); return; }
  catch (IJSE e) { Thread.sleep(1000); }
}
throw new IllegalStateException("deleteRole still contended: " + roleName);

Defensive patterns

Strategy: retry

Validate before calling

// Confirm role exists under this prefix before deleting
Map<String, BasicAuthorizerRole> roles =
    BasicAuthUtils.deserializeAuthorizerRoleMap(mapper, getCurrentRoleMapBytes(prefix));
if (!roles.containsKey(roleName)) return;

Try / catch

try {
  updater.deleteRole(prefix, roleName);
} catch (IJSE e) {
  await.atMost(Duration.ofSeconds(10)).untilAsserted(
      () -> updater.deleteRole(prefix, roleName));
}

Prevention

Avoid parallel role cleanup jobs
Single coordinator writer only
Add backoff between delete attempts
Increase numRetries if needed
Monitor CAS failure rates via logs

When it happens

Trigger: Calling deleteRole while other clients continuously mutate the same authorizer prefix's metadata, so tryUpdateGroupMappingMap CAS never succeeds within the retry budget.

Common situations: Parallel role cleanup by multiple automation jobs; role deletion racing with permission updates or user-role unassignments; multi-writer misconfiguration after coordinator failover.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/e22cc2d1fa14be49. Report an issue: GitHub.