apache/hadoop · error · IOException

Key ${name} already exists in ${this}

Error message

Key ${name} already exists in ${this}

What it means

UserProvider stores keys in the current user's UGI credentials (an in-memory credential store). createKey refuses when credentials.getSecretKey(name) already returns data, because each key name maps to one entry in that credential store. This mirrors the duplicate check JavaKeyStoreProvider performs against its keystore.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/UserProvider.java:87

  public synchronized Metadata getMetadata(String name) throws IOException {
    if (cache.containsKey(name)) {
      return cache.get(name);
    }
    byte[] serialized = credentials.getSecretKey(new Text(name));
    if (serialized == null) {
      return null;
    }
    Metadata result = new Metadata(serialized);
    cache.put(name, result);
    return result;
  }

  @Override
  public synchronized KeyVersion createKey(String name, byte[] material,
                               Options options) throws IOException {
    Text nameT = new Text(name);
    if (credentials.getSecretKey(nameT) != null) {
      throw new IOException("Key " + name + " already exists in " + this);
    }
    if (options.getBitLength() != 8 * material.length) {
      throw new IOException("Wrong key length. Required " +
          options.getBitLength() + ", but got " + (8 * material.length));
    }
    Metadata meta = new Metadata(options.getCipher(), options.getBitLength(),
        options.getDescription(), options.getAttributes(), new Date(), 1);
    cache.put(name, meta);
    String versionName = buildVersionName(name, 0);
    credentials.addSecretKey(nameT, meta.serialize());
    credentials.addSecretKey(new Text(versionName), material);
    return new KeyVersion(name, versionName, material);
  }

  @Override
  public synchronized void deleteKey(String name) throws IOException {
    Metadata meta = getMetadata(name);
    if (meta == null) {

View on GitHub (pinned to 2add963021)

Solutions

1. When the key already exists and you want fresh material, call rollNewVersion(name) instead
2. Call deleteKey(name) first if you truly intend to replace the key
3. Guard with getMetadata(name) == null before createKey

Example fix

// before
provider.createKey(name, material, options); // second time -> throws

// after
if (provider.getMetadata(name) != null) {
  provider.rollNewVersion(name, material);
} else {
  provider.createKey(name, material, options);
}

if (provider.getMetadata(name) == null) {
  provider.createKey(name, material, options);
} else {
  provider.rollNewVersion(name, material);
}

try { provider.createKey(name, material, options); } catch (IOException e) { if (String.valueOf(e.getMessage()).contains("already exists")) { provider.rollNewVersion(name, material); } else { throw e; } }

When it happens

Trigger: Calling createKey twice with the same name within one UGI session; retrying a request after a partial failure that already added the credential; test suites sharing a UserGroupInformation across cases.

Common situations: Retried job submissions; long-lived daemons that create keys on every request; integration tests that do not reset credentials between runs.

Related errors

• Wrong key length. Required ${bitLength}, but got ${actualBit
• Wrong key length. Required ${options.getBitLength()}, but go
• Key ${name} does not exist in ${this}
• Problem removing ${versionName} from ${this}
• Problem removing ${name} from ${this}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a777eefe0ad96ec3. Report an issue: GitHub.