apache/hadoop · error · IOException

addDirective: you cannot specify an ID for this operation.

Error message

addDirective: you cannot specify an ID for this operation.

What it means

Server-side wrapper for the add-cache-directive RPC: CacheManager assigns directive IDs itself, so the caller-supplied CacheDirectiveInfo must not carry one. Passing an info with a non-null id is treated as client misuse and rejected before any state change or edit-log write.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNDNCacheOp.java:41

import org.apache.hadoop.hdfs.protocol.CacheDirectiveInfo;
import org.apache.hadoop.hdfs.protocol.CachePoolEntry;
import org.apache.hadoop.hdfs.protocol.CachePoolInfo;
import org.apache.hadoop.security.AccessControlException;

import java.io.IOException;
import java.util.EnumSet;

class FSNDNCacheOp {
  static CacheDirectiveInfo addCacheDirective(
      FSNamesystem fsn, CacheManager cacheManager,
      CacheDirectiveInfo directive, EnumSet<CacheFlag> flags,
      boolean logRetryCache)
      throws IOException {

    final FSPermissionChecker pc = getFsPermissionChecker(fsn);

    if (directive.getId() != null) {
      throw new IOException("addDirective: you cannot specify an ID " +
          "for this operation.");
    }
    CacheDirectiveInfo effectiveDirective =
        cacheManager.addDirective(directive, pc, flags);
    fsn.getEditLog().logAddCacheDirectiveInfo(effectiveDirective,
        logRetryCache);
    return effectiveDirective;
  }

  static void modifyCacheDirective(
      FSNamesystem fsn, CacheManager cacheManager, CacheDirectiveInfo directive,
      EnumSet<CacheFlag> flags, boolean logRetryCache) throws IOException {
    final FSPermissionChecker pc = getFsPermissionChecker(fsn);

    cacheManager.modifyDirective(directive, pc, flags);
    fsn.getEditLog().logModifyCacheDirectiveInfo(directive, logRetryCache);
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Build the CacheDirectiveInfo for adds with only path/pool/replication/expiration — never setId
2. If the info came from an existing directive, create a fresh Builder and copy only the mutable fields
3. Use the returned id from addCacheDirective to track the new directive afterwards

Example fix

// before
CacheDirectiveInfo info = new CacheDirectiveInfo.Builder()
    .setId(37L)                      // rejected: IDs are assigned by the NameNode
    .setPath(new Path("/data"))
    .build();
dfs.addCacheDirective(info);

// after
CacheDirectiveInfo info = new CacheDirectiveInfo.Builder()
    .setPath(new Path("/data"))
    .setReplication((short) 1)
    .build();
long id = dfs.addCacheDirective(info);   // ID assigned server-side and returned

if (directive.getId() != null) {
  directive = new CacheDirectiveInfo.Builder(directive)
      .setId(null)   // IDs are assigned by the NameNode on add
      .build();
}
long id = dfs.addCacheDirective(directive);

static boolean isAddableCacheDirective(CacheDirectiveInfo info) {
  return info != null && info.getId() == null;
}

When it happens

Trigger: DFSClient addCacheDirective (DistributedFileSystem.addCacheDirective or 'hdfs dfsadmin -addDirective') with a CacheDirectiveInfo built via setId(...), or a Builder reused from a listCacheDirectives/modify flow that already carries an id.

Common situations: Copy-pasting a modify-directive path into an add path; reusing builder objects populated from existing directives; UI/tools copying a listed directive for re-creation.

Related errors

• unknown flags set in ModifyCacheDirectiveInfoOp: {}
• Unexpected HAServiceStateProto:
• Unrecognized section {}
• Image file is not found in {}
• Edits file is not found in {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fde696b11c7f7a86. Report an issue: GitHub.