{"record":{"id":"40630428edd51052","repo":"apache/hadoop","slug":"did-not-find-requested-id-id-406304","errorCode":null,"errorMessage":"Did not find requested id ${id}","messagePattern":"Did not find requested id (.+?)","errorType":"exception","errorClass":"RemoteException","httpStatus":null,"severity":"warning","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CacheDirectiveIterator.java","lineNumber":113,"sourceCode":"    BatchedEntries<CacheDirectiveEntry> entries;\n    try (TraceScope ignored = tracer.newScope(\"listCacheDirectives\")) {\n      entries = namenode.listCacheDirectives(prevKey, filter);\n    } catch (IOException e) {\n      if (e.getMessage().contains(\"Filtering by ID is unsupported\")) {\n        // Retry case for old servers, do the filtering client-side\n        long id = filter.getId();\n        filter = removeIdFromFilter(filter);\n        // Using id - 1 as prevId should get us a window containing the id\n        // This is somewhat brittle, since it depends on directives being\n        // returned in order of ascending ID.\n        entries = namenode.listCacheDirectives(id - 1, filter);\n        for (int i = 0; i < entries.size(); i++) {\n          CacheDirectiveEntry entry = entries.get(i);\n          if (entry.getInfo().getId().equals(id)) {\n            return new SingleEntry(entry);\n          }\n        }\n        throw new RemoteException(InvalidRequestException.class.getName(),\n            \"Did not find requested id \" + id);\n      }\n      throw e;\n    }\n    Preconditions.checkNotNull(entries);\n    return entries;\n  }\n\n  @Override\n  public Long elementToPrevKey(CacheDirectiveEntry entry) {\n    return entry.getInfo().getId();\n  }\n}\n","sourceCodeStart":95,"sourceCodeEnd":127,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CacheDirectiveIterator.java#L95-L127","documentation":"Thrown by CacheDirectiveIterator when talking to an old NameNode that does not support filtering cache directives by ID. The iterator retries client-side by listing from prevKey = id-1 and scanning the returned batch for the requested id; if the id is not in that window (or the directive no longer exists), it raises RemoteException(InvalidRequestException). It is a fallback-compatibility path, explicitly noted in-code as brittle.","triggerScenarios":"Calling DistributedFileSystem.listCacheDirectives with a CacheDirectiveInfo filter that has setId(...), against a NameNode old enough to answer 'Filtering by ID is unsupported'. The id is not found when: the directive was deleted between listing batches, more than one batch of directives exist after id-1 so the target falls outside the first window, or failover reordered ids.","commonSituations":"Clusters after a downgrade or with pre-HDFS-7309 NameNodes, tooling that lists a specific directive by id, or tests that remove directives concurrently with iteration.","solutions":["Treat the exception as 'directive no longer present' if the directive may have been removed; verify with listCacheDirectives without the id filter.","Upgrade the NameNode to a version that supports server-side ID filtering, which removes this client-side retry path entirely.","Avoid paging through directives while concurrently removing them; snapshot ids first, then list.","If the listing is large, page with explicit prevKey instead of relying on the id-filter single-shot path."],"exampleFix":"// before\nCacheDirectiveInfo filter = new CacheDirectiveInfo.Builder().setId(42L).build();\nRemoteIterator<CacheDirectiveEntry> it = dfs.listCacheDirectives(filter);\nCacheDirectiveEntry e = it.next(); // may throw 'Did not find requested id 42' on old NameNodes\n\n// after\ntry {\n  CacheDirectiveEntry e = it.next();\n} catch (RemoteException re) {\n  if (re.getClassName().equals(InvalidRequestException.class.getName())\n      && re.getMessage().contains(\"Did not find requested id\")) {\n    // directive 42 was deleted or fell outside the client-side scan window\n  } else { throw re; }\n}","handlingStrategy":"try-catch","validationCode":"// Confirm the directive still exists before iterating by id (new NameNodes filter server-side)\nboolean exists = false;\nRemoteIterator<CacheDirectiveEntry> scan = dfs.listCacheDirectives(null);\nwhile (scan.hasNext()) { if (scan.next().getInfo().getId() == id) { exists = true; break; } }","typeGuard":null,"tryCatchPattern":"try {\n  return it.next();\n} catch (RemoteException re) {\n  if (InvalidRequestException.class.getName().equals(re.getClassName())\n      && re.getMessage().contains(\"Did not find requested id\")) {\n    return null; // directive deleted while listing — treat as absent\n  }\n  throw re.unwrapRemoteException();\n}","preventionTips":["Prefer NameNode versions that support server-side id filtering so the brittle client fallback path is never taken.","Do not delete cache directives concurrently with listing them by id.","For large listings, page with explicit prevKey instead of relying on id-filter single-shot semantics."],"tags":["hdfs","caching","pagination","compatibility"],"backgroundTag":"pagination-consistency-error","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}