{"record":{"id":"55f93598bf202b9d","repo":"apache/hadoop","slug":"can-t-check-existence-of-next","errorCode":null,"errorMessage":"Can't check existence of ${next}","messagePattern":"Can't check existence of (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java","lineNumber":658,"sourceCode":"      }\n\n      private void advance() throws IOException {\n        while (i < rootDirs.length) {\n          next = new Path(rootDirs[i++], pathStr);\n          if (fs.exists(next)) {\n            return;\n          }\n        }\n        next = null;\n      }\n\n      @Override\n      public Path next() {\n        final Path result = next;\n        try {\n          advance();\n        } catch (IOException ie) {\n          throw new RuntimeException(\"Can't check existence of \" + next, ie);\n        }\n        if (result == null) {\n          throw new NoSuchElementException();\n        }\n        return result;\n      }\n\n      @Override\n      public void remove() {\n        throw new UnsupportedOperationException(\"read only iterator\");\n      }\n\n      @Override\n      public Iterator<Path> iterator() {\n        return this;\n      }\n    }\n","sourceCodeStart":640,"sourceCodeEnd":676,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java#L640-L676","documentation":"Thrown by the lazy iterator returned from LocalDirAllocator.getAllLocalPathsToRead(). PathIterator.next() calls fs.exists() on each candidate path under the configured local directories; because Iterator.next() cannot throw checked exceptions, any IOException from that existence check (permissions, vanished mount, disk failure) is wrapped in this RuntimeException.","triggerScenarios":"Iterating the Iterable<Path> from LocalDirAllocator#getAllLocalPathsToRead(String, Configuration) while LocalFileSystem.exists() throws on one of the configured local dirs (context key such as mapreduce.cluster.local.dir / yarn.nodemanager.local-dirs): unreadable directory, unmounted disk, permission change, or I/O error on the volume.","commonSituations":"Node-local disks failing or full, cleanup scripts deleting configured dir roots, ownership/permission changes after process start, NFS-backed local dirs hiccuping mid-iteration.","solutions":["Unwrap the RuntimeException's IOException cause and fix the local directory named in the message (permissions, mount, health)","Verify every dir listed in the configured key exists and is readable by the process user before running the job","Repair/free the failing disk or remount the volume, then rerun","Wrap iteration in try/catch, log the failing dir, and degrade gracefully instead of failing the whole job"],"exampleFix":"// before\nfor (Path p : alloc.getAllLocalPathsToRead(pathStr, conf)) { process(p); }\n// after\ntry {\n  for (Path p : alloc.getAllLocalPathsToRead(pathStr, conf)) { process(p); }\n} catch (RuntimeException e) {\n  if (e.getCause() instanceof IOException) {\n    LOG.warn(\"local dir existence check failed: \" + e.getCause());\n  } else { throw e; }\n}","handlingStrategy":"try-catch","validationCode":"for (String d : conf.getTrimmedStrings(\"yarn.nodemanager.local-dirs\")) {\n  File f = new File(d);\n  if (!f.isDirectory() || !f.canRead()) {\n    throw new IOException(\"local dir unusable: \" + d);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  for (Path p : alloc.getAllLocalPathsToRead(pathStr, conf)) { /* ... */ }\n} catch (RuntimeException e) {\n  if (e.getCause() instanceof IOException) {\n    LOG.warn(\"local dir existence check failed: {}\", e.getCause());\n  } else { throw e; }\n}","preventionTips":["Pre-create and chmod all configured local dirs before launching jobs","Monitor disk health and mounts of node-local dirs","Avoid NFS-backed local dirs for allocator roots","Never let cleanup scripts delete the configured dir roots"],"tags":["local-dirs","iterator","runtimeexception","io","disk"],"backgroundTag":"local-dir-unavailable","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}