{"record":{"id":"460be7963c9ca571","repo":"apache/hadoop","slug":"cannot-getlocatedblocks-through-a-symlink-to-a-non","errorCode":null,"errorMessage":"Cannot getLocatedBlocks through a symlink to a non-DistributedFileSystem: {} -> {}","messagePattern":"Cannot getLocatedBlocks through a symlink to a non-DistributedFileSystem: (.+?) -> (.+?)","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java","lineNumber":4112,"sourceCode":"   * @return a LocatedBlocks object\n   * @throws IOException\n   */\n  public LocatedBlocks getLocatedBlocks(Path p, long start, long len)\n      throws IOException {\n    final Path absF = fixRelativePart(p);\n    return new FileSystemLinkResolver<LocatedBlocks>() {\n      @Override\n      public LocatedBlocks doCall(final Path p) throws IOException {\n        return dfs.getLocatedBlocks(getPathName(p), start, len);\n      }\n      @Override\n      public LocatedBlocks next(final FileSystem fs, final Path p)\n          throws IOException {\n        if (fs instanceof DistributedFileSystem) {\n          DistributedFileSystem myDfs = (DistributedFileSystem)fs;\n          return myDfs.getLocatedBlocks(p, start, len);\n        }\n        throw new UnsupportedOperationException(\"Cannot getLocatedBlocks \" +\n            \"through a symlink to a non-DistributedFileSystem: \" + fs + \" -> \"+\n            p);\n      }\n    }.resolve(this, absF);\n  }\n\n  /**\n   * Return path of the enclosing root for a given path\n   * The enclosing root path is a common ancestor that should be used for temp and staging dirs\n   * as well as within encryption zones and other restricted directories.\n   *\n   * @param path file path to find the enclosing root path for\n   * @return a path to the enclosing root\n   * @throws IOException early checks like failure to resolve path cause IO failures\n   */\n  public Path getEnclosingRoot(final Path path) throws IOException {\n    statistics.incrementReadOps(1);\n    storageStatistics.incrementOpCounter(OpType.GET_ENCLOSING_ROOT);","sourceCodeStart":4094,"sourceCodeEnd":4130,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java#L4094-L4130","documentation":"DistributedFileSystem.getLocatedBlocks resolves the path through FileSystemLinkResolver; when the final hop of a symlink lands in a FileSystem that is not a DistributedFileSystem (local, s3a, viewfs target, etc.), it cannot fetch HDFS LocatedBlocks and throws UnsupportedOperationException. It is an inherent capability boundary: block-location queries only make sense inside HDFS. The message prints the target filesystem and path.","triggerScenarios":"Calling an API that needs block locations (getFileBlockLocations, or internal getLocatedBlocks used by checksums/tools) on an HDFS path that is a symlink whose target resolves against another FileSystem scheme; symlink chains that leave HDFS via ViewFileSystem mounts or har:// / local targets.","commonSituations":"A user-created HDFS symlink pointing at a mount table entry or non-HDFS URI; data-lake layouts mixing hdfs:// and object-store mounts behind symlinks; tools (Spark locality, DistCp, checksum utilities) following symlinks across filesystem boundaries.","solutions":["Resolve the symlink first (FileContext.getFileStatus or FileStatus.getSymlink traversal) and only call block-location APIs when the resolved FS is a DistributedFileSystem","Fix or remove the HDFS symlink so it points to a real HDFS path if block-level operations are required","Use the target filesystem's own API for non-HDFS targets instead of HDFS block-location calls"],"exampleFix":"// before\nLocatedBlocks lbs = ((DistributedFileSystem) fs)\n    .getLocatedBlocks(symlinkPath, 0, Long.MAX_VALUE); // UnsupportedOperationException if target is not HDFS\n\n// after\nPath real = symlinkPath; // resolve manually if needed\nFileSystem targetFs = real.getFileSystem(conf);\nif (targetFs instanceof DistributedFileSystem) {\n  LocatedBlocks lbs = ((DistributedFileSystem) targetFs)\n      .getLocatedBlocks(real, 0, Long.MAX_VALUE);\n} else {\n  // handle non-HDFS target without block locations\n}","handlingStrategy":"type-guard","validationCode":"Path target = path;\nFileSystem targetFs = FileSystem.get(target.toUri(), conf);\n// follow one symlink hop if needed\nif (fs.getFileLinkStatus(path).isSymlink()) {\n  target = fs.getFileLinkStatus(path).getSymlink();\n  targetFs = FileSystem.get(target.toUri(), conf);\n}\nboolean isHdfs = targetFs instanceof DistributedFileSystem;","typeGuard":"private static boolean resolvesToHdfs(FileSystem fs, Path p) throws IOException {\n  FileSystem target = fs.getFileLinkStatus(p).isSymlink()\n      ? FileSystem.get(fs.getFileLinkStatus(p).getSymlink().toUri(), fs.getConf())\n      : fs;\n  return target instanceof org.apache.hadoop.hdfs.DistributedFileSystem;\n}","tryCatchPattern":"try {\n  return dfs.getLocatedBlocks(p, start, len);\n} catch (UnsupportedOperationException e) {\n  if (e.getMessage().contains(\"non-DistributedFileSystem\")) {\n    // degrade gracefully: no block locations for non-HDFS symlink targets\n    return Collections.emptyList();\n  }\n  throw e;\n}","preventionTips":["Before block-location or checksum calls, resolve symlinks and verify the target filesystem is DistributedFileSystem","Keep symlinks inside HDFS if tooling needs block-level metadata","Teach tools to skip or special-case non-HDFS symlink targets instead of assuming uniform HDFS layout"],"tags":["hdfs","symlink","unsupported-operation","block-location","file-system"],"backgroundTag":"symlink-crossing-filesystem","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}