{"record":{"id":"7bb35f63b8295549","repo":"apache/beam","slug":"unable-to-copy-resource-s-to-s-no-further-information","errorCode":null,"errorMessage":"Unable to copy resource %s to %s. No further information provided by underlying filesystem.","messagePattern":"Unable to copy resource (.+?) to (.+?)\\. No further information provided by underlying filesystem\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java","lineNumber":220,"sourceCode":"      // to use the inefficient implementation found in FileUtil which copies all the bytes through\n      // the local machine.\n      //\n      // HDFS FileSystem does define a concat method but could only find the DFSFileSystem\n      // implementing it. The DFSFileSystem implemented concat by deleting the srcs after which\n      // is not what we want. Also, all the other FileSystem implementations I saw threw\n      // UnsupportedOperationException within concat.\n      final boolean success =\n          FileUtil.copy(\n              fs,\n              srcResourceIds.get(i).toPath(),\n              fs,\n              destResourceIds.get(i).toPath(),\n              false,\n              true,\n              fs.getConf());\n      if (!success) {\n        // Defensive coding as this should not happen in practice\n        throw new IOException(\n            String.format(\n                \"Unable to copy resource %s to %s. No further information provided by underlying filesystem.\",\n                srcResourceIds.get(i).toPath(), destResourceIds.get(i).toPath()));\n      }\n    }\n  }\n\n  /**\n   * Renames a {@link List} of file-like resources from one location to another.\n   *\n   * <p>The number of source resources must equal the number of destination resources. Destination\n   * resources will be created recursively.\n   *\n   * @param srcResourceIds the references of the source resources\n   * @param destResourceIds the references of the destination resources\n   * @throws FileNotFoundException if the source resources are missing. When rename throws, the\n   *     state of the resources is unknown but safe: for every (source, destination) pair of\n   *     resources, the following are possible: a) source exists, b) destination exists, c) source","sourceCodeStart":202,"sourceCodeEnd":238,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java#L202-L238","documentation":"HadoopFileSystem.copy() throws this IOException when FileSystem.rename/copy calls report boolean failure without an exception — a defensive branch that should not occur in practice. The Hadoop API only returns false, giving no underlying reason.","triggerScenarios":"Calling HadoopFileSystem.copy(OpenMode, srcResourceIds, destResourceIds) where the underlying Hadoop FileSystem copy/rename returns false for some src/dest pair (e.g. missing source, permission issue, or dest filesystem error).","commonSituations":"HDFS NameNode issues, missing source files after a failed earlier stage, or quota/permission problems on the destination path where Hadoop swallows the cause and just returns false.","solutions":["Verify each source resource exists and is readable before copying.","Check HDFS permissions and quota on both source and destination paths.","Inspect Hadoop client/NameNode logs for the suppressed cause."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"for (ResourceId src : srcs) {\n  if (fs.match(src.toString()).status() != MatchResult.Status.OK) {\n    throw new FileNotFoundException(src.toString());\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  fileSystem.copy(mode, srcs, dests);\n} catch (IOException e) {\n  // check source existence & HDFS health, then retry with backoff\n  retryCopy(srcs, dests);\n}","preventionTips":["Verify sources exist before copy","Pre-create/verify destination directories","Watch HDFS NameNode health and quota before bulk copies"],"tags":["io","hadoop","hdfs","filesystem"],"backgroundTag":"file-write-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}