{"record":{"id":"ab5fc349ed9a7cf9","repo":"apache/beam","slug":"unable-to-rename-resource-s-to-s-as-source-not-found","errorCode":null,"errorMessage":"Unable to rename resource %s to %s as source not found.","messagePattern":"Unable to rename resource (.+?) to (.+?) as source not found\\.","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":404,"severity":"error","filePath":"sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java","lineNumber":280,"sourceCode":"      final org.apache.hadoop.fs.FileSystem fs = srcPath.getFileSystem(configuration);\n\n      // rename in HDFS requires the target directory to exist or silently fails (BEAM-4861)\n      mkdirs(destPath);\n\n      boolean success = fs.rename(srcPath, destPath);\n\n      // If the failure was due to the file already existing, delete and retry (BEAM-5036).\n      // This should be the exceptional case, so handle here rather than incur the overhead of\n      // testing first\n      if (!success && fs.exists(srcPath) && fs.exists(destPath)) {\n        LOG.debug(LOG_DELETING_EXISTING_FILE, Path.getPathWithoutSchemeAndAuthority(destPath));\n        fs.delete(destPath, false); // not recursive\n        success = fs.rename(srcPath, destPath);\n      }\n\n      if (!success) {\n        if (!fs.exists(srcPath)) {\n          throw new FileNotFoundException(\n              String.format(\n                  \"Unable to rename resource %s to %s as source not found.\", srcPath, destPath));\n\n        } else if (fs.exists(destPath)) {\n          throw new FileAlreadyExistsException(\n              String.format(\n                  \"Unable to rename resource %s to %s as destination already exists and couldn't be deleted.\",\n                  srcPath, destPath));\n\n        } else {\n          throw new IOException(\n              String.format(\n                  \"Unable to rename resource %s to %s. No further information provided by underlying filesystem.\",\n                  srcPath, destPath));\n        }\n      }\n    }\n  }","sourceCodeStart":262,"sourceCodeEnd":298,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java#L262-L298","documentation":"When Hadoop's fs.rename() returns false, rename() diagnoses the cause: if the source path no longer exists it throws FileNotFoundException with this message. Hadoop returns false instead of throwing, so Beam translates the boolean into a precise exception.","triggerScenarios":"Calling HadoopFileSystem.rename() with a srcResourceIds entry whose path does not exist in the underlying Hadoop filesystem, causing fs.rename to return false and fs.exists(srcPath) to be false.","commonSituations":"Renaming files produced by an upstream pipeline step that failed or was skipped; stale paths after a job rerun; typo'd or wrong-filesystem path.","solutions":["Verify the source path exists (fs.exists / HadoopFileSystem.match) before renaming.","Fix the path/scheme (e.g. hdfs:// vs local) used to construct the resource id.","Add retry/dependency ordering so the rename only runs after the producing step succeeds."],"exampleFix":"// before\nfileSystem.rename(ImmutableList.of(src), ImmutableList.of(dest));\n// after\nMatchResult match = fileSystem.match(src.toString());\nif (match.status() == Status.OK) {\n  fileSystem.rename(ImmutableList.of(src), ImmutableList.of(dest));\n}","handlingStrategy":"validation","validationCode":"MatchResult mr = fileSystem.match(src.toString());\nif (mr.status() != MatchResult.Status.OK) {\n  throw new FileNotFoundException(\"Source missing: \" + src);\n}","typeGuard":null,"tryCatchPattern":"try {\n  fileSystem.rename(srcs, dests);\n} catch (FileNotFoundException e) {\n  LOG.warn(\"Rename source missing, skipping: {}\", e.getMessage());\n}","preventionTips":["Match/verify source paths before renaming","Order pipeline steps so renames follow successful writes","Use consistent schemes/authorities when building ResourceIds"],"tags":["hadoop","hdfs","filenotfound","beam"],"backgroundTag":"file-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}