{"record":{"id":"a96775567e0aaf8c","repo":"apache/hadoop","slug":"the-dst-path-dst-already-exists-the-admin-shoul","errorCode":null,"errorMessage":"The dst path={dst} already exists. The admin should delete it before submitting the initial distcp job.","messagePattern":"The dst path=(.+?) already exists\\. The admin should delete it before submitting the initial distcp job\\.","errorType":"validation","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-federation-balance/src/main/java/org/apache/hadoop/tools/fedbalance/DistCpProcedure.java","lineNumber":459,"sourceCode":"  }\n\n  private LocalJobStatus getCurrentLocalJob() throws IOException {\n    if (localJob != null) {\n      Job latestJob;\n      try {\n        latestJob = localJob.getCluster().getJob(JobID.forName(jobId));\n      } catch (InterruptedException e) {\n        throw new IOException(e);\n      }\n      return latestJob == null ? null : new LocalJobStatus(latestJob);\n    } else {\n      return null;\n    }\n  }\n\n  private void pathCheckBeforeInitDistcp() throws IOException {\n    if (dstFs.exists(dst)) { // clean up.\n      throw new IOException(\"The dst path=\" + dst + \" already exists. The admin\"\n          + \" should delete it before submitting the initial distcp job.\");\n    }\n    Path snapshotPath = new Path(src,\n        HdfsConstants.DOT_SNAPSHOT_DIR_SEPARATOR + CURRENT_SNAPSHOT_NAME);\n    if (srcFs.exists(snapshotPath)) {\n      throw new IOException(\"The src snapshot=\" + snapshotPath +\n          \" already exists. The admin should delete the snapshot before\"\n          + \" submitting the initial distcp.\");\n    }\n    srcFs.allowSnapshot(src);\n  }\n\n  /**\n   * Submit distcp job and return jobId.\n   */\n  private String submitDistCpJob(String srcParam, String dstParam,\n      boolean useSnapshotDiff) throws IOException {\n    List<String> command = new ArrayList<>();","sourceCodeStart":441,"sourceCodeEnd":477,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-federation-balance/src/main/java/org/apache/hadoop/tools/fedbalance/DistCpProcedure.java#L441-L477","documentation":"pathCheckBeforeInitDistcp() runs when (re)submitting the initial distcp: it requires dst to be absent and then immediately enables snapshots on src. The message names the admin action - delete dst - because in practice dst is a stale tree left by a previous failed or killed balance run that never got cleaned. Distinguish from error 5042, which is the preCheck at job setup.","triggerScenarios":"Re-running a fedbalance job after a prior attempt already created dst (the scheduler journal recovers into INIT_DISTCP and finds dstFs.exists(dst) true); scheduler recovery replaying a job whose output survived.","commonSituations":"A first balance attempt failed mid-copy and the operator re-submits without cleanup; killing the fedbalance client/process and starting over; journal recovery after a scheduler restart.","solutions":["Delete the leftover destination tree ('hdfs dfs -rm -r <dst>'), then submit the initial distcp job again.","If the leftover dst holds data you need, rename it aside instead of deleting.","When cancelling a balance job, use the tool's cancel path so journal state and dst are cleaned consistently instead of kill -9."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Before (re)submitting the initial distcp job\nPath dst = context.getDst();\nif (dst.getFileSystem(conf).exists(dst)) {\n  LOG.warn(\"Leftover dst {} from a prior run; delete it before resubmitting\", dst);\n  // stop and require operator cleanup, or delete if policy allows\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Wrap fedbalance job restarts in a cleanup step that removes dst.","Cancel balance jobs via the tool's cancel path so journal and output stay consistent."],"tags":["fedbalance","federation-balance","destination-exists","job-recovery","distcp"],"backgroundTag":"destination-path-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}