apache/hadoop · error · IOException

The src snapshot={snapshotPath} already exists. The admin sh

Error message

The src snapshot={snapshotPath} already exists. The admin should delete the snapshot before submitting the initial distcp.

What it means

Second clause of pathCheckBeforeInitDistcp(): the snapshot fedbalance uses as its working 'current' snapshot (src/.snapshot/<CURRENT_SNAPSHOT_NAME>) must not already exist before the initial distcp is submitted. A leftover snapshot from an aborted earlier run blocks resubmission until an admin deletes it; the code would otherwise call allowSnapshot(src) and rely on creating that snapshot itself.

Source

Thrown at hadoop-tools/hadoop-federation-balance/src/main/java/org/apache/hadoop/tools/fedbalance/DistCpProcedure.java:465

        latestJob = localJob.getCluster().getJob(JobID.forName(jobId));
      } catch (InterruptedException e) {
        throw new IOException(e);
      }
      return latestJob == null ? null : new LocalJobStatus(latestJob);
    } else {
      return null;
    }
  }

  private void pathCheckBeforeInitDistcp() throws IOException {
    if (dstFs.exists(dst)) { // clean up.
      throw new IOException("The dst path=" + dst + " already exists. The admin"
          + " should delete it before submitting the initial distcp job.");
    }
    Path snapshotPath = new Path(src,
        HdfsConstants.DOT_SNAPSHOT_DIR_SEPARATOR + CURRENT_SNAPSHOT_NAME);
    if (srcFs.exists(snapshotPath)) {
      throw new IOException("The src snapshot=" + snapshotPath +
          " already exists. The admin should delete the snapshot before"
          + " submitting the initial distcp.");
    }
    srcFs.allowSnapshot(src);
  }

  /**
   * Submit distcp job and return jobId.
   */
  private String submitDistCpJob(String srcParam, String dstParam,
      boolean useSnapshotDiff) throws IOException {
    List<String> command = new ArrayList<>();
    command.addAll(Arrays
        .asList(new String[] {"-async", "-update", "-append", "-pruxgpcab"}));
    if (useSnapshotDiff) {
      command.add("-diff");
      command.add(LAST_SNAPSHOT_NAME);
      command.add(CURRENT_SNAPSHOT_NAME);

View on GitHub (pinned to 2add963021)

Solutions

1. List snapshots ('hdfs dfs -ls <src>/.snapshot') and delete the fedbalance current snapshot with 'hdfs dfsadmin -deleteSnapshot <src> <name>', then resubmit.
2. For a clean restart also remove dst and any fedbalance LAST snapshot on src, then re-run the job from the beginning.
3. Do not hand-create snapshots on src while a balance job is expected to run.

Example fix

hdfs dfs -ls hdfs://ns1/data/.snapshot
# before: fedbalance's current snapshot still there, job submit fails
hdfs dfsadmin -deleteSnapshot hdfs://ns1/data <CURRENT_SNAPSHOT_NAME>
# after: resubmit
hdfs fedbalance -src hdfs://ns1/data -dst hdfs://ns2/data

Path snap = new Path(src, ".snapshot/" /* + fedbalance CURRENT_SNAPSHOT_NAME */);
if (srcFs.exists(snap)) {
  LOG.warn("Leftover fedbalance snapshot {} blocks resubmission; delete it first", snap);
}

When it happens

Trigger: A previous fedbalance run created its CURRENT snapshot on src and died before finishing; on the next submission srcFs.exists(new Path(src, '.snapshot/' + CURRENT_SNAPSHOT_NAME)) is true and the IOException is thrown.

Common situations: Restarting a balance job after the client or scheduler crashed mid-procedure; killing a run between snapshot creation and completion.

Related errors

• {src} shouldn't enable snapshot.
• The dst path={dst} already exists. The admin should delete i
• {src} should be a directory.
• {dst} already exists.
• DistCp failed. jobId={jobId} failure={failureInfo}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6ba8cff8c3561cc1. Report an issue: GitHub.