apache/hadoop · error · IllegalArgumentException

block list path must already exist on remote fs!

Error message

block list path must already exist on remote fs!

What it means

Error "block list path must already exist on remote fs!" thrown in apache/hadoop.

Source

Thrown at hadoop-tools/hadoop-dynamometer/hadoop-dynamometer-infra/src/main/java/org/apache/hadoop/tools/dynamometer/Client.java:471

    } else {
      this.hadoopBinary = DynoInfraUtils.fetchHadoopTarball(
          new File(".").getAbsoluteFile(),
          commandLine.getOptionValue(HADOOP_VERSION_ARG), getConf(), LOG)
          .toString();
    }
    this.amOptions = AMOptions.initFromParser(commandLine);
    this.clientTimeout = Integer
        .parseInt(commandLine.getOptionValue(TIMEOUT_ARG, TIMEOUT_DEFAULT));
    this.tokenFileLocation = commandLine.
        getOptionValue(TOKEN_FILE_LOCATION_ARG);

    amOptions.verify();

    Path blockPath = new Path(blockListPath);
    FileSystem blockListFS = blockPath.getFileSystem(getConf());
    if (blockListFS.getUri().equals(FileSystem.getLocal(getConf()).getUri())
        || !blockListFS.exists(blockPath)) {
      throw new IllegalArgumentException(
          "block list path must already exist on remote fs!");
    }
    numTotalDataNodes = blockListFS.listStatus(blockPath,
        DynoConstants.BLOCK_LIST_FILE_FILTER).length;

    if (commandLine.hasOption(WORKLOAD_REPLAY_ENABLE_ARG)) {
      if (!commandLine.hasOption(WORKLOAD_INPUT_PATH_ARG)
          || !commandLine.hasOption(WORKLOAD_START_DELAY_ARG)) {
        throw new IllegalArgumentException("workload_replay_enable was "
            + "specified; must include all required workload_ parameters.");
      }
      launchWorkloadJob = true;
      workloadInputPath = commandLine.getOptionValue(WORKLOAD_INPUT_PATH_ARG);
      workloadOutputPath = commandLine.getOptionValue(WORKLOAD_OUTPUT_PATH_ARG);
      workloadThreadsPerMapper = Integer
          .parseInt(commandLine.getOptionValue(WORKLOAD_THREADS_PER_MAPPER_ARG,
              String.valueOf(AuditReplayMapper.NUM_THREADS_DEFAULT)));
      workloadRateFactor = Double.parseDouble(commandLine.getOptionValue(

View on GitHub (pinned to 2add963021)

Solutions

1. Upload the block list file to the remote filesystem first, or correct the --block_list_path to an existing path.

When it happens

Trigger: The workload block list path given to the dynamometer Client does not exist on the remote filesystem (e.g. HDFS), so the workload input cannot be staged.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/293cce142035b0b9. Report an issue: GitHub.