apache/hadoop · error · ExitUtil.ExitException

55

55

Error message

S3 Select is no longer supported

What it means

The 'select' subcommand of the S3Guard tool throws immediately with exit code 55 (EXIT_UNSUPPORTED_VERSION). AWS retired the S3 Select service and Hadoop removed the S3A Select implementation; the switch case for SelectConstants.NAME exists solely to fail fast with 'S3 Select is no longer supported' instead of dispatching into deleted code.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3GuardTool.java:992

    if (UNSUPPORTED_COMMANDS.contains(subCommand)) {
      throw s3guardUnsupported();
    }
    switch (subCommand) {

    case BucketInfo.NAME:
      command = new BucketInfo(conf);
      break;
    case BucketTool.NAME:
      command = new BucketTool(conf);
      break;
    case MarkerTool.MARKERS:
      command = new MarkerTool(conf);
      break;
    case Uploads.NAME:
      command = new Uploads(conf);
      break;
    case SelectConstants.NAME:
      throw new ExitUtil.ExitException(
          EXIT_UNSUPPORTED_VERSION, SELECT_UNSUPPORTED);
    default:
      printHelp();
      throw new ExitUtil.ExitException(E_USAGE,
          "Unknown command " + subCommand);
    }
    try {
      return ToolRunner.run(conf, command, otherArgs);
    } finally {
      IOUtils.cleanupWithLogger(LOG, command);
    }
  }

  /**
   * Main entry point. Calls {@code System.exit()} on all execution paths.
   * @param args argument list
   */
  public static void main(String[] args) {

View on GitHub (pinned to 2add963021)

Solutions

1. Remove the select subcommand from scripts and read whole objects through the normal S3A input stream, filtering rows in the compute engine
2. For server-side querying, run Amazon Athena (or another query engine) and write results back to S3, then read them with S3A
3. Grep code and configuration for 's3guard select', 'fs.s3a.select' and 'SelectQuerySpec' to find every removed usage
4. Note that pinning to old Hadoop does not help long term - AWS retired the underlying service itself, so all usages must move off S3 Select

Example fix

# before
hadoop aws s3guard select -query "SELECT * FROM S3Object s" s3a://bucket/data.csv
# after: read the object and filter client-side
hadoop fs -text s3a://bucket/data.csv | grep 'pattern'

String sub = firstRemainingArg(conf, args);
if (SelectConstants.NAME.equals(sub)) {
  throw new IllegalArgumentException("S3 Select usage removed - use plain S3A reads or Athena");
}

try {
  int rc = S3GuardTool.run(conf, args);
} catch (ExitUtil.ExitException e) {
  if (e.getExitCode() == 55) {
    // EXIT_UNSUPPORTED_VERSION: permanently removed feature, do not retry
  }
}

When it happens

Trigger: Running 'hadoop aws s3guard select ...'; any script or management tool that passes 'select' as the subcommand to S3GuardTool.run; jobs migrated from Hadoop 3.3.x or earlier that used S3 Select push-down (fs.s3a.select.* configuration, SelectQuerySpec-based readers).

Common situations: Upgrading a cluster from Hadoop 3.3.x where S3 Select CSV/JSON reads worked; AWS's retirement of S3 Select breaking pipelines that pushed SQL predicates into S3A reads; legacy Spark/Hive jobs or distcp-style scripts still invoking the select command.

Related errors

• Cannot seek to a negative offset <targetPos>
• <uri>: Stream is closed!
• End of file reached before reading fully.
• End of stream reached draining data between ranges; expected
• HTTP stream closed before all bytes were read. Expected %,d

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d61da98aa911202d. Report an issue: GitHub.