apache/beam · error · UnsupportedOperationException

Support for move options is not yet implemented.

Error message

Support for move options is not yet implemented.

What it means

HadoopFileSystem.rename() only supports renaming with no MoveOptions; passing any MoveOptions value throws UnsupportedOperationException. Move-option semantics (like atomic moves or skip-if-exists) are not implemented for Hadoop-backed file systems in Beam.

Solutions

  1. Call rename() with no move options.
  2. Implement the desired option semantics manually (e.g. check dest existence before renaming).
  3. Use a different FileSystem implementation that supports MoveOptions if the option is essential.

Example fix

// before
fs.rename(src, dest, MoveOptions.SKIP_IF_DESTINATION_EXISTS);
// after
fs.rename(src, dest);
Defensive patterns

Strategy: validation

Validate before calling

if (moveOptions != null && moveOptions.length > 0) {
  throw new IllegalArgumentException("HadoopFileSystem does not support move options");
}

Try / catch

try {
  fileSystem.rename(srcs, dests, moveOptions);
} catch (UnsupportedOperationException e) {
  fileSystem.rename(srcs, dests); // retry without options
}

Prevention

When it happens

Trigger: Calling HadoopFileSystem.rename(src, dest, moveOptions...) with one or more MoveOptions values, e.g. MoveOptions.SKIP_IF_DESTINATION_EXISTS.

Common situations: Code written generically against Beam's FileSystem API that passes move options and works on other backends but hits the Hadoop implementation.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/26e91a640616a9e3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-file-system/src/main/java/org/apache/beam/sdk/io/hdfs/HadoopFileSystem.java:254

   * @throws FileNotFoundException if the source resources are missing. When rename throws, the
   *     state of the resources is unknown but safe: for every (source, destination) pair of
   *     resources, the following are possible: a) source exists, b) destination exists, c) source
   *     and destination both exist. Thus no data is lost, however, duplicated resource are
   *     possible. In such scenarios, callers can use {@code match()} to determine the state of the
   *     resource.
   * @throws FileAlreadyExistsException if a target resource already exists and couldn't be
   *     overwritten.
   * @throws IOException if the underlying filesystem indicates the rename was not performed but no
   *     other errors were thrown.
   */
  @Override
  protected void rename(
      List<HadoopResourceId> srcResourceIds,
      List<HadoopResourceId> destResourceIds,
      MoveOptions... moveOptions)
      throws IOException {
    if (moveOptions.length > 0) {
      throw new UnsupportedOperationException("Support for move options is not yet implemented.");
    }
    for (int i = 0; i < srcResourceIds.size(); ++i) {

      final Path srcPath = srcResourceIds.get(i).toPath();
      final Path destPath = destResourceIds.get(i).toPath();

      // this enforces src and dest file systems to match
      final org.apache.hadoop.fs.FileSystem fs = srcPath.getFileSystem(configuration);

      // rename in HDFS requires the target directory to exist or silently fails (BEAM-4861)
      mkdirs(destPath);

      boolean success = fs.rename(srcPath, destPath);

      // If the failure was due to the file already existing, delete and retry (BEAM-5036).
      // This should be the exceptional case, so handle here rather than incur the overhead of
      // testing first
      if (!success && fs.exists(srcPath) && fs.exists(destPath)) {

View on GitHub (pinned to 12126d8942)