apache/hadoop · error · PathIOException

Must specify either create or overwrite

Error message

Must specify either create or overwrite

What it means

Error "Must specify either create or overwrite" thrown in apache/hadoop.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/CreateFileBuilder.java:167

      String key = option.getKey();
      if (key.startsWith(headerPrefix) && key.length() > prefixLen) {
        headers.put(key.substring(prefixLen), option.getValue());
      }
    }

    // and add the mimetype
    if (options.get(FS_OPTION_CREATE_CONTENT_TYPE, null) != null)  {
      headers.put(CONTENT_TYPE, options.get(FS_OPTION_CREATE_CONTENT_TYPE, null));
    }

    EnumSet<CreateFlag> flags = getFlags();
    if (flags.contains(CreateFlag.APPEND)) {
      throw new UnsupportedOperationException("Append is not supported");
    }

    if (!flags.contains(CreateFlag.CREATE) &&
        !flags.contains(CreateFlag.OVERWRITE)) {
      throw new PathIOException(path.toString(),
          "Must specify either create or overwrite");
    }

    // build the other switches
    if (isRecursive()) {
      createFileSwitches.add(Recursive);
    }
    if (Performance.isEnabled(options)) {
      createFileSwitches.add(Performance);
    }
    if (CreateMultipart.isEnabled(options)) {
      createFileSwitches.add(CreateMultipart);
    }
    if (ConditionalOverwrite.isEnabled(options)) {
      createFileSwitches.add(ConditionalOverwrite);
    }
    // etag is a string so is checked for then extracted.
    final String etag = options.get(FS_OPTION_CREATE_CONDITIONAL_OVERWRITE_ETAG, null);

View on GitHub (pinned to 2add963021)

Solutions

1. Set either create(true) or overwrite(true) on the create file builder.

When it happens

Trigger: Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/CreateFileBuilder.java:167 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/014140f924bfd8db. Report an issue: GitHub.