apache/hadoop · error · HadoopIllegalArgumentException

Unkown CreateOpts of type {iOpt.getClass().getName()}

Error message

Unkown CreateOpts of type {iOpt.getClass().getName()}

What it means

The option-scanning loop in AbstractFileSystem.create only recognizes the eight built-in CreateOpts types (BlockSize, BufferSize, ReplicationFactor, BytesPerChecksum, ChecksumParam, Perms, Progress, CreateParent); any other CreateOpts instance falls to the else-branch and throws HadoopIllegalArgumentException("Unkown CreateOpts of type ...") — the typo "Unkown" is in the source. Custom CreateOpts subclasses are therefore not a supported extension point here.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java:595

        if (permission != null) {
          throw new HadoopIllegalArgumentException(
              "Perms option is set multiple times");
        }
        permission = ((CreateOpts.Perms) iOpt).getValue();
      } else if (CreateOpts.Progress.class.isInstance(iOpt)) {
        if (progress != null) {
          throw new HadoopIllegalArgumentException(
              "Progress option is set multiple times");
        }
        progress = ((CreateOpts.Progress) iOpt).getValue();
      } else if (CreateOpts.CreateParent.class.isInstance(iOpt)) {
        if (createParent != null) {
          throw new HadoopIllegalArgumentException(
              "CreateParent option is set multiple times");
        }
        createParent = ((CreateOpts.CreateParent) iOpt).getValue();
      } else {
        throw new HadoopIllegalArgumentException("Unkown CreateOpts of type " +
            iOpt.getClass().getName());
      }
    }
    if (permission == null) {
      throw new HadoopIllegalArgumentException("no permission supplied");
    }


    FsServerDefaults ssDef = getServerDefaults(f);
    if (ssDef.getBlockSize() % ssDef.getBytesPerChecksum() != 0) {
      throw new IOException("Internal error: default blockSize is" + 
          " not a multiple of default bytesPerChecksum ");
    }
    
    if (blockSize == -1) {
      blockSize = ssDef.getBlockSize();
    }

View on GitHub (pinned to 2add963021)

Solutions

  1. Remove the custom CreateOpts and convey the setting via Configuration keys or FSDataOutputStreamBuilder instead
  2. If your own AbstractFileSystem needs extra create parameters, add them to your createInternal signature/your own API rather than the shared opts varargs
  3. Align hadoop module versions on the classpath so option classes match the loop in use

Example fix

// before
class MyOpt extends Options.CreateOpts { ... }
fc.create(f, flag, CreateOpts.perms(perm), new MyOpt()); // Unkown CreateOpts of type ...

// after
conf.setBoolean("myfs.create.fast", true);
fc = FileContext.getFileContext(myFsUri, conf);
fc.create(f, flag, CreateOpts.perms(perm));
Defensive patterns

Strategy: try-catch

Validate before calling

for (Options.CreateOpts o : opts) {
  if (!(o instanceof Options.CreateOpts.BlockSize || o instanceof Options.CreateOpts.BufferSize
      || o instanceof Options.CreateOpts.ReplicationFactor || o instanceof Options.CreateOpts.BytesPerChecksum
      || o instanceof Options.CreateOpts.ChecksumParam || o instanceof Options.CreateOpts.Perms
      || o instanceof Options.CreateOpts.Progress || o instanceof Options.CreateOpts.CreateParent)) {
    throw new IllegalArgumentException("Unsupported CreateOpts: " + o.getClass().getName());
  }
}

Try / catch

try { fc.create(f, flag, opts); } catch (HadoopIllegalArgumentException e) { if (e.getMessage().contains("CreateOpts of type")) { /* strip custom opt / fix versions */ } else throw e; }

Prevention

When it happens

Trigger: Defining your own class extending Options.CreateOpts and passing it into AbstractFileSystem.create or FileContext.create; a newer/other jar on the classpath injecting an option class this version's loop has no branch for.

Common situations: Custom filesystem frameworks trying to piggyback extra parameters on CreateOpts; version-skewed hadoop jars on a client classpath; copy of an option class under a different package name.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1487bcb1c830834e. Report an issue: GitHub.