What causes "Buffer size must be greater than 0" in apache/hadoop?

Set io.file.buffer.size (or your key) to a positive value, e.g. 4096

apache/hadoop · error · IllegalArgumentException

Buffer size must be greater than 0

Error message

Buffer size must be greater than 0

What it means

Options.CreateOpts.BufferSize validates that the stream buffer size passed to FileSystem.create(path, CreateOpts.bufferSize(bs), ...) is a positive int; bs <= 0 throws IllegalArgumentException from the constructor.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Options.java:103

    }
    
    public static class ReplicationFactor extends CreateOpts {
      private final short replication;
      protected ReplicationFactor(short rf) { 
        if (rf <= 0) {
          throw new IllegalArgumentException(
                      "Replication must be greater than 0");
        }
        replication = rf;
      }
      public short getValue() { return replication; }
    }
    
    public static class BufferSize extends CreateOpts {
      private final int bufferSize;
      protected BufferSize(int bs) {
        if (bs <= 0) {
          throw new IllegalArgumentException(
                        "Buffer size must be greater than 0");
        }
        bufferSize = bs; 
      }
      public int getValue() { return bufferSize; }
    }

    /** This is not needed if ChecksumParam is specified. **/
    public static class BytesPerChecksum extends CreateOpts {
      private final int bytesPerChecksum;
      protected BytesPerChecksum(short bpc) { 
        if (bpc <= 0) {
          throw new IllegalArgumentException(
                        "Bytes per checksum must be greater than 0");
        }
        bytesPerChecksum = bpc; 
      }
      public int getValue() { return bytesPerChecksum; }

View on GitHub (pinned to 2add963021)

Solutions

Set io.file.buffer.size (or your key) to a positive value, e.g. 4096
Clamp to a sane floor before constructing the option: Math.max(1, n)
Validate all size inputs before building CreateOpts

Example fix

// before
CreateOpts.bufferSize(conf.getInt("io.file.buffer.size", 0))
// after
int buf = Math.max(1, conf.getInt("io.file.buffer.size", 4096));
CreateOpts.bufferSize(buf);

Defensive patterns

Strategy: validation

Validate before calling

int buf = Math.max(1, conf.getInt("io.file.buffer.size", 4096));
if (buf <= 0) throw new IllegalArgumentException("buffer size must be > 0");
fs.create(out, CreateOpts.bufferSize(buf));

Prevention

Keep io.file.buffer.size positive in config
Clamp computed sizes to a floor of 1
Validate size inputs before constructing CreateOpts

When it happens

Trigger: CreateOpts.bufferSize(n) with n <= 0 — typically io.file.buffer.size misconfigured to 0, a size computation underflowing, or an unvalidated user setting forwarded directly.

Common situations: Bad buffer-size config defaults, environment-specific overrides (small containers) set to 0, unit bugs (KB vs bytes miscalculation to zero).

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b4e80664d2e357e8. Report an issue: GitHub.