apache/hadoop · error · IllegalArgumentException

Unsupported block buffer "{}"

Error message

Unsupported block buffer "{}"

What it means

Thrown by OBSDataBlocks.createFactory when initializing the fast-upload buffering mechanism: the value of fs.obs.fast.upload.buffer does not match any of the three supported block buffer types. The connector only accepts 'array' (heap byte[] blocks), 'disk' (files on local disk), or 'bytebuffer' (off-heap NIO ByteBuffers), so any other string aborts FileSystem initialization with an IllegalArgumentException. It surfaces at initialize()/new OBSFileSystem() time, before any I/O happens.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSDataBlocks.java:98

  /**
   * Create a factory.
   *
   * @param owner factory owner
   * @param name  factory name -the option from {@link OBSConstants}.
   * @return the factory, ready to be initialized.
   * @throws IllegalArgumentException if the name is unknown.
   */
  static BlockFactory createFactory(final OBSFileSystem owner,
      final String name) {
    switch (name) {
    case OBSConstants.FAST_UPLOAD_BUFFER_ARRAY:
      return new ByteArrayBlockFactory(owner);
    case OBSConstants.FAST_UPLOAD_BUFFER_DISK:
      return new DiskBlockFactory(owner);
    case OBSConstants.FAST_UPLOAD_BYTEBUFFER:
      return new ByteBufferBlockFactory(owner);
    default:
      throw new IllegalArgumentException(
          "Unsupported block buffer" + " \"" + name + '"');
    }
  }

  /**
   * Base class for block factories.
   */
  abstract static class BlockFactory {
    /**
     * OBS file system type.
     */
    private final OBSFileSystem owner;

    protected BlockFactory(final OBSFileSystem obsFileSystem) {
      this.owner = obsFileSystem;
    }

    /**

View on GitHub (pinned to 2add963021)

Solutions

1. Set fs.obs.fast.upload.buffer to exactly one of: array, disk, or bytebuffer (lowercase, no quotes/whitespace) in core-site.xml or the Configuration used to instantiate the filesystem
2. If migrating from S3A, re-check every fs.obs.fast.upload.* property against OBSConstants/OBS documentation instead of copying fs.s3a.* values verbatim
3. Remove the property entirely to fall back to the documented default (disk buffering) rather than leaving an invalid string
4. Add a startup assertion or integration test that mounts the OBS filesystem with the production configuration so bad values fail in CI, not at runtime

Example fix

<!-- before -->
<property><name>fs.obs.fast.upload.buffer</name><value>arrays</value></property>

<!-- after -->
<property><name>fs.obs.fast.upload.buffer</name><value>array</value></property>

import java.util.Set;
import org.apache.hadoop.conf.Configuration;

Set<String> VALID = Set.of("array", "disk", "bytebuffer");

void checkBufferConfig(Configuration conf) {
  String v = conf.get("fs.obs.fast.upload.buffer", "disk");
  if (!VALID.contains(v)) {
    throw new IllegalArgumentException(
        "fs.obs.fast.upload.buffer must be one of " + VALID + ", got: '" + v + "'");
  }
}

try {
  FileSystem fs = path.getFileSystem(conf);
} catch (IllegalArgumentException e) {
  if (e.getMessage() != null && e.getMessage().contains("Unsupported block buffer")) {
    // fix fs.obs.fast.upload.buffer, then re-create the FileSystem
  }
  throw e;
}

When it happens

Trigger: Setting fs.obs.fast.upload.buffer to a misspelled or unsupported value (e.g. 'arrays', 'ByteBuffer', 'memory', 'heap', or an S3A-style value like 'bytebuffer' with different casing). Also happens when the property is empty/whitespace after someone comments out the intended value, because createFactory is invoked with the raw configured string during OBSFileSystem.initialize.

Common situations: Copying S3A configuration (fs.s3a.fast.upload.buffer) into the OBS keys and assuming the same vocabulary; upgrading connector versions where the accepted set changed; XML config typos or trailing spaces in core-site.xml; automated templating tools substituting an empty variable into the property.

Related errors

• The trash feature(fs.obs.trash.enable) is enabled, but the c
• Cannot seek to a negative offset
• Requested more bytes than destination buffer size: request l
• Cannot seek to a negative offset {}
• Unsupported comparator: {comparator}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c2cd376f5d32afdb. Report an issue: GitHub.