apache/hadoop · error · IOException

Not supported

Error message

Not supported

What it means

RawFileSystem.append() unconditionally throws IOException("Not supported"). TOS (like S3 and most object stores) stores immutable objects: bytes cannot be appended to an existing object in place, so the Hadoop append API cannot be implemented. The connector fails fast rather than silently corrupting data.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java:232

  @Override
  public FSDataOutputStream createNonRecursive(
      Path path,
      FsPermission permission,
      EnumSet<CreateFlag> flag,
      int bufferSize,
      short replication,
      long blockSize,
      Progressable progress) throws IOException {
    Path qualified = makeQualified(path);
    return create(qualified, permission, flag.contains(CreateFlag.OVERWRITE),
        bufferSize, replication, blockSize, progress);
  }

  @Override
  public FSDataOutputStream append(Path f, int bufferSize, Progressable progress)
      throws IOException {
    throw new IOException("Not supported");
  }

  /**
   * Rename src path to dest path, if dest path is an existed dir,
   * then FS will rename the src path under the dst dir.
   * E.g. rename('/a/b', '/a/c') and dest 'c' is an existed dir,
   * then the source path '/a/b' will be renamed with dest path '/a/b/c' internally.
   *
   * <ul>
   *   <li>Return false if src doesn't exist</li>
   *   <li>Return false if src is root</li>
   *   <li>Return false if dst path is under src path, e.g. rename('/a/b', '/a/b/c')</li>
   *   <li>Return false if dst path already exists</li>
   *   <li>Return true if rename('/a/b', '/a/b') and 'b' is an existed file</li>
   *   <li>Return true if rename('/a/b', '/a') and 'a' is an existed dir,
   *   fs will rename '/a/b' to '/a/b' internally</li>
   *   <li>Return false if rename('/a/b', '/a/b') and 'b' is an existed dir,
   *   because fs will try to rename '/a/b' to '/a/b/b', which is under '/a/b', this behavior

View on GitHub (pinned to 2add963021)

Solutions

1. Do not use append on tos://: write each chunk to a new object (unique part name) and merge/partition at read time
2. Move append-dependent components (e.g. HBase WAL) to HDFS and keep only bulk data on TOS
3. Probe capability before calling: fs.hasPathCapability(path, CommonPathCapabilities.APPEND) and skip the append path when false
4. For log aggregation, use timestamped/rotated object keys instead of one growing file

Example fix

// before
try (FSDataOutputStream out = fs.append(path, bufferSize, null)) { out.write(data); }

// after: write a new object per chunk
Path part = new Path(path.getParent(), path.getName() + "-" + UUID.randomUUID());
try (FSDataOutputStream out = fs.create(part, true)) { out.write(data); }

if (fs.hasPathCapability(path, "append")) {
  out = fs.append(path, bufferSize, null);
} else {
  // tos://: rotate to a new object key per write
}

try { out = fs.append(path, bufferSize, null); }
catch (IOException e) {
  if ("Not supported".equals(e.getMessage())) { /* rotate object keys instead */ }
  else throw e;
}

When it happens

Trigger: Calling fs.append(f, bufferSize, progress) on any tos:// path; frameworks that probe or require append (HBase WAL placement, some log writers, append-based streaming sinks) invoking append on the tos:// FileSystem.

Common situations: Pointing a workload that requires append (HBase, node logs, legacy FileOutputCommitter assumptions) at a tos:// root instead of HDFS; engines that try append-then-fallback when reopening an output file.

Related errors

• %s is not appendable because append non-existed object with
• Append is not supported by S3AFileSystem
• non-posix bucket. Append is not supported by OBSFileSystem
• {} already exists
• File not found %s

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/151d07da89e81de2. Report an issue: GitHub.