apache/hadoop · error · NotAppendableException

%s is not appendable because append non-existed object with

Error message

%s is not appendable because append non-existed object with zero byte is not supported.

What it means

In HNS (directory-bucket) mode, TOS.append routes to hnsAppend. TOS forbids appending to a non-existent object, and a zero-byte append cannot pre-create one (the code only pre-creates via putObject when contentLength > 0), so append(key, empty) on a missing key throws NotAppendableException ("<key> is not appendable because append non-existed object with zero byte is not supported."), a RuntimeException documented on ObjectStorage.append.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/tos/TOS.java:478

  @Override
  public byte[] append(String key, InputStreamProvider streamProvider, long contentLength) {
    if (bucketInfo.isDirectory()) {
      return hnsAppend(key, streamProvider, contentLength);
    } else {
      return fnsAppend(key, streamProvider, contentLength);
    }
  }

  private byte[] hnsAppend(String key, InputStreamProvider streamProvider, long contentLength) {
    checkAvailableClient();

    long offset = 0;
    String preCrc64;

    TosObjectInfo obj = innerHead(key);
    if (obj == null) {
      if (contentLength == 0) {
        throw new NotAppendableException(String.format(
            "%s is not appendable because append non-existed object with "
                + "zero byte is not supported.", key));
      }

      // In HNS, append non-existed object is not allowed. Pre-create an empty object before
      // performing appendObject.
      PutObjectOutput res = client.put(bucket, key, () -> EMPTY_STREAM, 0, defaultAcl);
      preCrc64 = res.getHashCrc64ecma();
    } else {
      if (contentLength == 0) {
        return obj.checksum();
      }
      offset = obj.size();
      preCrc64 = obj.crc64ecma();
    }

    AppendObjectOutput res =
        client.appendObject(bucket, key, streamProvider, offset, contentLength, preCrc64,

View on GitHub (pinned to 2add963021)

Solutions

Skip the append when contentLength == 0 and the object doesn't exist; create the marker with put(key, emptyStream, 0) instead
head(key) first: if null and the payload is empty, avoid calling append
Buffer data so every append carries at least one byte
If the file must exist, write it through the normal create path rather than append

Example fix

// before
storage.append(key, new byte[0]);

// after
if (storage.head(key) == null && data.length == 0) {
  storage.put(key, () -> new ByteArrayInputStream(data), 0); // create empty marker
  return;
}
storage.append(key, data);

Defensive patterns

Strategy: validation

Validate before calling

if (storage.head(key) == null && data.length == 0) {
  // zero-byte append cannot create an object in HNS mode; use put instead
  storage.put(key, () -> new ByteArrayInputStream(data), 0);
  return;
}
storage.append(key, data);

Try / catch

try {
  storage.append(key, data);
} catch (NotAppendableException e) {
  if (data.length == 0 && storage.head(key) == null) {
    storage.put(key, () -> new ByteArrayInputStream(data), 0);
  } else {
    throw e;
  }
}

Prevention

Never append zero bytes to a non-existent key
Create empty markers with put(), not append()
Buffer writes so appends always carry bytes

When it happens

Trigger: append(key, new byte[0]) on a key that does not exist in an HNS/directory bucket; opening an append stream and closing it without writing any bytes; jobs that 'create' empty files purely via append.

Common situations: Jobs creating empty markers then appending later; manifest/commit writers opening streams for zero-length files; behavior differences between FNS (general-purpose) and HNS directory buckets.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8986b90871f37eee. Report an issue: GitHub.