{"record":{"id":"a95d8e9fffa735be","repo":"apache/hadoop","slug":"non-posix-bucket-append-is-not-supported-by-obsfi","errorCode":null,"errorMessage":"non-posix bucket. Append is not supported by OBSFileSystem","messagePattern":"non-posix bucket\\. Append is not supported by OBSFileSystem","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSFileSystem.java","lineNumber":786,"sourceCode":"   * @throws IOException io exception\n   */\n  @Override\n  @SuppressWarnings(\"checkstyle:parameternumber\")\n  public FSDataOutputStream create(\n      final Path f,\n      final FsPermission permission,\n      final EnumSet<CreateFlag> flags,\n      final int bufferSize,\n      final short replication,\n      final long blkSize,\n      final Progressable progress,\n      final ChecksumOpt checksumOpt)\n      throws IOException {\n    LOG.debug(\"create: Creating new file {}, flags:{}, isFsBucket:{}\", f,\n        flags, isFsBucket());\n    if (null != flags && flags.contains(CreateFlag.APPEND)) {\n      if (!isFsBucket()) {\n        throw new UnsupportedOperationException(\n            \"non-posix bucket. Append is not supported by \"\n                + \"OBSFileSystem\");\n      }\n      String key = OBSCommonUtils.pathToKey(this, f);\n      FileStatus status;\n      long objectLen = 0;\n      try {\n        // get the status or throw an FNFE\n        status = getFileStatus(f);\n        objectLen = status.getLen();\n        // if the thread reaches here, there is something at the path\n        if (status.isDirectory()) {\n          // path references a directory: automatic error\n          throw new FileAlreadyExistsException(f + \" is a directory\");\n        }\n      } catch (FileNotFoundException e) {\n        LOG.debug(\"FileNotFoundException, create: Creating new file {}\",\n            f);","sourceCodeStart":768,"sourceCodeEnd":804,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSFileSystem.java#L768-L804","documentation":"In OBSFileSystem.create with CreateFlags, if the caller passes CreateFlag.APPEND but the connector was mounted against a plain object (non-POSIX) OBS bucket, it throws UnsupportedOperationException('non-posix bucket. Append is not supported by OBSFileSystem'). Append semantics only exist for OBS POSIX buckets (isFsBucket()); the plain object-storage mode has no server-side append, so the client refuses up front. This is a capability mismatch between the requested operation and the bucket/filesystem mode.","triggerScenarios":"Calling create(..., EnumSet.of(CreateFlag.APPEND), ...) while fs.obs.bucket.type / the filesystem configuration does not indicate a POSIX bucket; running a framework (e.g. HBase WAL writers, some Iceberg/Hive setups) that requests APPEND against a standard object bucket; configs that worked on a POSIX-protocol endpoint pointed later at a plain OBS endpoint.","commonSituations":"Migrating workloads from HDFS or OBS POSIX buckets to plain OBS object buckets without removing APPEND flags; defaulting an application to append-mode output; provisioning new buckets without the POSIX feature enabled.","solutions":["Use a bucket with the OBS POSIX (fs) protocol enabled so isFsBucket() returns true, if append is a hard requirement","Otherwise switch the writer to create-with-overwrite or read-modify-rewrite (rewrite the object with appended content) — object storage has no append","Intercept APPEND in application code: detect this UnsupportedOperationException and fall back to non-appending write strategy","Verify fs.obs.* bucket-type configuration matches the actual bucket capability before deploying"],"exampleFix":"// before\nfs.create(path, FsPermission.getDefault(),\n    EnumSet.of(CreateFlag.APPEND), 4096, replication, blockSize, null); // non-POSIX bucket -> UnsupportedOperationException\n\n// after\nfs.create(path, FsPermission.getDefault(),\n    EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE), 4096, replication, blockSize, null);","handlingStrategy":"fallback","validationCode":"// capability check without touching files: try append on a temp key once at startup\nboolean appendSupported;\ntry {\n  Path probe = new Path(\"/tmp/.append-probe-\" + System.nanoTime());\n  try (FSDataOutputStream ignored = fs.create(probe, true)) { ignored.write(1); }\n  try (FSDataOutputStream ignored = fs.append(probe)) { }\n  fs.delete(probe, false);\n  appendSupported = true;\n} catch (UnsupportedOperationException e) {\n  appendSupported = false;\n}","typeGuard":null,"tryCatchPattern":"try {\n  fs.create(f, perm, EnumSet.of(CreateFlag.APPEND), bufSize, repl, blkSize, progress);\n} catch (UnsupportedOperationException e) {\n  if (String.valueOf(e.getMessage()).contains(\"Append is not supported\")) {\n    // fall back to non-appending strategy: CREATE/OVERWRITE or rewrite\n    fs.create(f, perm, EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE), bufSize, repl, blkSize, progress);\n  } else {\n    throw e;\n  }\n}","preventionTips":["Know your bucket protocol (POSIX vs object) before requesting APPEND flags","Gate append usage behind a startup capability probe","Keep HDFS-specific write modes out of shared connector configurations"],"tags":["append","unsupported-operation","bucket-capability","hadoop-obs"],"backgroundTag":"unsupported-append-on-object-store","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}