{"record":{"id":"57531689949e38c8","repo":"apache/hadoop","slug":"append-is-not-supported","errorCode":null,"errorMessage":"Append is not supported!","messagePattern":"Append is not supported!","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java","lineNumber":100,"sourceCode":"  private int blockOutputActiveBlocks;\n  private AliyunOSSFileSystemStore store;\n  private int maxKeys;\n  private int maxReadAheadPartNumber;\n  private int maxConcurrentCopyTasksPerDir;\n  private ExecutorService boundedThreadPool;\n  private ExecutorService boundedCopyThreadPool;\n\n  private static final PathFilter DEFAULT_FILTER = new PathFilter() {\n    @Override\n    public boolean accept(Path file) {\n      return true;\n    }\n  };\n\n  @Override\n  public FSDataOutputStream append(Path path, int bufferSize,\n      Progressable progress) throws IOException {\n    throw new IOException(\"Append is not supported!\");\n  }\n\n  @Override\n  public void close() throws IOException {\n    try {\n      store.close();\n      boundedThreadPool.shutdown();\n      boundedCopyThreadPool.shutdown();\n    } finally {\n      super.close();\n    }\n  }\n\n  @Override\n  public FSDataOutputStream create(Path path, FsPermission permission,\n      boolean overwrite, int bufferSize, short replication, long blockSize,\n      Progressable progress) throws IOException {\n    String key = pathToKey(path);","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java#L82-L118","documentation":"AliyunOSSFileSystem.append() unconditionally throws IOException(\"Append is not supported!\"). Object stores have no server-side append: OSS objects are immutable once written, so the Hadoop append contract cannot be implemented. The connector fails fast for any code path that requests an append-mode output stream.","triggerScenarios":"Calling FileSystem.append(path), FSDataOutputStream appends, or any engine operation that maps to append (e.g., Hive table appends writing to existing object files, HBase WAL-style appends, some StreamingFileSink configurations).","commonSituations":"Porting jobs written for HDFS that use append or lazy persist writes; Hive INSERT INTO tables whose writer attempts to append to existing files instead of creating new ones; frameworks defaulting to append semantics when the scheme is oss://.","solutions":["Write to a new file (create with overwrite as appropriate) instead of appending; design outputs as new objects per write","For Hive, rely on ACID/new-file semantics so INSERT INTO creates new files rather than appending","If you truly need append semantics, use HDFS or a filesystem that supports it, or buffer externally and rewrite the object atomically"],"exampleFix":"// before\nFSDataOutputStream out = fs.append(path); // throws\n\n// after\nFSDataOutputStream out = fs.create(path, /*overwrite*/ true);","handlingStrategy":"validation","validationCode":"// probe once at startup\nif (fs instanceof AliyunOSSFileSystem) { /* appends unavailable: use create() paths only */ }","typeGuard":null,"tryCatchPattern":"catch (IOException e) { if (e.getMessage().contains(\"Append is not supported\")) { throw new UnsupportedOperationException(\"oss:// output must use create(), not append()\", e); } throw e; }","preventionTips":["Design outputs as new objects per write; never rely on append semantics on object stores","Check FileSystem support with fs.hasPathCapability(path, \"append\") style probes before choosing an append-based writer"],"tags":["aliyun-oss","unsupported-operation","append","hadoop-connector"],"backgroundTag":"append-not-supported-on-object-store","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}